Getting Screen Focal Point From HUD | Jeylon Guidi | 02/14/2025

The Problem:

One of the features planned for our game is the ability to use energy blasts, which requires some aiming elements to be introduce into our project. Specifically one element of aiming that is useful as a developer, is knowing the in world position that the players screen is focusing on.  This information can help with implementing certain aim based mechanics, such as aiming precise range attacks, targeting interactable objects, or even just debugging issues pertaining to where the player is trying to aim. 

While at a glance getting the players focal position seems like a simple and straightforward concept, however as I will explain the process is a lot more convoluted than it initially seems. It involved a lot of math conversions from 2D to 3D space, which can be a difficult concept to grasp. But once you understand how these conversions work and how they are applied, it becomes understandable process.

The Solution:

 The way I went about solving the problem involves using the players in game HUD to determine important information related to where their screen exists in world space. In Unreal Engine User Widget classes (the class of HUD elements) have a thing called a CachedGeometry, which provides 2D geometric/position information related to specific HUD class/element. What I did was create an invisible crosshair HUD element at the center of the players HUD, which would be used as a starting point for the needed calculations. Using this crosshairs CachedGeometry allowed me to get its 2D position, which I was able to convert to a screen space position with the use of a function called LocalToViewport. This function converts the 2D information of the cached geometry to a screen position. 

With the crosshair position converted into screen space, I then had to convert it to world space with the use of a function called DeprojectScreenPositionToWorld. What this function does is calculate a 3D world position and direction based on the 2D screen space position provided. With this done I now had a position in work space representing the crosshairs screen position. 

Using this 3D position as a starting point I would draw a line out into space based on the direction that was also returned in the deproject function. In Unreal you can create a line trace, that allows you to account for collisions that may occur when tracing the line. When a collision occurs it produces a hit result, which contains a lot of valuable information about said collision. One piece of information it contains is the 3D world position where the collision occurred. 

In other words, this collision position is the physical location that the crosshair is aiming at.