Optimize game graphics with Variable Rate Shading

At Arm we like to think about video games a lot. Games are created with many different goals in mind - creative, artistic, technical, educational and many others. What is common in all cases is that they are there to entertain. For a variety of reasons, including performance, developers may rely on illusions to create games that are more immersive and entertaining. In this blog, we are going to highlight Variable Rate Shading (VRS) as a tool that can be used to continue the developer quest for “more” immersion and entertainment in games, while maintaining performance and efficiency targets.

Not all pixels are equal - game design and human congnition

Humans typically benefit from binocular vision that grants us a field of view of 120 degrees in the horizontal plane. We have about 210 degrees of forward vision when we consider regions that only one of our eyes can see. Not every portion of this region is equal in terms of clarity. For example, the region most dense in cone cells that allows us to perceive objects in colour only extends two degrees. Outside the foveal region, our vision is dominated by cone cells that are sensitive to motion and not light. The impression that we have complete clarity of vision is a cognitive illusion.

Figure 1: Human vision

These factors can be exploited in real-time graphics and games. If we can make an educated guess about where the player’s attention will be focused (e.g. in a racing game) or even know precisely where it is (e.g. in VR headsets that have pupil tracking sensors), then we can combine that information with human limitations. This conspires with our own brains to recognise that not all pixels are equal and do not require the same level of investment.

There are a variety of other aspects in graphical and application design that allow us to come to similar conclusions. If an object lies in a region of deep shadow, it is known to be occluded by other objects (e.g. UI, particle effects or motion blur) or contains details that only vary slowly, then it is tempting to think about how these factors could be exploited in rendering scenes. This sets the context for VRS and how it helps games.

What is Variable Rate Shading

Firstly, what is VRS? Consider a traditional render pipeline and the process of executing vertex and fragment shaders. Vertex shaders operate on streams of geometry, calculating geometrical and material characteristics that can be interpolated to covered pixels. These can later be presented for fragment shaders, determining the pixel colours that should be presented on the screen. Somewhere in-between, we may also run compute jobs on the scene to calculate buffers of information, such as lighting data that be used in the process of composing a frame for presentation.

Figure 2: The stages of the graphics pipleine

All of the above are the responsibility of the GPU, with VRS being the technique that can be used to control the shading rate of the fragment shaders at work within a scene. Allowing for the high resolution of modern screens, for the reasons outlined above, it is interesting to consider ways to regulate the amount of investment made in executing fragment shader jobs.

VRS does not have an impact on rasterization or blending rates, rather it offers more granular controls on blocks of pixels setting rates from 1x1 to 4x4 (and all power-of-two combinations in-between). The developer can choose to apply VRS to whole draw calls, individual primitives or to specific tiles in the frame buffer, using a screen-space shading rate attachment. Operations on draw calls are called “tier 1” VRS, whereas per primitive and per tile techniques are referred to as “tier 2” VRS. The reader may be able to relate the examples given in the previous section to different tiers of VRS.

Figure 3: Regular Shading vs. Variable Rate Shading

Demonstrating the potential of Variable Rate Shading

At Vulkanised 2023, we presented findings based on a study of tier 2 VRS and the application of per tile shading rate attachments to lighting. Using the familiar Sponza reference, indirect diffuse, indirect specular and light shading were optimized by VRS.

Overall, we can claim that when applied with consideration , as befits its purpose. In our studies, we saw a 12 percent reduction in GPU cycles working with 8x8 tiles and max shading rate of 2x2. For transparency, we should mention that this technique increased read bandwidth consumed by about three percent and reduced write bandwidth by around one percent.

Conclusions

VRS is a technique that is supported by Vulkan, and DirectX and used to regulate the expense of fragment shading. This is useful when graphical workloads present bottlenecks in the fragment processing. If you are thinking about the content within graphical scenes and how it is presented to users, the adoption of VRS could be a sensible approach in your applications. Arm introduced support for VRS in Immortalis-G715, Mali-G715 and Mali-G615, making it accessible to developers worldwide. We will continue to support VRS and look forward to playing the increasing number of game titles that employ it.

Learn more about Variable Rate Shading