Texture streaming system. Screenshot of a 3D render showing airplanes and terrain with high resolution textures
Using high resolution textures or many unique textures in a 3D scene has a big impact on the quality, detail and sharpness of the final rendered images. To generate these images, real-time 3D engines need to load the texture data into the video memory on the graphics card or game console.
The textures are read from disk (hard drive or Blu-ray disk) and copied into the video memory when needed. The straightforward approach, which has been used for many years, is to load all the texture data at the start of an application or game.
This approach has two major downsides
To get around this last limitation, developers have been breaking up their game into levels that are loaded separately. However, the same issues arise with these levels. Because of the memory limitation, increasing the size of the virtual environment leads to less texture density, or, in other words, blurrier textures.
Texture streaming is the process of continuously loading texture data in the background while running an application or playing a game. The streaming process is hidden from the end-user (no loading screens). A texture streaming system allows more textures to be used in a seemingly continuous 3D environment than the amount that would fit into video memory.
Earlier systems split the 3D environment into many separate areas. The system would detect when to load the data for these areas based on a set of parameters or using a manually set trigger point. Although this enables large open world games, a number of these area data chunks need to be loaded in video memory at one given time. This again limits the amount of textures or the texture resolution that can be used.
More advanced systems aim to have a finer loading granularity. The smaller the data chunks, the more that memory can be optimized to increase the texture density while keeping memory usage constant.
Mipmap streaming systems
Mipmap streaming systems automatically load individual texture mipmaps based on the distance of the virtual camera from a visible object. Keeping only the mipmaps in memory that are actually needed allows memory usage to be minimized while having many objects in view in the distance. However, this approach can potentially result in many disk seeks at a given time. Mipmap streaming also cannot handle high resolution textures (8Kx8K and higher) well.
With increasing texture resolution, you’re also adding increasingly larger mipmaps to the textures which have to be loaded as a single chunk and thus introduce high latency. Such systems also have difficulties with many objects close to the camera. Assuming an object with three 8K textures (BC5/7 compressed), then suddenly, when the object gets close enough, 192MB of data needs to be read from disk and uploaded to the video memory. This results in long disk locks, blurry textures and popping artifacts.
Finally, the most advanced class of texture streaming systems subdivide each mipmap into small texture tiles. Granite SDK is such a tile-based texture streaming system. The system will automatically load only the texture tiles that are actually visible. In many cases only parts of mipmaps become visible at one given time.
Combining tile-based streaming with virtual texturing Granite combines a tile-based streaming system with virtual texturing techniques to minimize the amount of video memory needed. Tiles are loaded into a GPU cache instead of conventional texture resources. As a result, the memory usage is completely decoupled from the amount of textures in the scene and the resolution of these textures. For example, increasing all the textures from 2K to 4K in a scene will need the same amount of memory.
The memory usage depends mostly on the output screen resolution and the amount of channels per texel. These system properties allow for very dense 3D scenes with many unique textures or a high texel density per object. It also allows importing very large textures up to 262.144 x 262.144 pixels.