Photogrammetry in games screenshot
Photogrammetry is the science of making measurements from photographs. More specifically, stereophotogrammetry allows estimating the 3D surface of objects from two or more photographs, each taken from a different position.
In the context of video games, photogrammetry can lead to ultra-detailed textures. But trying to use these textures at the highest resolution can collide with the technical limitations imposed by modern hardware and real-time 3D engine software. An advanced texture streaming system can overcome these limitations.
This science of photogrammetry has led to a set of algorithms that are useful for the 3D scanning of physical objects and environments. When providing enough overlapping photographs that cover all the sides of an object, the algorithms can reconstruct a detailed digital 3D representation of that object. First, these algorithms generate a 3D point cloud from the aligned photographic images.
From this point cloud, a 3D mesh is generated for efficient rendering in real-time. In the last stage, the original images are projected back on the mesh to get ultra-detailed and realistic textures. To get the best results, high quality DSL cameras are normally used.
The film industry has been using photogrammetry for years to realize highly realistic special effects. It has also been used in video games, although mainly for characters. Recently, a number of game developers started to use photogrammetry to scan objects and entire environments as a way to get highly detailed 3D worlds. An elaborate article can be found here.
When using photogrammetry, developers end up with ultra-detailed, high resolution textures. Depending on the workflow and tools used, these textures can be, for example, huge 32K by 32K images, dozens of independent smaller textures (2K or 4K), or UDIM textures with dozens of 4K or 8K patches. However, game engines are not designed to handle dense scenes with many high resolution textures.
This results in high video memory usage and long game loading times. Also, modern game engines cannot import large textures or textures in the UDIM format.
Granite SDK, a tile-based texture streaming system
Granite SDK is a tile-based texture streaming system that is focused on very dense scenes. The middleware integrates into any 3D or game engine. It is designed to handle the high pixel densities that result from using photogrammetry.
Such a system handles 3D scenes with high density unique textures with ease. It also limits video memory usage to 1GB or less by employing virtual texturing techniques. An additional benefit is the reduction of noticeable loading times to practically zero. Granite prevents downscaling of textures that were acquired using photogrammetry due to hardware limitations.
And by importing any kind of texture format and texture size, it allows the best tool and workflow for the job to be used, without losing time converting formats. More specifically, Granite can import very large image files or a set of image patches. It also supports importing UDIM textures for use in a real-time game engine.