In the early days of the internet, the only way to listen to a recording or watch a movie was to download the whole file in one go. Pioneering companies like RealAudio created the first large-scale internet radio stations that took advantage of emerging streaming audio formats. Later, YouTube and Netflix built extensive empires on the back of streaming video protocols.

Today, 3D games, digital twins and the metaverse are primarily delivered via large files. As a result, these experiences are restricted to a single playing field or building. 3D Tiles, an evolving Open Geospatial Consortium standard, promises to help stream and scale the metaverse. Eventually, this could empower the next wave of metaverse startups in the same way that streaming media enabled YouTube, Netflix, Spotify, Disney+ and hundreds of other new media empires.

3D Tiles is an open standard for massive, heterogeneous 3D geospatial datasets such as point clouds, buildings, photogrammetry and vector data. It is built on top of glTF and other 3D data types. Whereas standards like glTF compress and optimize 3D assets for runtime efficiency and sharing, 3D Tiles takes that to the global scale by creating a spatial index of 3D content. The standard is widely used in the geospatial community and is gaining more traction in 3D games, digital twins and the industrial metaverse. 

The 3D Tiles specification was first introduced in 2015 and standardized in 2019. It got a significant update last year with the introduction of 3D Tiles Next, which improves 3D analytics, can query 3D data more efficiently and improves support for contextual data. For example, Cesium is currently working with some large construction enterprises to analyze how the 3D terrain changes at a large site over time.

From 3D objects to worlds

VentureBeat caught up with Patrick Cozzi, CEO of Cesium, who conceived the idea for 3D Tiles. His team at Cesium was using glTF for individual 3D models like satellites, ground vehicles and aircraft, but there was no good way to efficiently share a collection of them. So, he began exploring ways to allow incremental streaming of the data.

“We realized that we needed to be able to transfer these massive models with terabytes of terrain, point clouds at centimeter resolution, with the geometry, textures and metadata across the web for efficient visualization and analysis,” Cozzi explained.

For example, a 3D Tiles-enabled app could deliver a view of a large city like Los Angeles, starting with the street and nearby buildings in high resolution, with progressively lower resolution for the buildings and landscape in the distance because they take up less screen space.

Building on GIS tiles

Cozzi took inspiration from related approaches for streaming GIS data using 2D tiles. These techniques are widely used in apps like Google maps that allow you to zoom from the edge of the earth to an individual house. But 3D presented additional challenges. Today, apps like Google Maps Street View constrain you to hopping between points. Apps built on the 3D Tiles will allow you to walk smoothly along the road without downloading the whole world first.

The significant innovation was using hierarchical level of detail to show the highest resolution for things nearby and incrementally lower resolution in the distance. The same approach can provide a similar experience for seamlessly scrolling and scaling through a world for both 2D and 3D.

“We want to be able to stream the most accurate model but with the least amount of data transfer in the form of geometry, textures and metadata,” Cozzi said.

In this case, the geometry is data about the triangles used to describe the physical representation of the world. The textures represent the world’s colors, reflections and other visual properties. The metadata provides additional context for indicating which pixels are part of a window, door or solar panel, and information about their properties.

“The last category of metadata is essential for digital twins so that you can interact with the visualization, do analytics or create more accurate simulations,” Cozzi explained. For example, it could help you model RF propagation in a city, estimate solar capacity or count the number of pools.

One big challenge is efficiently parsing a large model into multiple representations at different scales. 3D Tiles builds a hierarchy that includes full-resolution source data and progressively lower-resolution versions. However, each version takes advantage of compression baked into glTF, dramatically reducing the file size. “Even though you have to store multiple levels of resolution, often the 3D Tileset can be smaller than the source data,” Cozzi said.

The next level

The 3D Tiles community has recently released 3D Tiles Next, which is currently going through the OGC standards process. One significant improvement is more efficient random access. This promises to make it easier to query 3D Tiles data for artificial intelligence (AI) and analytics use cases, such as counting the number of solar panels and total window area near a particular point. It also provides the ability to connect metadata to individual pixels. The interoperability with glTF has also been improved.

Down the road, Cozzi hopes to explore ways to improve how 3D Tiles can bring massive scale to USD models, improve support for more game engines, and unlock new 3D AI capabilities. Game engines are increasingly integrated into GIS, digital twins and industrial metaverse tools.

“I think supporting many different game engines is really important to bring massive-scale 3D geospatial to as many people as possible,” Cozzi said.