Generation of High-Level Concepts in 3D Scene Graphs via Autoregressive Diffusion
This paper proposes a unified autoregressive diffusion-based model that jointly learns graph structure and spatial features to generate complete, hierarchical Indoor 3D Scene Graphs from observed geometric primitives, outperforming existing baselines across diverse datasets and introducing a novel Fused Gromov-Wasserstein metric for principled evaluation.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Robots moving through the inside of a building face a fundamental challenge: they see the world as a chaotic collection of raw data points. A laser scanner might detect thousands of flat surfaces—walls, ceilings, and floors—but to a machine, these are just geometric shapes without meaning. To navigate effectively, a robot needs to understand that these shapes form a room, that rooms make up a floor, and that floors belong to a building. This mental map, known as a 3D Scene Graph, acts as a bridge between the raw sensory input and the high-level concepts humans use to describe their surroundings. For years, researchers have struggled to teach robots how to build these maps automatically. Traditional methods relied on rigid, hand-written rules that could only recognize simple shapes like rectangular rooms, while newer learning-based approaches often required separate systems to figure out the structure and the location of objects, making it difficult to scale up to complex, multi-story environments.
A team of researchers has now introduced a new approach that allows robots to construct these complex, multi-level maps from the ground up, starting with the basic walls they detect and working their way up to entire buildings and cities. Instead of using separate tools to guess the layout and then place the rooms, their system uses a single, unified process that learns to generate the entire hierarchy at once. The method works by taking the initial set of flat surfaces a robot sees and progressively adding new layers of meaning. It decides how many new elements, such as a new room or a new floor, should be added, determines what those elements are called, and calculates exactly where they should be located in 3D space. This process happens step-by-step, with the system refining its guesses until the full map is complete.
The researchers tested this system on a wide variety of environments, including computer-generated synthetic scenes, real architectural floor plans, and actual data recorded by robots equipped with laser sensors. They compared their new method against existing techniques that rely on fixed rules or separate models for different parts of the task. The results showed that their unified approach consistently produced more accurate and complete maps than the previous methods. It successfully handled complex layouts that were not perfectly rectangular and could generate maps that extended all the way up to the city level, a task that earlier learning-based systems could not perform. In fact, on the most complex datasets involving entire buildings and cities, their system outperformed even a powerful one-shot model that was given a secret advantage: knowing the exact number of rooms and floors in advance before it started generating the map.
One of the most significant aspects of this work is how it handles the uncertainty of the real world. In a typical scenario, a robot does not know how many rooms or floors exist in a building before it starts exploring. Older methods often struggled with this because they required the final size of the map to be known beforehand. The new system, however, learns to decide when the map is finished on its own. It continues to add new layers of structure until it determines that no more information is needed, effectively figuring out the scale of the environment as it builds the map. This capability makes the technology much more practical for real-world robotics, where the size and complexity of a building are rarely known in advance.
To ensure their system was truly working, the researchers developed a new way to measure success. Instead of just checking if the robot got the number of rooms right, they created a metric that evaluates how well the generated map matches the real one in terms of shape, location, and the relationships between different parts. This new measurement tool, which combines geometric distance with structural similarity, confirmed that the maps produced by their system were significantly closer to the truth than those produced by other methods. The team also made their data and code publicly available, providing a foundation for other scientists to build upon this work. By demonstrating that a single, unified model can learn to generate complex, multi-level spatial hierarchies, this research offers a promising path toward robots that can truly understand and navigate the intricate structures of the human world.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.