💻 computer science

Occupancy-Grounded Room Segmentation for Hierarchical 3D Scene Graphs

This paper introduces an occupancy-grounded pipeline for constructing hierarchical 3D scene graphs that anchors room nodes to tracked free-space regions with explicit polygonal footprints, demonstrating superior room instance recovery compared to state-of-the-art place-connectivity baselines on Matterport3D scenes despite a trade-off in precision.

Original authors: Carlos Cueto Zumaya, Iacopo Catalano, Jorge Peña-Queralta, Wallace Moreira Bessa

Published 2026-06-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Carlos Cueto Zumaya, Iacopo Catalano, Jorge Peña-Queralta, Wallace Moreira Bessa

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a robot trying to understand the layout of a house. To do this, it builds a mental map called a 3D Scene Graph. Think of this graph as a family tree for the house: at the bottom, it knows about individual objects (a chair, a lamp); in the middle, it knows about the floor space it can walk on; and at the top, it needs to understand the "rooms" (the kitchen, the bedroom).

The problem with most robots today is that their "room" layer is a bit fuzzy. Some robots just guess, "Hey, these chairs are close together, so they must be in a room." Others look at the walls. But because they all guess differently, it's hard to tell if the robot actually understands where the rooms are or if it's just making things up.

The New Approach: "The Floor Plan Detective"

The authors of this paper propose a new way to build that top layer of the map. Instead of guessing based on where objects are, they anchor the rooms to free space—the actual empty floor the robot can walk on.

Here is how their system works, using a simple analogy:

The 3D Scan (The Raw Data): The robot scans the room with a camera that sees depth (like a 3D eye). It builds a giant 3D block of data, like a digital version of a cloud of dust.
Flattening the Cloud (The 2D Map): The robot ignores the ceiling and the tops of tall bookshelves. It looks straight down and asks, "Is there enough room for me to walk here?" It turns that 3D cloud into a flat, 2D map of just the walkable floor.
Cutting the Pizza (Decomposition): Now, imagine this flat map is a giant pizza. The robot uses a special algorithm (called DUDE) to slice the pizza into distinct pieces. It looks for natural "bottlenecks," like doorways or narrow hallways, to decide where one room ends and another begins.
Anchoring the Rooms: Every time the robot slices off a piece of the "pizza," it says, "This piece is a Room." It gives that room a specific, drawn-out shape (a polygon) on the floor.
The Family Tree: Finally, it attaches the objects (chairs, tables) and the robot's own location to these specific room shapes.

The Big Test: Did It Work?

The researchers tested this on 12 different virtual houses (from a dataset called Matterport3D). They compared their new method against a top-tier robot system called Hydra.

The Goal: To see if the robot could correctly identify and count the actual rooms in the house.
The Result:
- Finding More Rooms: The new method was much better at finding rooms. If there were 10 rooms in a house, the new method found about 4, while the old method (Hydra) only found 1 or 2. It was much better at "remembering" that a room exists.
- The Trade-off: However, the new method wasn't perfect at drawing the exact walls. Sometimes it drew a room that was a little too big or included a hallway that shouldn't have been there. The old method was very careful and precise, but it was so careful it often missed entire rooms.

The "Wall" Problem

The paper admits a major limitation: The walls are still hard to get right.

Even though the robot is great at finding the space inside a room, it struggles to draw the exact boundary where the room ends and the next one begins. If the robot's initial 3D scan has a small gap or a mistake, the "pizza slicing" step might merge two separate rooms into one giant room, or split one room into two. The robot is limited by how good its initial map is.

The Bottom Line

This paper introduces a way to make robot maps more honest about what a "room" is. Instead of guessing based on furniture, it builds rooms based on the actual empty floor space.

Pros: It finds way more rooms than previous methods.
Cons: It sometimes draws the room boundaries a bit loosely, and it still struggles to get the walls perfectly accurate.

The authors conclude that while we are getting better at finding rooms, getting the exact shape of every room right is still a puzzle that hasn't been fully solved yet. They didn't test this on real-world tasks like "cleaning the kitchen" or "finding a lost cat," so we don't know if this helps with those specific jobs yet; they only tested how well the robot can draw the map.

Technical Summary: Occupancy-Grounded Room Segmentation for Hierarchical 3D Scene Graphs

Problem Statement
Hierarchical 3D Scene Graphs (3DSGs) are essential for indoor robots to organize geometric and semantic information across spatial scales, linking object-level perception to room-scale reasoning. However, the construction of the "room layer" in existing systems lacks a standardized geometric definition. Current approaches derive room nodes from disparate spatial substrates, such as place-connectivity clusters, wall planes, or segmentation outputs. Consequently, room nodes are not evaluated against a common geometric criterion, making it difficult to determine if a predicted room node actually corresponds to a physical room instance in the environment. Furthermore, existing evaluations often rely on indirect metrics (e.g., trajectory accuracy) rather than direct geometric comparisons to ground-truth room layouts.

Methodology
The authors propose an occupancy-grounded 3DSG pipeline that anchors room nodes to tracked free-space regions derived from occupancy decomposition. The pipeline processes RGB-D input through four main stages:

Geometric Perception: RGB-D frames are integrated into a 3D occupancy octree map. This map is collapsed into a 2D free-space layer via vertical-clearance analysis. A ground cell is marked as "free" only if its vertical voxel column contains a gap of at least 0.3 m, effectively preserving architectural boundaries (walls) while suppressing low or overhead clutter.
Region Grounding (Core Novelty): The 2D free-space layer is partitioned into polygonal regions using an incremental form of DUDE (Decomposition of Unstructured Discrete Environments). These regions are tracked over time using Intersection-over-Union (IoU) association to ensure stability against mapping noise. Crucially, each stable tracked region is anchored to exactly one room node, providing every room with an explicit, persistent polygonal footprint.
Graph Construction: The system assembles a hierarchical graph containing agent, object, navigation, and room nodes. Objects and navigation nodes are assigned to room nodes based on polygon containment. Room adjacency is established only when supporting regions are connected by a navigable crossing.
Evaluation Framework: Unlike previous methods, this approach allows the room layer to be evaluated as a standard 2D room segmentation task. Predicted room polygons are matched one-to-one with annotated ground-truth room instances using Hungarian assignment based on IoU.

Key Contributions

Explicit Geometric Anchoring: The paper introduces a method where room nodes are not abstract clusters but are explicitly grounded to tracked free-space polygons, giving them a measurable spatial extent.
Standardized Evaluation: It applies 2D room segmentation metrics (Recall, Precision, F1, mIoU, Boundary F1) to 3DSG room layers for the first time, enabling direct comparison of room instance recovery against ground truth.
Pipeline Implementation: A complete pipeline is provided that fuses geometric perception, free-space decomposition, and region tracking to generate a hierarchical 3DSG.

Results
The method was evaluated on 12 Matterport3D scenes and compared against Hydra, a state-of-the-art place-connectivity baseline.

Instance Recovery: The occupancy-grounded approach significantly outperformed Hydra in Recall (0.379 vs. 0.152) and F1 score (0.427 vs. 0.241). On average, the proposed method recovered nearly five ground-truth rooms per scene, compared to fewer than two for Hydra.
Precision Trade-off: The proposed method exhibited lower Precision (0.518 vs. 0.703). The authors attribute this to unmatched room nodes whose region polygons were too poorly bounded to satisfy strict IoU matching criteria, often due to boundary fidelity issues.
Failure Modes: Qualitative analysis revealed that while Hydra suffers from "collapse" (merging distinct rooms into single nodes due to dense connectivity), the proposed method's failures are often local, stemming from the free-space decomposition merging non-convex layouts or narrow connections. Additionally, boundary accuracy is limited by errors in the underlying occupancy map reconstruction.
Ablation: Varying the DUDE threshold ( $d_{thr}$ ) demonstrated a trade-off between decomposition granularity and recall; coarser thresholds reduced the number of predicted rooms and recall significantly.

Significance and Claims
The paper claims that occupancy-grounded anchoring provides a measurable and comparable way to evaluate room-level reasoning in 3DSGs. By exposing the room layer as explicit polygons, the authors demonstrate that it is possible to recover substantially more room instances than place-connectivity baselines.

However, the authors maintain a modest tone regarding the ultimate success of the approach. They explicitly state that room-instance recovery and boundary-accurate room footprints remain open problems. The results highlight a fundamental coverage-precision trade-off: while the method recovers more rooms, it struggles with precise boundary delineation. The paper concludes that the quality of the room layer is ultimately bounded by the quality of the input occupancy map and the granularity of the free-space decomposition, rather than the anchoring strategy alone.

The New Approach: "The Floor Plan Detective"

The Big Test: Did It Work?

The "Wall" Problem

The Bottom Line

Technical Summary: Occupancy-Grounded Room Segmentation for Hierarchical 3D Scene Graphs

More like this