💻 computer science

Fixed External Cameras as Common Prior Maps for Active 3D Scene Graph Generation

This paper presents a hardware-agnostic, RGB-only framework that utilizes fixed external cameras as Common Prior Maps to bootstrap and guide active, incremental 3D scene graph generation, significantly improving initial object recall and exploration efficiency by fusing wide-field external views with onboard robot observations.

Original authors: Giorgia Modi, Davide Buoso, Giuseppe Averta, Daniele De Martini

Published 2026-05-19

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Giorgia Modi, Davide Buoso, Giuseppe Averta, Daniele De Martini

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a robot sent into a new house to clean or deliver a package. Usually, the robot has to start from scratch: it wanders around, bumps into things, and slowly builds a mental map of where the sofa, the fridge, and the doors are. This is slow and inefficient.

This paper proposes a clever shortcut: give the robot a "cheat sheet" before it even starts moving.

Here is how the system works, explained through simple analogies:

1. The "Cheat Sheet" (Common Prior Maps)

Most buildings already have security cameras or fixed monitoring cameras on the walls. The authors say, "Let's use those!"

The Analogy: Imagine you are trying to solve a jigsaw puzzle in a dark room. Usually, you have to feel around for every piece. But what if someone handed you a picture of the finished puzzle before you started? You wouldn't know exactly where every single piece is yet, but you'd know the big picture: "The sky is blue, the tree is in the middle, and the house is on the right."
In the Paper: These fixed cameras act as that "finished picture." They provide a wide view of the room (a "Common Prior Map") that tells the robot, "Hey, there's a table here and a chair there," before the robot takes its first step.

2. The "Magic Eye" (RGB-Only Vision)

Robots usually need special depth sensors (like laser scanners) to understand 3D space. This paper removes that requirement.

The Analogy: Think of how a human can look at a 2D photo of a room and guess how far away the couch is based on shadows and perspective. The system uses a smart AI model (called MapAnything) that does the same thing. It looks at standard 2D photos from the security cameras and the robot's own camera and "imagines" the 3D shape of the room.
The Benefit: The robot doesn't need expensive, heavy hardware. It can use the same "eyes" (standard cameras) for both the security cameras on the wall and its own eyes.

3. The "Mental Organizer" (3D Scene Graph)

Instead of just making a blurry 3D map, the robot builds a structured list of relationships.

The Analogy: A regular map is like a photo of a messy desk. A Scene Graph is like a to-do list that says: "The lamp is on top of the desk, and the desk is next to the window."
In the Paper: The system creates a network of objects (nodes) and their relationships (edges). It knows not just what is there, but how things relate to each other.

4. The "Smart Explorer" (Active Exploration)

Once the robot has its "cheat sheet" and its "mental organizer," it doesn't just wander randomly. It plays a game of "Where am I missing information?"

The Analogy: Imagine you are playing a game of "20 Questions" with a friend who is hiding a toy. If you already know the toy is in the "kitchen" (thanks to your cheat sheet), you don't waste time asking, "Is it in the garage?" You go straight to the kitchen to find the specific spot.
In the Paper: The robot looks at its current map, sees what it doesn't know (like "I see a door, but I don't know what's behind it"), and uses a smart algorithm to decide exactly where to move next to fill in those gaps.

What Did They Find?

The researchers tested this in computer simulations of apartments and offices.

The Result: When the robot started with just its own camera, it found about 47% of the objects after 30 steps.
The Boost: When they gave it just one fixed security camera as a "cheat sheet" before it started, it immediately knew about 79% more objects right from the start.
The Takeaway: Even a single, old security camera can act as a powerful head-start, helping the robot understand the room much faster and more accurately than if it had to start from zero.

In summary: This paper shows that we don't need to build expensive new robots to understand rooms better. We can just use the cameras we already have hanging on the walls to give robots a "head start," making them smarter and more efficient from the very first second they turn on.

Technical Summary: Fixed External Cameras as Common Prior Maps for Active 3D Scene Graph Generation

Problem Statement

Modern robotic systems operating in unstructured human environments (e.g., homes, hospitals) require rich contextual visual understanding to perceive, decide, and act autonomously. Traditional perception pipelines often fail to provide this, producing either low-level dense metric maps lacking semantic content or object-centric outputs that miss relational structure. While 3D Scene Graphs (3DSGs) offer a structured representation capturing entities, their 3D locations, and inter-object relationships, constructing them robustly in real-world scenarios remains challenging, particularly when prior knowledge is incomplete.

Furthermore, existing Active Semantic Perception (ASP) systems typically rely on egocentric (onboard) sensors and often depend on depth data (RGB-D) or proprietary Large Language Models (LLMs) for relationship inference. There is a lack of frameworks that can seamlessly integrate fixed external camera infrastructure—already deployed in many buildings for surveillance—as a source of prior knowledge to bootstrap scene understanding, specifically in an RGB-only setting.

Methodology

The authors propose an RGB-only framework for active, incremental 3D scene graph generation that treats observations from fixed external cameras as Common Prior Maps (CPMs). The system operates within a unified, hardware-agnostic pipeline that processes both onboard robot cameras and external fixed cameras identically.

System Architecture

The system functions as an incremental perception–action loop:

Initialization (t=0): RGB observations from one or more fixed external cameras serve as a CPM. These are processed to initialize a point cloud and a 3DSG, providing semantic and geometric structure before the robot moves.
Active Exploration (t>0): The robot acquires new RGB images from its current viewpoint. The same pipeline updates the point cloud and scene graph. An active exploration module then selects the Next-Best-View (NBV) to maximize information gain, guiding the robot to regions of high semantic uncertainty.

Core Components

1. RGB-Only 3DSG Generation

Geometry Estimation: The pipeline utilizes MapAnything [8] to infer scene geometry from RGB images. In a single forward pass, the model predicts per-view ray directions, up-to-scale depth maps, camera poses, and a global metric scale factor. This allows the system to back-project 2D pixels into a metrically scaled 3D point cloud without physical depth sensors.
Scene Graph Construction: The system adapts ConceptGraphs [12]. It uses SAM [21] for instance masks and CLIP [22] for semantic embeddings. Crucially, instead of querying proprietary LLMs for relationships, the authors employ a deterministic, geometry-only procedure. Spatial edges are derived from oriented 3D bounding boxes using fixed-priority predicates (e.g., on top of, supported by, inside, next to) based on vertical contact, containment, and proximity.

2. Active Semantic Perception (ASP)
The framework integrates ASP [18] to guide exploration. An LLM (specifically gemini-2.5-pro in experiments) samples plausible completions of the unobserved scene conditioned on the current graph. The system calculates the expected information gain for candidate viewpoints using mutual information between predicted observations and the current graph:
$I(Y_{k+1}; G_k | x) = H(Y_{k+1} | x) - H(Y_{k+1} | x, G_k)$
The robot moves to the viewpoint that maximizes this gain, prioritizing the resolution of semantic ambiguities.

3. Common Prior Maps (CPMs)
External RGB camera observations are processed identically to onboard images. When used as a CPM, they provide a partial semantic and geometric structure before deployment, allowing the ASP module to exploit richer initial context and select more informative viewpoints from the very first step.

Key Contributions

RGB-Only Pipeline: An end-to-end pipeline for 3D scene graph generation that enables the seamless integration of fixed external cameras without requiring depth sensors or hardware modifications.
Active Exploration Loop: A semantic exploration mechanism that utilizes the generated scene graph to select informative viewpoints, actively refining the prior.
Empirical Validation: Experimental evidence demonstrating that camera-infrastructure priors substantially improve scene graph completeness and exploration efficiency.

Experimental Results

The method was evaluated in three settings using the Replica and ReplicaCAD datasets:

1. Static Pipeline Validation
Comparing the original ConceptGraphs (using ground-truth depth/poses) against the proposed RGB-only variant (using MapAnything predictions) on the Replica dataset showed strong parity.

F1-Score: Nearly identical (0.499 for CG vs. 0.500 for CG-RGB).
Recall: The RGB-only variant slightly improved recall (0.436 vs. 0.401), likely because noisier depth predictions prevented the over-aggressive merging of closely situated objects.

2. Active Exploration with CPM Initialization
Experiments in the Habitat simulator (ReplicaCAD apartments) compared ASP operating on onboard observations alone versus ASP initialized with a single external camera.

Initial Impact: Initializing with one external camera increased the initial node count by +74% (from ~23 to ~40) and initial recall by +79% (from ~0.15 to ~0.26).
Sustained Performance: By step 30, the CPM-initialized system maintained a higher trajectory, reaching ~107 nodes and a recall of ~0.56, compared to ~98 nodes and ~0.47 recall for the onboard-only baseline.

3. Standalone CPM Value
Evaluating the system using only fixed external cameras (1–3 cameras) without a mobile robot:

Complex Apartments: Three cameras achieved a recall of 0.301.
Furnished Rooms: Three cameras achieved a recall of 0.555.
Conclusion: While static cameras cannot match the coverage of active robotic exploration, they successfully establish the dominant structure of the environment, functioning as high-quality priors even without a mobile robot.

Significance and Claims

The paper claims that treating fixed external cameras as Common Prior Maps offers a practical, cost-free method to enhance robotic autonomy. By leveraging existing surveillance infrastructure, the system provides a structured semantic and geometric representation that can be incrementally refined.

The authors emphasize that this approach:

Bootstraps Exploration: Even a single external camera significantly increases initial object recall and provides a richer context for subsequent active exploration.
Hardware Agnostic: It removes dependencies on depth sensors and proprietary LLMs for relationship inference, relying instead on feed-forward 3D reconstruction and deterministic geometric rules.
Improves Robustness: The wide-field views from external cameras help anchor the reconstruction in a consistent global reference frame, potentially improving localization and mapping stability across rooms.

The work positions itself as the first system to enable 3DSG generation in an RGB-only setting that integrates external camera inputs, opening new directions for scalable, infrastructure-assisted robot autonomy.