💬 NLP

SceneCritic: A Symbolic Evaluator for 3D Indoor Scene Synthesis

The paper introduces SceneCritic, a symbolic evaluator grounded in a structured spatial ontology (SceneOnto) that provides stable, object-level assessments of 3D indoor scene layouts, demonstrating superior alignment with human judgment compared to vision-language models and revealing that image-based refinement is most effective for correcting semantic and orientation errors.

Original authors: Kathakoli Sengupta, Kai Ao, Paola Cascante-Bonilla

Published 2026-04-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Kathakoli Sengupta, Kai Ao, Paola Cascante-Bonilla

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are an interior designer trying to build a perfect living room using a magical robot. You tell the robot, "Put a piano, a table, and some chairs in here," and it generates a 3D room. But how do you know if the robot did a good job? Did it put the piano floating in the air? Did it make the table the size of a toy? Did it make the chairs face the wrong way?

This is the problem the paper SceneCritic tries to solve.

The Problem: The "Hallucinating" Judge

Currently, when we want to check if a robot-generated room is good, we ask a super-smart AI (called a Vision-Language Model or VLM) to look at a picture of the room and give it a grade.

Think of this AI judge like a tourist with a camera.

The Flaw: If the tourist takes a photo from the front, they might see a beautiful arrangement. But if they walk around to the side, they might see a chair floating in mid-air or a table that is way too small.
The Result: The tourist (the AI judge) gives the room a high score in one photo and a terrible score in another, just because of the angle. They also get confused easily. They might "hallucinate" and say, "Oh, I see a cat on the table!" even though there is no cat. They are unreliable because they are judging a 3D world through a 2D lens.

The Solution: SceneCritic (The "Blueprint Inspector")

The authors built a new tool called SceneCritic. Instead of looking at a picture, SceneCritic looks at the blueprint (the mathematical data) of the room.

Think of SceneCritic as a strict building inspector who doesn't care about how the room looks in a photo. Instead, they carry a clipboard with a giant rulebook called SceneOnto.

What is SceneOnto?
Imagine a massive library of knowledge about how real human rooms work. It was built by studying thousands of real houses (from datasets like 3D-FRONT and ScanNet). It knows:

Scale: A piano is huge; a coffee table is small. If you put a giant piano in a tiny room, the rulebook says "Fail."
Co-occurrence: You usually find a bed and a nightstand together. If you have a bed but no nightstand, the rulebook says "Missing item."
Orientation: Chairs should face the table. If a chair is facing the wall, the rulebook says "Wrong direction."

How SceneCritic Works

When the robot generates a room, SceneCritic doesn't take a photo. It walks through the blueprint like a detective:

It checks the math: "Is this chair 2 meters tall? No, that's impossible. Fail."
It checks the relationships: "Is this lamp sitting on the floor or floating? Fail."
It checks the vibe: "Do these objects belong together? Yes, a desk and a chair go together. Pass."

Because it checks the actual rules and numbers, it never gets confused by camera angles. It gives a stable, fair score every single time.

The Big Experiment: Who is the Better Teacher?

The authors didn't just build a judge; they built a training gym. They let different AI models try to design rooms, and they used three different "coaches" to give feedback:

The Rule Coach: "You have a collision! Fix it." (Basic rules).
The Text Coach: An AI that reads the room description and says, "This feels wrong." (Language only).
The Image Coach: An AI that looks at the rendered photo and says, "That looks bad." (Vision + Language).

The Surprising Results:

The Image Coach is the best at fixing orientation: If a chair is facing the wrong way, looking at the picture helps the robot fix it faster than just reading text.
The Text Coach is surprisingly good at layout: Sometimes, a smart text-only AI (without vision) actually designs better semantic layouts (getting the right objects in the right places) than the fancy vision models.
SceneCritic is the ultimate truth-teller: When the authors compared the scores, the "Tourist" (VLM judge) was all over the place, changing its mind based on the angle. SceneCritic (the Blueprint Inspector) agreed with human experts 94% of the time, while the tourist only agreed 58% of the time.

The Takeaway

If you want to build a realistic 3D world, don't just ask an AI to "look" at it and guess. You need a system that understands the rules of the game (physics, size, and human habits).

SceneCritic is that system. It's like replacing a confused tourist with a professional architect who knows exactly how a room should be built, ensuring that the digital worlds we create are not just pretty pictures, but logical, usable spaces.

1. Problem Statement

The generation of 3D indoor scenes using Large Language Models (LLMs) and Vision-Language Models (VLMs) has advanced significantly, often producing layouts via intermediate structures like scene graphs or floor plans. However, evaluating these generated scenes remains a critical bottleneck.

Instability of Current Evaluators: The standard evaluation protocol relies on VLMs judging a small number of rendered 2D views. This approach is highly unstable due to:
- Viewpoint Sensitivity: Scores vary drastically depending on the camera angle (e.g., occlusion hides object relationships).
- Hallucination: VLMs frequently hallucinate object presence or spatial relationships.
- Prompt Sensitivity: Minor rephrasing of evaluation prompts leads to significant score fluctuations.
Lack of Granularity: Existing methods often provide a single aggregate score or coarse ranking without identifying which specific spatial constraints (e.g., scale, orientation, overlap) were violated.
Limitations of Programmatic Metrics: Traditional metrics (FID, collision checks) capture global statistics or geometric feasibility but fail to assess semantic coherence (e.g., whether a bed and nightstand are placed together, or if a chair faces a table).

2. Methodology

The paper introduces SceneCritic, a symbolic evaluator that operates directly on the 3D floor-plan layout (before rendering) to provide stable, interpretable, and view-independent assessments.

A. SceneOnto: A Structured Spatial Ontology

SceneCritic's constraints are grounded in SceneOnto, a dataset constructed by aggregating spatial priors from three sources:

3D-FRONT: Professional indoor scenes with detailed geometric annotations.
ScanNet: Real-world RGB-D reconstructions.
Visual Genome: Object co-occurrence and support relationships from annotated images.

SceneOnto encodes four types of relational priors per room type:

Object Dimensions: Statistical distributions (percentiles, mean, std) for width, height, and depth.
Support Relations: Which surfaces objects rest on (e.g., floor, table).
Co-occurrence Statistics: Conditional probabilities $P(B|A)$ and normalized Pointwise Mutual Information (nPMI) for object pairs (e.g., a bed is likely to co-occur with a nightstand).
Orientation Relationships: Canonical orientations, including "back-to-wall," "faces-center," and "faces-pair" (e.g., chairs facing a table).

B. The SceneCritic Evaluator

SceneCritic traverses the SceneOnto graph to verify the generated layout against three axes:

Spatial Semantics:
- Scale: Checks if object dimensions fall within the 5th–95th percentile range of the ontology.
- Co-occurrence: Verifies if semantically linked objects are placed in proximity.
- Completeness: Ensures all requested objects are present.
Orientation Correctness:
- Verifies if objects align with canonical directions (e.g., a sofa's back is against a wall, a TV faces the room center).
- Uses angular deviation thresholds (soft/hard fail) relative to wall normals or room centroids.
Overlap Detection:
- Proximity Overlap: Uses Axis-Aligned Bounding Boxes (AABB) to detect objects that are too close.
- True Overlap: Uses Oriented Bounding Boxes (OBB) and the Separating Axis Theorem (SAT) to detect actual geometric collisions.

C. Iterative Refinement Testbed

To study how models revise spatial structures, the authors paired SceneCritic with an iterative refinement pipeline. Models generate an initial layout and then refine it under three critic modalities:

Rule-based Critic: Provides feedback based on collision and boundary constraints.
LLM Critic: Operates on the layout as text.
VLM Critic: Operates on rendered 2D images (image-only, image+text, or semantics+text).

3. Key Contributions

SceneOnto: A new, dataset-grounded ontology for indoor layout evaluation containing object dimensions, co-occurrence statistics, support surfaces, and orientation preferences.
SceneCritic: A symbolic evaluator that provides object-level and relationship-level assessments. It identifies specific violations (e.g., "Chair 2 should face the table") rather than just assigning a global score.
Iterative Refinement Framework: A testbed to probe how different post-training objectives (RL, RLHF, RLVR) affect a model's ability to build and correct spatial structures.
Comprehensive Evaluation: A large-scale study comparing SceneCritic against VLM evaluators and human judgments across multiple generation backbones and refinement strategies.

4. Key Results

A. Alignment with Human Judgment

Superiority over VLMs: SceneCritic aligns significantly better with human judgments than VLM-based evaluators.
- Easy Scenes: SceneCritic achieves 94.44% agreement vs. 58.82% for VLMs.
- Complex Scenes: SceneCritic achieves 83.33% agreement vs. 47.06% for VLMs.
Stability: Unlike VLMs, which show high variance in scores based on the rendered view (e.g., left vs. right view), SceneCritic produces consistent scores because it evaluates the 3D layout directly.

B. Comparative Performance of Models

Text-only LLMs vs. VLMs: Surprisingly, text-only LLMs (specifically DeepSeek-3.2V) outperformed proprietary VLMs (like Gemini-2.5-Flash) on semantic layout quality when refined via text-based critics. This suggests that text-based spatial reasoning can be more effective than visual-semantic knowledge for layout generation.
Math-Oriented Training: Models trained with math-oriented objectives (e.g., Qwen2.5-14B-GRPO) achieved the best orientation scores, outperforming much larger models. This implies that precise angular computation benefits from math-focused training.
Most Effective Critic Modality: Image-based VLM refinement was the most effective modality for correcting semantic and orientation errors. Visual feedback provided richer spatial information than text-only inputs. However, adding text to images sometimes degraded performance, suggesting models may prioritize textual cues over visual spatial data.

5. Significance

Reliable Evaluation: SceneCritic solves the "unstable evaluator" problem by removing the dependency on rendered views and VLM hallucinations, offering a ground-truth-like assessment for 3D scene generation.
Interpretability: By pinpointing specific violations (e.g., scale issues, wrong orientation), it provides actionable feedback for model improvement, unlike black-box scoring systems.
Insight into AI Reasoning: The study reveals that current VLMs struggle with 3D spatial reasoning from 2D projections, while text-based LLMs can excel at semantic layout planning. It also highlights the importance of math-oriented training for geometric precision.
Extensibility: The SceneOnto framework is designed to be easily extended with new room types and object categories, facilitating future research in embodied AI and virtual environment generation.

In conclusion, SceneCritic establishes a new standard for evaluating 3D indoor scenes, demonstrating that symbolic, ontology-driven evaluation is more reliable, stable, and aligned with human preferences than current VLM-based approaches.