💻 computer science

Dehallu3D: Hallucination-Mitigated 3D Generation from Single Image via Cyclic View Consistency Refinement

Dehallu3D is a plug-and-play optimization framework for 3D mesh generation that mitigates hallucinations caused by viewpoint discontinuities through a balanced multi-view continuity constraint and an adaptive smoothness mechanism, while introducing the Outlier Risk Measure (ORM) to quantify geometric fidelity.

Original authors: Xiwen Wang, Shichao Zhang, Hailun Zhang, Ruowei Wang, Mao Li, Chenyu Zhou, Qijun Zhao, Ji-Zhe Zhou

Published 2026-03-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Xiwen Wang, Shichao Zhang, Hailun Zhang, Ruowei Wang, Mao Li, Chenyu Zhou, Qijun Zhao, Ji-Zhe Zhou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are an artist trying to build a 3D statue of a cat, but you only have one single photograph of it.

This is a tricky job. Your brain has to guess what the back of the cat looks like, what the tail does, and how the ears are shaped from the other side. Most current AI models try to do this by taking that one photo, imagining a few other angles (like looking at the cat from the left, right, top, and bottom), and then stitching them together into a 3D model.

The Problem: The "Hallucination" Glitch
The problem is that these AI models sometimes get too creative. Because there are big gaps between the angles they imagine, the AI fills in the blanks with things that aren't real.

It might give the cat a second tail that doesn't exist.
It might create a weird hole in its side.
It might make a bump on its head that looks like a tumor.

In the paper, the authors call these mistakes "hallucinations" or "outliers." If you tried to 3D print this statue, it might fall apart. If you used it in a video game, players would trip over invisible bumps or fall through holes.

The Solution: Dehallu3D (The "Smooth Bridge" Builder)
The authors created a new system called Dehallu3D. Think of it as a super-smart editor that fixes the statue before it's finished.

Here is how it works, using a simple analogy:

1. The "Gap" Problem

Imagine the AI is trying to walk across a river by jumping on a few floating stones (the few angles it generated). Because the stones are far apart, the AI has to guess what's in the water between them. Sometimes it guesses wrong and creates a fake stone (a hallucination).

2. The Magic Fix: "Cyclic View Consistency"

Dehallu3D decides to fill the river with stepping stones. Instead of just looking at the cat from 4 angles, it generates 72 angles (a full 360-degree circle) with tiny gaps between them.

The Analogy: Imagine walking around the cat and taking a photo every 5 degrees. Now, you have a continuous loop of images.
The Rule: The system enforces a rule: "The view from the left must look exactly like the view from the right, just shifted slightly."
The Result: If the AI tries to put a fake bump on the cat's back, the system checks the "neighbors" (the views just before and after). Since the bump doesn't exist in the neighbors, the system says, "That doesn't make sense! Remove it." This smooths out the weird glitches.

3. The "Don't Erase the Details" Guardrail

There is a catch. If you smooth things out too much, you might erase the cat's sharp whiskers or pointy ears, making it look like a blob of clay.

To fix this, Dehallu3D uses a special "Adaptive Smoothness" filter.

The Analogy: Imagine you are polishing a rough wooden sculpture. You sand down the weird knots and splinters (the hallucinations), but you have a special sensor that says, "Stop! Don't sand the sharp point of the ear!"
This ensures the AI removes the garbage (outliers) but keeps the cool, sharp details (like spikes or edges) intact.

4. The New "Lie Detector" (ORM)

The authors realized that old ways of measuring 3D quality were bad at spotting these specific "fake bumps." So, they invented a new metric called ORM (Outlier Risk Measure).

The Analogy: Think of this as a "Lie Detector Test" for 3D shapes. Instead of just asking "Is the picture clear?", it asks, "Are there any parts of this statue that look like they don't belong?"
A low score means the statue is clean and real. A high score means it's full of weird, fake glitches.

Why Does This Matter?

For 3D Printing: You don't want a printer to try to print a floating hole or a weird protrusion that breaks the machine. Dehallu3D ensures the object is solid and printable.
For Video Games: It makes virtual worlds feel real. You won't walk into a wall that isn't there or fall through a floor that shouldn't have a hole.

In Summary:
Dehallu3D is like a perfectionist architect who builds a 3D model by checking every single angle around the object. It bridges the gaps between views to stop the AI from making things up, but it's careful enough to keep the cool, sharp details. The result is a 3D object that looks real, feels solid, and doesn't have any "ghost" parts.

1. Problem Statement

Large 3D reconstruction models have revolutionized content generation but suffer from hallucinations. Unlike 2D generation where hallucinations might manifest as artistic artifacts, in 3D reconstruction, they appear as structural outliers (e.g., odd holes, protrusions, or geometric deformities) that deviate from the input data.

Root Cause: Existing methods often rely on sparsely generated multi-view images. Large viewpoint gaps and discontinuities between these sparse views mislead the model into generating intricate details unsupported by the input data.
Impact: These outliers cause manufacturing failures in 3D printing and break user immersion in virtual reality/gaming.
Gap: Current research lacks specific mechanisms to detect and mitigate these 3D-specific hallucinations, and there are no standard metrics to quantify "outlier risk" in 3D geometry.

2. Methodology: Dehallu3D

The authors propose Dehallu3D, a framework designed to generate high-fidelity 3D meshes from a single image by eliminating outliers through a Cyclic View Consistency Refinement (CVCR) module. The pipeline consists of three main stages:

A. Mesh Initialization & Coarse Reconstruction

Input: A single high-resolution image.
Multi-view Generation: The system generates orthographic multi-view color images and normal maps.
Initialization: A rough mesh is created using Poisson reconstruction based on front/back normal maps.
Coarse Optimization: A differentiable rendering stage refines the global topology.
- Loss Function: Combines Mask Loss, Normal Loss, and a novel Surface Exposure-weighted Normal Loss ( $L_{SE}$ ).
- Mechanism: $L_{SE}$ prioritizes geometric constraints from highly visible regions (large projected areas) to ensure a reliable global structure while suppressing noise from low-visibility views.

B. Cyclic View Consistency Refinement (CVCR)

This is the core plug-and-play module designed to bridge gaps between sparse views and remove outliers. It enforces consistency across a full 360° rotation using dense intermediate viewpoints (72 views, 5° intervals).
The optimization objective includes:
$\mathcal{L}_{CVCR} = \mathcal{L}_{mask} + \mathcal{L}_{normal} + \lambda_1 \mathcal{L}_{DC} + \lambda_2 \mathcal{L}_{DS}$

Cyclic Depth Consistency Loss ( $L_{DC}$ ):
- Goal: Enforce geometric continuity between adjacent views (small angular differences).
- Mechanism: Uses SSIM and Cosine Similarity to measure alignment between rendered depth maps of adjacent cyclic views.
- Effect: Prevents abrupt depth discontinuities that cause outliers.
Adaptive Depth Smoothness Loss ( $L_{DS}$ ):
- Goal: Prevent over-smoothing (which would erase sharp features like spikes or edges) while still removing noise.
- Mechanism: Dynamically adjusts the smoothness penalty based on image gradients. In regions with high color gradients (indicating sharp geometric features), the smoothness constraint is relaxed to preserve details. In flat regions, it enforces smoothness to remove hallucinated noise.

3. Key Contributions

Dehallu3D Framework: A novel single-image-to-3D mesh generation pipeline that explicitly targets and eliminates hallucination-induced outliers.
CVCR Module: A plug-and-play refinement module that introduces dense cyclic view consistency and adaptive smoothness constraints. It bridges the gap between sparse multi-view generations without erasing fine geometric details.
Outlier Risk Measure (ORM): A new evaluation metric based on Conditional Value-at-Risk (CVaR).
- It converts the mesh to a point cloud and calculates the "tail risk" of the point distribution.
- It combines global reconstruction loss (VAE-based) and local neighborhood density to quantify the degree of outliers, addressing the lack of metrics for 3D geometric fidelity regarding hallucinations.
State-of-the-Art Performance: Demonstrated superior results in both visual quality and geometric accuracy compared to existing SOTA methods.

4. Experimental Results

Datasets: Experiments conducted on the Google Scanned Objects (GSO) dataset.
Baselines: Compared against SF3D, Unique3D, CRM, InstantMesh, TripoSR, and Wonder3D.
Quantitative Metrics:
- Visual Quality: Dehallu3D achieved the highest PSNR (21.84), SSIM (0.8966), and Clip-Similarity, and the lowest LPIPS.
- Geometric Quality: Achieved the best Chamfer Distance (0.02023) and F-Score (0.4212).
- Outlier Risk (ORM): Dehallu3D achieved the lowest ORM score (0.6440), indicating the fewest outliers, whereas Unique3D had the highest (0.6835).
Qualitative Analysis: Visual comparisons show Dehallu3D effectively removes protrusions and holes seen in baselines while preserving sharp edges (e.g., spikes on a model) that other methods either distort or smooth out.
Ablation Studies:
- Removing $L_{DC}$ leads to significant quality degradation due to lack of cross-view consistency.
- Removing $L_{DS}$ results in over-smoothing and loss of sharp features.
- Angular Interval: A 5° interval was found to be the optimal trade-off between reconstruction quality and computational efficiency. 3° intervals caused over-smoothing, while larger intervals (10°+) failed to bridge gaps effectively.

5. Significance

Practical Application: By mitigating structural outliers, Dehallu3D makes single-image 3D generation viable for 3D printing (where geometric errors cause manufacturing failure) and immersive gaming/VR (where visual inconsistencies break presence).
Methodological Shift: The paper moves beyond standard multi-view consistency by introducing cyclic consistency and adaptive smoothness, acknowledging that 3D generation requires balancing global continuity with local feature preservation.
Evaluation Standard: The introduction of the ORM metric provides a necessary tool for the community to evaluate and compare 3D generation models specifically regarding their tendency to hallucinate geometric anomalies, a critical but previously overlooked aspect of 3D AI.

In summary, Dehallu3D addresses a critical flaw in current 3D generation models (hallucinated outliers) by enforcing dense cyclic view consistency and adaptive feature preservation, resulting in high-fidelity, printable, and immersive 3D assets.