💻 computer science

RoomRecon: High-Quality Textured Room Layout Reconstruction on Mobile Devices

RoomRecon is a real-time, mobile-based pipeline that leverages AR-guided capture and generative AI to reconstruct high-quality, customizable 3D room models by focusing on permanent structural elements, thereby outperforming existing methods in both texturing quality and on-device efficiency.

Original authors: Seok Joon Kim, Dinh Duc Cao, Federica Spinola, Se Jin Lee, Kyu Sung Cho

Published 2026-04-22

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Seok Joon Kim, Dinh Duc Cao, Federica Spinola, Se Jin Lee, Kyu Sung Cho

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you want to build a perfect, digital twin of your living room to use in a video game or a virtual reality tour. You grab your phone, walk around, and scan the room. But when you look at the result, it's a mess: the walls are blurry, the floor looks like a smudged painting, and there are weird gaps where the furniture blocked the camera.

This is the problem RoomRecon solves. It's a new mobile app that turns your phone into a high-tech scanner, creating crisp, realistic 3D models of rooms that look like the real thing.

Here is how it works, broken down into simple analogies:

1. The Problem: The "Blurry Snapshot" Issue

Current 3D scanners are like taking a thousand photos of a room from every angle and trying to glue them together.

The Blur: If you move your hand too fast, the photo is blurry. If you use too many photos, the computer gets confused about which one to use, resulting in a "ghostly" or smeared texture.
The Gaps: If a chair is in the way, the scanner can't see the wall behind it, leaving a hole in the digital model.
The Rigidity: If you move your sofa, the digital model doesn't know how to update. You have to scan the whole room again.

2. The Solution: RoomRecon's "Divide and Conquer" Strategy

RoomRecon changes the game by acting more like a professional photographer than a robot vacuum.

Step A: The "Architect's Sketch" (Ignoring the Clutter)

Instead of trying to scan every single object (your messy coffee table, the pile of laundry), RoomRecon focuses only on the permanent skeleton of the room: the walls, the floor, and the ceiling.

Analogy: Imagine you are building a house. You don't paint the furniture; you paint the walls. RoomRecon builds a clean, empty "shell" of your room. This means if you move your sofa later, the digital room doesn't need to be rebuilt; you just add a new digital sofa to the existing shell.

Step B: The "AR Tour Guide" (Taking the Perfect Photo)

This is the magic part. Instead of letting the user just wave the phone around randomly, RoomRecon uses Augmented Reality (AR) to act as a tour guide.

The Analogy: Imagine a tour guide standing in front of a wall saying, "Okay, stand right here, hold the phone perfectly still, and take one photo."
How it works: The app calculates the exact spot where your phone needs to be to capture the wall perfectly. It draws a red box on your screen showing the target area and a green arrow showing where to stand. It tells you, "You're too close, step back," or "You're too high, lower your hand."
The Result: You take very few photos, but they are crystal clear and taken from the perfect angle. This eliminates the blur and the "seams" where photos usually don't match up.

Step C: The "Digital Painter" (Fixing the Mistakes)

Even with perfect photos, sometimes a chair blocks a corner, or a light switch looks weird. RoomRecon has a "Post-Texturing" phase.

The Analogy: Think of this like a digital artist using Photoshop or AI.
- Inpainting: If a chair blocked a part of the wall, the app uses AI to "hallucinate" (guess) what the wall looks like behind the chair and fills it in seamlessly.
- Sample Mode: For floors and ceilings, you can take a tiny sample of your actual floor tile, and the app uses that to paint the entire digital floor, making it look exactly like your real wood or carpet.

3. Why This Matters

Speed: Because it takes fewer, better photos, it processes the room much faster than older methods.
Quality: The textures are sharp, not blurry. The lines on the floor tiles are straight, not wavy.
Flexibility: Because it separates the "walls" from the "furniture," you can easily swap out digital furniture later without re-scanning the whole room.

The Bottom Line

RoomRecon is like upgrading from a point-and-shoot camera that takes 1,000 blurry photos to a professional studio setup where a guide tells you exactly where to stand to take one perfect shot. It turns your smartphone into a tool that can create movie-quality 3D rooms for Virtual Reality, interior design, and real estate tours, all while you're sitting on your couch.

1. Problem Statement

While RGB-D sensors and 3D reconstruction technologies have advanced, creating high-fidelity, realistic 3D room models on mobile devices remains challenging due to three primary limitations:

Static Nature & Lack of Semantics: Current methods produce static meshes that require full recapture when furniture changes. They lack semantic understanding, preventing users from selectively editing specific objects (e.g., removing a table but keeping a bed).
Texturing Quality Issues: Environmental factors (furniture occlusion, shadows, thin objects) and technological limitations (camera pose drift, motion blur) result in 3D models with untextured areas, visible seams, blurriness, and lighting inconsistencies.
Inefficiency: Existing state-of-the-art (SOTA) texturing methods often require processing hundreds of images, leading to high computational costs and long processing times unsuitable for real-time mobile applications.

2. Methodology

RoomRecon proposes an interactive, real-time, two-phase pipeline designed specifically for mobile devices (iOS) to generate high-quality textured room layouts.

Phase 1: Scanning and Mesh Processing

Capturing: Users scan a room using an RGB-D sensor (e.g., iPhone LiDAR). The system generates an original mesh ( $M_{original}$ ) and room layout (planes, normals, corners).
Mesh Processing (Divide-and-Conquer Strategy):
- Filtering: The system filters $M_{original}$ to remove mesh faces close to walls (to prevent texture bleeding) while retaining object meshes for occlusion testing.
- Remeshing: The room layout (walls, floor, ceiling) is remeshed using Constrained Delaunay Triangulation (CDT) to create a clean, watertight composite mesh ( $M_{combined}$ ).
- Semantic Segmentation: The mesh is semantically labeled to distinguish between permanent structural elements (walls/floors) and dynamic objects.

Phase 2: AR-Guided Texturing and Refinement

AR-Guided Image Acquisition (Divide and Conquer):
- Instead of using the raw, often blurry scan images, the system guides the user to capture a small set of high-quality images specifically for texturing.
- Divide: A recursive algorithm splits large walls into sub-planes that fit within the camera's Field of View (FoV) at an optimal distance.
- Conquer: An AR interface provides visual cues (arrows, 3D phone models, haptic feedback) to guide the user to the exact optimal position ( $X^*$ ) to capture each sub-plane, ensuring sharpness and minimizing the number of images needed.
Texturing: The system applies assignment-based texturing (using SOTA methods like SeamlessTex or MVSTex) but utilizes the newly captured high-quality images ( $C_{recaptured}$ ) instead of the original scan data.
Plane2Image Rendering: The 3D textured planes are rendered into 2D images. This decoupling allows for standard 2D image processing.
Post-Texturing (Enhancement):
- ImageMode: Uses ZIT (an incremental transformer-based inpainting model) to fill untextured areas (occlusions) and remove artifacts. It also supports Stable Diffusion for creative customization.
- SampleMode: Allows users to capture a single sample of a floor or ceiling texture and apply it programmatically with adjustable parameters (angle, offset, scale) to cover the entire surface, solving the repetitive pattern alignment issue.

3. Key Contributions

RoomRecon Framework: A mobile framework that efficiently generates complete, high-fidelity 3D room models focusing on permanent layout elements, allowing for easy customization without full recapture.
AR-Guided Divide-and-Conquer Texturing: A novel strategy that minimizes the number of input images by guiding users to capture optimal, sharp images for specific plane sections, significantly reducing seams and blur.
Plane2Image Module: A rendering pipeline that converts 3D planar structures into 2D images, enabling the use of advanced 2D image processing (inpainting, generative AI) to refine textures.
Comprehensive Evaluation: Extensive experiments and user studies demonstrating superior performance over SOTA methods in both quantitative metrics and user preference.

4. Experimental Results

The authors evaluated RoomRecon on 13 indoor spaces (offices and homes) using an iPhone 12 Pro, comparing it against six methods (ColorMapOpt, PlaneOpt, MVSTex, SeamlessTex, and their adaptations).

Texture Quality:
- Local Quality: RoomRecon achieved a 9–13% improvement in sharpness (Blurriness metric) and higher PSNR compared to baselines.
- Global Quality: Compared to the baseline SeamlessTex, RoomRecon (Our-SeamlessTex) improved PSNR by 18.1%, SSIM by 13.6%, and reduced Blurriness by 14.7%.
Efficiency:
- Image Reduction: The method reduced the number of images required for texturing from an average of 258.85 (standard sampling) to 11.38 (recaptured).
- Processing Time: On a Linux server, texturing time dropped from 32.62s (SeamlessTex) to 4.44s (RoomRecon). On the mobile device, the total pipeline time was significantly reduced, with the texturing step taking only 4.72s for the proposed method versus 18.71s for standard MVSTex.
User Study: In a study with 30 participants, 43.6% and 42.1% voted for the RoomRecon variants (Our-MVSTex and Our-SeamlessTex) as the best, significantly outperforming the baselines.

5. Significance

RoomRecon addresses the critical gap between high-quality 3D reconstruction and practical mobile usability. By shifting the burden from complex algorithmic optimization to interactive, AR-guided user capture, it achieves:

Higher Realism: Elimination of seams and blur through controlled, high-quality image acquisition.
Customizability: The focus on permanent layout elements allows users to easily swap furniture or decor in VR/AR applications without re-scanning the room.
Mobile Feasibility: The pipeline is optimized for on-device computation, making high-fidelity room modeling accessible to non-experts for applications in real estate, interior design, and the Metaverse.

The paper concludes that focusing on the process of image acquisition (via AR guidance) and post-processing (via 2D image refinement) is more effective for mobile texturing than relying solely on complex 3D reconstruction algorithms.