Synthetic Dataset Generation for Partially Observed Indoor Objects

1. Problem Statement

Learning-based methods for 3D scene reconstruction and object completion require large datasets pairing partial scans (what a sensor sees) with complete ground-truth geometry (the full object/scene, including occluded parts).

• Real-world limitations: Acquiring such data physically is costly, time-consuming, and often impossible because accurate ground truth for occluded regions cannot be captured without multiple scanning positions or destructive access.
• Existing synthetic limitations: Current synthetic datasets often rely on direct surface sampling of 3D meshes. This approach fails to simulate physical scanner constraints, resulting in unrealistic point distributions, missing visibility constraints, and a lack of measurement noise or range limitations.
• The Gap: There is a need for a scalable framework that generates realistic partial scans with accurate ground truth, explicitly modeling sensor physics (noise, occlusion, range) and providing object-level annotations.

2. Methodology

The authors propose a Virtual Scanning Framework implemented in Unity, integrated with a procedural indoor scene generation pipeline.

A. Procedural Scene Generation

• Layout Generation: Rooms are generated algorithmically with variable dimensions, walls, doors, windows, and ceilings.
• Furniture Placement: Objects are placed in two stages (wall-mounted and free-standing) using bounding box intersection tests to ensure collision-free, physically plausible layouts.
• Asset Management: Visual styles are managed via Unity ScriptableObjects, allowing for diverse architectural themes and furniture collections while using the same underlying generation logic.
• Determinism: Generation is seed-based, allowing for reproducible scenes.

B. Virtual Scanner Simulation

Instead of sampling mesh surfaces, the system simulates a physical scanner using ray-casting:

• Scan Vector Generation: A spherical array of vectors is generated from a virtual viewpoint. Vectors are distributed uniformly on vertical disks to simulate a blind zone beneath the sensor and ensure uniform coverage.
• Ray-Casting & Acquisition: Rays are cast against the scene. Intersection points are recorded with 3D positions and surface normals.
• Physical Modeling:
  • Noise: Gaussian noise and distance-dependent error are applied to simulate system inaccuracies.
  • Range & FOV: Configurable maximum range and vertical field-of-view filter points, mimicking real sensor limitations.
  • Presets: Parameters can emulate specific real-world devices (e.g., Leica P30, NavVis VLX, iPhone Pro LiDAR).
• Coloring: A 360° panoramic image (cubemap to equirectangular projection) is captured at the scanner location. Scanned points are colored by mapping their angular coordinates to the panorama, preserving lighting and material consistency.

C. Ground Truth & Occlusion Analysis

• Object Isolation: Non-structural objects are assigned scripts to isolate points within their Oriented Bounding Boxes (OBB).
• Empty Scene Rescan: To obtain complete ground truth, the scene is "rescanned" with all furniture removed, generating a full empty-scene point cloud.
• Voxel Occlusion Grids: A voxel grid is defined over the scene and objects. Rays are cast from the scanner to voxel centers; voxels intersected by other geometry are marked as occluded. This provides explicit, ground-truth visibility maps for both the whole scene and individual objects.

3. Key Contributions

1. Virtual Scanner for Unity: A plug-and-play module that simulates real-world scanning behavior (noise, range, occlusion) via ray-casting rather than surface sampling.
2. Procedural Scene Generation: An automated pipeline creating diverse, collision-free indoor layouts and furniture arrangements.
3. Occlusion Voxel Grids: A novel method for computing full-scene and per-object occlusion grids, explicitly representing visibility constraints.
4. V-Scan Dataset: A new synthetic dataset containing:
   • Partially scanned indoor scenes (furnished).
   • Complete ground-truth geometry (empty scene scans).
   • Object-level partial point clouds with OBBs.
   • Voxel-based occlusion annotations.
   • Panoramic imagery and normals.

4. Results

• Dataset Scale: The authors generated approximately 100 furnished virtual scans.
• Diversity: The dataset includes multiple environment styles and furniture configurations generated procedurally.
• Data Completeness: Unlike real-world datasets, every sample in V-Scan includes paired partial observations and complete ground truth, along with explicit occlusion masks.
• Realism: By incorporating distance-dependent noise and sensor-specific presets, the generated point clouds closely mimic the statistical properties of real-world LiDAR and ToF sensors.

5. Significance

• Bridging the Gap: The framework effectively bridges the gap between simple, unrealistic surface sampling and expensive, limited real-world data acquisition.
• Supervised Learning: The availability of ground-truth occlusion and complete geometry enables the training of advanced deep learning models for:
  • 3D object completion.
  • Scene reconstruction from partial views.
  • Occlusion reasoning and robotic perception.
• Scalability: The procedural nature of the pipeline allows for the infinite generation of diverse training data, addressing the data scarcity bottleneck in 3D computer vision.
• Future Impact: The system lays the groundwork for training models that can robustly handle real-world sensor noise and occlusion, with future work planned to include mobile scanning trajectories and increased environmental diversity.