Knowledge-Driven Dimension Estimation from a Single Image -3D Asset Generation Technology for Digital Twin Construction

Technical Summary: Knowledge-Driven Dimension Estimation from a Single Image

Problem Statement
The verification of in-vehicle cameras for autonomous driving increasingly relies on digital twin simulations. However, a critical gap exists between virtual and real environments: while virtual objects may appear visually similar, their metric dimensions often do not match reality. This discrepancy reduces verification accuracy, as the detection distance of a sign in a simulation depends heavily on its physical scale.

Current methods for constructing digital twins, such as LiDAR and stereo cameras, struggle to capture elevated roadside infrastructure (e.g., overhead guide signs) with sufficient fidelity due to vehicle-mounted sensor limitations. Furthermore, acquiring new high-elevation point clouds is costly, and manually measuring numerous elevated assets is impractical. Consequently, there is a need for a method to estimate the absolute scale of such objects from a single monocular image when point cloud data is incomplete or unavailable, without relying solely on visual realism which often lacks metric accuracy.

Methodology
The authors propose a knowledge-driven framework that decomposes a target traffic sign into structural elements to estimate its scale and generate a 3D asset. The workflow consists of three primary stages:

1. Parts Extraction and Detection:
   The system first detects the target traffic sign in a monocular image. It then employs a custom-trained object detection model (based on YOLO) to identify specific structural components, including the sign board, support post, connecting pole, arrows, and route number indicators. To train this model, a dataset was created by manually annotating component regions, including perspective transformations for planar elements like arrows and route numbers.

2. Knowledge-Driven Size Estimation:
   Once parts are detected, their pixel dimensions are converted to real-world scale using two complementary strategies:

   • Parts Data Catalog: The system references a database of standardized dimensions for specific components (e.g., arrow width, character height, route number sign size) derived from national and local government design guidelines. By comparing the detected pixel size of a known component against its registered standard dimension, the system estimates the scale for that specific part.
   • Design Data Catalog (Fail-Safe Mechanism): To mitigate detection errors caused by occlusion, noise, or lighting, the system cross-references the detected components against a catalog of design drawings. This catalog stores structural parameters and proportional relationships (e.g., the ratio of column diameter to length, or pole-to-pole distance to sign width). If a direct size estimate deviates significantly from standards, the system validates the detection ratios against these design specifications. If the ratios match a known design record, the system infers the dimensions of the entire structure based on that record, ensuring robustness even with incomplete detection.

3. 3D Object Generation:
   Unlike generative approaches that produce a single, monolithic mesh, this method reconstructs the object at the component level. Using the estimated dimensions and design information, basic geometric primitives (cylinders, planes) are parametrically generated for each part. These independent 3D objects are then assembled according to the structural relationships defined in the design data, resulting in a modular 3D asset.

Key Contributions
The paper outlines three primary contributions:

1. Knowledge-Driven Framework: A novel approach for estimating the actual scale of traffic signs from monocular images, specifically addressing scenarios where high-altitude point cloud acquisition is difficult.
2. Robust Size Estimation: A method that decomposes objects into parts and utilizes both "Parts Data Catalogs" (standardized component dimensions) and "Design Data Catalogs" (structural ratios) to achieve fail-safe dimensional estimation.
3. Modular 3D Reconstruction: A technique for generating 3D assets at the part level rather than as a single mesh. This improves editability and reusability within digital twin spaces, ensuring scale consistency for in-vehicle camera verification.

Results and Significance
The proposed method successfully constructs 3D assets with dimensions approximating the real environment by leveraging external knowledge regarding design rules and geometric relationships. The system achieved a detection accuracy of over 80% for the target components in the constructed dataset.

The significance of this work lies in its ability to impart dimensional consistency, not just visual similarity, to digital twins. By linking component-level detection with design data, the method provides a clear basis for dimensional estimation and allows for error analysis, a feature lacking in existing visual-appearance-based 3D generation methods. The resulting 3D assets are suitable for improving the verification accuracy of autonomous driving systems in virtual environments, particularly for scenarios involving elevated infrastructure where traditional sensing fails.

Limitations and Future Work
The authors note that the current framework assumes objects with standardized designs and primarily estimates scale for components on the same plane or depth. Components at different depths require depth estimation (e.g., via stereo cameras) to correct for perspective projection, which is identified as a future task. Future work also includes quantitative evaluation using diverse real-world data, formalizing uncertainty measures for reference selection, and expanding the design knowledge base to accommodate a broader range of target objects.