💻 computer science

Optimization of Monodepth2-Based Monocular Vision Measurement Method Using Prior Geometric Features and Semantic Segmentation

This paper proposes an optimized Monodepth2-based monocular vision measurement method that integrates semantic segmentation and fixed-weight spatial correction using prior geometric features to achieve sub-millimeter accuracy in construction scenarios, demonstrating over 96% error reduction compared to conventional approaches.

Original authors: Shuzhan Wu, Lizhen Huang, Jingwen Liu, Hualin Tang, Kai Zhao, Tianfan Zhang

Published 2026-07-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Shuzhan Wu, Lizhen Huang, Jingwen Liu, Hualin Tang, Kai Zhao, Tianfan Zhang

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Seeing 3D with One Eye

Imagine you are trying to guess how far away a car is just by looking at a single photograph. This is what monocular vision does—it tries to figure out depth (distance) using only one camera, like a human eye.

The problem? A flat photo has no depth. It's like looking at a painting of a mountain; you can see the colors, but you can't tell exactly how far away the peak is. Existing computer programs (like the one called Monodepth2) are good at guessing, but they often get the scale wrong. They might think a small toy car is a real car, or they might blur the edges of objects, making them look fuzzy.

This paper presents a new "superpower" for these computer programs. It teaches the computer to use two extra tricks to fix its mistakes:

Semantic Segmentation: Knowing what it is looking at (like distinguishing a person from a tree).
Prior Geometric Features: Knowing the rules of the object (like knowing a concrete block is a perfect rectangle).

The Three-Step "Fix-It" Process

The authors built a system that works in three stages, which we can compare to a detective solving a mystery.

Step 1: The Rough Sketch (Monodepth2)

First, the computer takes a single photo and makes a "rough sketch" of the depth. It guesses where things are close and where they are far.

The Flaw: This sketch is often blurry. The edges of objects are fuzzy, and the distances might be slightly off, like a map drawn by someone who has never seen the territory.

Step 2: The "Highlighter" (Semantic Segmentation)

Next, the system uses a tool called UNet++ (a type of AI) to act like a highlighter.

The Analogy: Imagine you are looking at a crowded stadium photo. You want to measure the distance to the players on the field, but the crowd in the stands is distracting. The "highlighter" paints over the players in green and the crowd in red.
The Result: The computer now ignores the background noise (the crowd) and focuses only on the "Region of Interest" (the players). This prevents the computer from getting confused by things it doesn't need to measure.

Step 3: The "Ruler" (Prior Geometric Features)

This is the paper's biggest innovation. The computer knows that the object it is measuring (a precast concrete block) has specific, perfect geometric rules.

The Analogy: Imagine you are looking at a photo of a brick wall. Even if the photo is slightly distorted, you know that bricks are perfect rectangles with straight edges. If the computer's "rough sketch" says the brick is curved or wobbly, the computer uses its "ruler" (the known geometry) to force the sketch back into a straight line.
The "Fixed-Weight" Trick: Instead of constantly changing how it corrects the image (which is slow and complicated), the system uses a fixed-weight correction. Think of this as a pre-set template. If the computer sees a concrete block, it applies a specific "straightening" rule that is known to work for that shape. It doesn't need to guess; it just applies the rule.

The Results: From "Fuzzy" to "Sub-Millimeter"

The researchers tested this method on concrete blocks (like those used in building construction).

Before the fix: The computer's depth estimates were quite messy. The error was large, like trying to measure a room with a rubber ruler that stretches.
After the fix: By using the "highlighter" to focus on the object and the "ruler" to straighten the edges, the errors dropped dramatically.
- They improved the accuracy by over 96%.
- The final measurements were accurate to within sub-millimeters (less than the thickness of a coin).

Why This Matters (According to the Paper)

The paper claims this method allows a single camera to measure large construction objects with the precision usually reserved for expensive, multi-sensor systems. It solves the problem of "scale ambiguity" (not knowing if something is small and close or big and far) by forcing the computer to respect the known shape of the object.

In short: They took a computer that was good at guessing distances but bad at details, gave it a highlighter to focus on the right object, and a ruler to force the object to look like the perfect shape it actually is. The result is a highly accurate 3D measurement system using just one camera.

Technical Summary: Optimization of Monodepth2-Based Monocular Vision Measurement

Problem Statement
Monocular vision systems offer significant advantages in cost and simplicity for large-scale spatial measurement but suffer from the irreversible loss of depth information during 2D imaging. Existing deep learning-based monocular depth estimation methods, particularly unsupervised approaches like Monodepth2, face three critical limitations in practical applications:

Scale Inconsistency: Global depth estimation often lacks absolute depth references, leading to scale ambiguity and cumulative errors.
Detail Smoothing: Local geometric features, such as object edges and contours, are prone to smoothing, hindering the recovery of fine structural information.
Foreground-Background Confusion: Standard methods struggle to utilize target-specific geometric constraints to adaptively optimize depth maps, often resulting in errors where target objects are confused with the background.

While previous studies have attempted to incorporate geometric priors, they frequently rely on rigid templates or oversimplified assumptions that lack the flexibility to model complex contour features dynamically.

Methodology
The paper proposes a hybrid framework that integrates semantic segmentation and prior geometric feature constraints into the Monodepth2 unsupervised learning pipeline. The methodology consists of three primary stages:

Global Depth Foundation (Monodepth2):
The system utilizes the Monodepth2 framework, which employs a self-supervised learning strategy based on image reconstruction constraints. It uses an encoder-decoder architecture (ResNet-based) to generate initial depth maps without requiring dense ground-truth depth labels. This establishes a baseline spatial representation.
Semantic-Guided ROI Extraction:
To address foreground-background confusion, the method employs a UNet++ architecture for image semantic segmentation. This module isolates the Region of Interest (ROI)—specifically precast concrete components—from the background. By focusing depth estimation on the segmented ROI, the system preserves local details and reduces errors caused by irrelevant scene elements.
Prior Geometric Feature Correction:
A fixed-weight correction module is introduced to rectify the estimated depth maps using known geometric constraints.
- Geometric Modeling: The target surface is modeled as an ideal rectangle with known physical dimensions ( $W$ and $H$ ).
- Perspective Transformation: The system maps the estimated quadrilateral in the image back to a standard rectangle using perspective transformation.
- Error Modeling: Depth estimation errors are assumed to grow linearly with distance. A correction coefficient ( $k$ ) is calculated using least squares fitting to minimize the deviation between the estimated pixel dimensions and the known physical dimensions.
- Grid Partitioning: To handle non-uniform error distribution, the ROI is divided into a grid (e.g., $36 \times 30$ ). Correction coefficients are calculated individually for each grid cell, allowing for localized spatial regularization without the computational complexity of dynamic weight adjustment.

Key Contributions

Integration of Semantic and Geometric Priors: The paper introduces a collaborative optimization approach that combines semantic segmentation for ROI localization with fixed-weight geometric correction for depth refinement.
Fixed-Weight Correction Strategy: Unlike methods requiring dynamic weight adjustment or rigid templates, this approach uses a computationally efficient fixed-weight correction based on explicit geometric modeling (e.g., symmetry and known dimensions of concrete components).
Grid-Based Local Refinement: The implementation of a multi-scale grid partitioning strategy allows for the mitigation of scale inconsistency in global estimation while preserving local geometric details.

Experimental Results
The method was validated using a setup involving an Intel RealSense D415 depth camera (for ground truth and RGB acquisition) and a smartphone camera (simulating monocular input). Experiments were conducted on standard concrete specimens under various operating conditions.

Accuracy Improvement: Under a $36 \times 30$ grid partitioning configuration, the proposed method achieved a Mean Absolute Error (MAE) of 0.0071 and a Root Mean Square Error (RMSE) of 0.0096.
Comparison: These results represent an improvement of over 96% compared to conventional methods (Baseline MAE: 0.2416; RMSE: 0.2678).
Robustness: The method demonstrated sub-millimeter accuracy in indoor testing. It maintained low error rates across different operating conditions, including scenarios with multiple independent objects and complex backgrounds, outperforming traditional plane-fitting methods (Mean Normal Plane, Least Squares, and RANSAC).
Grid Sensitivity: Experiments showed that increasing grid density (from $12 \times 10$ to $60 \times 50$ ) progressively reduced correction errors, validating the effectiveness of finer grid partitions for local detail optimization.

Significance and Claims
The authors claim that this work provides a new paradigm for quantitative applications in construction scenarios, specifically for the measurement of precast concrete components. By achieving sub-millimeter accuracy under single-image monocular conditions, the method offers a robust solution for intelligent construction and industrial metrology.

The study acknowledges current limitations, noting that the method is primarily validated on planar objects and may face challenges with curved or irregular geometries. Additionally, the accuracy of ROI extraction remains dependent on the precision of semantic segmentation boundaries. Future work is proposed to extend verification to complex geometries and explore LiDAR-vision fusion for enhanced robustness.