Breaking the Sub-Millimeter Barrier: Eyeframe Acquisition from Color Images

This paper presents a novel artificial vision pipeline that utilizes multi-view RGB and depth images from an InVision system to achieve sub-millimeter eyeframe lens tracing, thereby eliminating the need for specialized mechanical equipment and streamlining the optical workflow.

The Gist

The Big Problem: The "Ruler" is Too Clunky

Imagine you are a tailor trying to cut a piece of fabric to fit a very specific, oddly shaped body. In the optical world, "tailors" (opticians) need to cut glass lenses to fit perfectly inside eyeglass frames. If the measurement is off by even a tiny fraction of a millimeter, the glasses won't fit right, and your vision will be blurry.

For decades, opticians have used mechanical tracers. Think of these like a high-tech, heavy-duty caliper. The optician has to physically place the glasses on the machine, clamp them down, and run a mechanical arm around the rim to measure it.

• The downsides: It's slow, it requires expensive equipment, it needs constant calibration (like tuning a guitar), and it's a bit of a hassle to set up.

The New Solution: The "Magic Eye" Camera

This paper proposes a new way to do this using Artificial Vision (computer eyes). Instead of a mechanical arm, they use a camera system to take pictures and let a computer "see" the shape of the glasses.

Think of it like this: Instead of feeling the shape of a cookie with your fingers (the old way), you take a high-resolution photo of the cookie, and a super-smart AI instantly calculates its exact shape and size without ever touching it.

How It Works: The Three-Step Recipe

The researchers built a digital pipeline that acts like a three-course meal to get the perfect measurement:

1. The "Cutout" (Segmentation)

First, the computer looks at a photo of a person wearing glasses. It needs to find the glasses and ignore everything else (the nose, the ears, the background).

• The Analogy: Imagine you are using a digital scissors to cut a person out of a magazine photo. You want to cut only the glasses, leaving the face and the background behind.
• The Tech: They used a cutting-edge AI model (based on something called "SAM2") that is really good at finding boundaries. It's like having a very precise laser cutter that knows exactly where the glass rim ends and the skin begins.

2. The "3D Map" (Depth Estimation)

A flat photo is 2D (height and width), but glasses are 3D (they curve around your face). To measure them accurately, the computer needs to know how deep the frame is.

• The Analogy: Imagine looking at a drawing of a sphere. It looks like a flat circle. But if you add "depth shading" (shadows and highlights), your brain instantly knows it's a ball, not a flat disk.
• The Tech: The system uses a special AI to guess the "depth map." It turns the flat photo into a topographical map, showing how far away every part of the frame is from the camera. They use a "relative" depth map, which is like a topographic map showing hills and valleys, even if it doesn't know the exact altitude in meters.

3. The "Four-Eyed Detective" (Multi-View Tracing)

This is the secret sauce. The system doesn't just take one photo; it uses a tower with four cameras that snap pictures at the exact same time from different angles.

• The Analogy: Imagine trying to guess the shape of a statue in a dark room. If you look at it from one side, you might miss a bump on the back. But if you have four friends standing in a circle around it, all describing what they see, you can build a perfect 3D model in your head.
• The Tech: The computer takes the four photos, combines the "cutout" (step 1) and the "depth map" (step 2) for all four angles, and fuses them together. It's like solving a 3D puzzle where every piece helps fill in the gaps of the others.

The Results: Breaking the Barrier

The researchers tested this system and found it could measure the glasses with sub-millimeter precision.

• What does that mean? It means the error is smaller than the thickness of a human hair.
• The Comparison: In their tests, the new camera method was actually more accurate than some high-tech industrial methods that use lasers and complex light patterns.
• The Benefit: It removes the need for heavy, clunky machines. An optician just needs to put the glasses in front of the camera tower, snap a picture, and the computer does the rest instantly.

Why This Matters

This is like upgrading from a manual typewriter to a word processor.

• Old Way: Slow, requires physical tools, prone to human error, needs a dedicated room.
• New Way: Fast, digital, automated, and can be done anywhere with a camera.

The paper proves that we don't need expensive mechanical arms to measure tiny, curved objects anymore. We just need good cameras and smart software that can "see" in 3D. This makes the process of making glasses faster, cheaper, and more accurate for everyone.

Technical Summary

1. Problem Statement

In the optical industry, measuring the geometry of eyeglass frames (eyeframe tracing) is critical for ensuring precise lens cutting and fitting. Traditional methods rely on mechanical tracers that require:

• Precise physical positioning and calibration.
• Specialized, expensive equipment.
• Time-consuming workflows.
• High skill levels from technicians.

These mechanical limitations create inefficiencies and bottlenecks. The paper addresses the need for a non-contact, vision-based alternative that achieves sub-millimeter precision (specifically for measuring frame radius) using only standard RGB images, thereby eliminating the need for mechanical tools and complex calibration.

2. Methodology

The proposed solution is a computer vision pipeline utilizing multi-view data from an InVision system (a tower-mounted setup with four calibrated cameras capturing visible and IR light). The pipeline consists of three main stages:

A. Data Acquisition & Preprocessing

• System: Uses an InVision tower capturing synchronous images from four cameras at ~50cm distance.
• Input: 1296×1296 RGB images.
• Preprocessing: Includes color correction and rectification based on camera distortion coefficients.

B. Frame Segmentation

• Goal: Isolate the eyeframe from the background and user's face.
• Model: Based on SAM2 (Segment Anything Model 2), a transformer-based architecture with streaming memory.
• Architecture: Utilizes a pre-trained Hiera encoder (masked autoencoders) and a memory attention mechanism to maintain context across frames.
• Training: Fine-tuned on a custom dataset of 1,002 images (labeled via CVAT.ai) from 20+ InVision systems.
• Augmentation: Extensive data augmentation (geometric, noise, color, blur) was applied to ensure robustness.
• Outcome: Generates high-precision masks to remove background interference, focusing the subsequent model solely on the frame structure.

C. Depth Estimation

• Goal: Recover 3D spatial information to assist in trace measurement.
• Model: Uses Depth Anything, a pre-trained monocular depth estimation model.
• Strategy: Employs relative depth maps (0–255 scale) rather than absolute depth. This was chosen because relative depth offers more robust predictions and is less susceptible to noise/holes, which is sufficient for constraining the neural model to learn accurate trace measurements without needing ground-truth metric depth.
• Selection: The ViT-Base variant was selected as the optimal trade-off between accuracy and computational cost.

D. Trace Measurement (The Core Algorithm)

• Input: A 4D tensor combining RGB images and Depth maps (concatenated per view).
• Architecture: A Multi-View Stereo (MVS) inspired framework using EfficientNetV2 as the backbone.
• Strategy:
  • Input Modality: The system processes single-eye frames (splitting the dual-eye image) to reduce complexity.
  • Fusion: Two fusion strategies were tested: Early Fusion (at convolutional layers) and Late Fusion (aggregating feature vectors).
  • Optimal Configuration: The best results were achieved using Late Fusion with Max-Pooling on the EfficientNetV2-Small (S) backbone, utilizing Grayscale + Depth input (rather than RGB+Depth).
• Output: A series of 600 radial measurements (radius from the optical center to the inner frame edge) per eye.

3. Key Contributions

1. Novel Vision-Based Pipeline: First proposed end-to-end system for eyeframe tracing using only multi-view RGB images and deep learning, removing the need for mechanical tracers.
2. Custom Datasets: Creation of two specialized datasets:
   • A segmentation dataset with 1,002 labeled masks.
   • A trace measurement dataset with 500 acquisitions containing ground-truth radial measurements.
3. Sub-Millimeter Precision: Demonstrated that computer vision can achieve the <1mm accuracy required for industrial optical manufacturing.
4. Efficient Architecture: Identified that smaller models (EfficientNetV2-S) combined with relative depth and late fusion outperform larger, more complex models for this specific task.

4. Experimental Results

The system was evaluated on real-world data with the following key findings:

• Segmentation: The fine-tuned SAM2 (Base+) model achieved an IoU of 0.958, significantly outperforming DeepLabV3+ (0.931).
• Trace Measurement Accuracy:
  • Best Configuration: EfficientNetV2-S + Grayscale+Depth + Late Fusion (Max-Pooling).
  • Mean Error: 0.4238 mm.
  • Maximum Error: 1.8466 mm.
  • Performance Gain: This represents a >50% reduction in mean error compared to a vanilla RGB-only network (0.8910 mm) and a 53% improvement over baseline solutions.
  • Reliability: 88% of all measurements fell within the 1 mm error margin.
• Comparison: While no direct camera-based eyeframe benchmarks exist, the method outperforms indirect comparisons with metrology techniques (e.g., phase-shifting profilometry) which typically report errors around 1.0–2.0 mm.
• Failure Cases: Errors primarily occurred in regions heavily occluded from multiple viewpoints (e.g., upper corners of the frame), suggesting that future improvements could come from increasing sample diversity in occluded scenarios.

5. Significance

This work represents a significant shift in optical manufacturing workflows:

• Workflow Optimization: Eliminates the need for labor-intensive mechanical calibration and specialized tracing hardware.
• Accessibility: Reduces the barrier to entry for high-precision measurements, making sub-millimeter accuracy accessible via standard imaging systems.
• Industrial Viability: Validates that AI-driven computer vision can meet the stringent quality standards of the optical industry, paving the way for fully automated, high-speed lens fitting processes.

In conclusion, the paper successfully demonstrates that a multi-view, deep-learning-based approach can break the sub-millimeter barrier for eyeframe acquisition, offering a faster, cheaper, and highly accurate alternative to traditional mechanical tracing.