🔬 optics

Using a Digital Twin for Fringe Projection Profilometry Optimisation

This paper presents an automated digital twin framework implemented in Blender to optimize fringe projection profilometry by systematically tuning system geometry and algorithmic parameters, which successfully reduced the required image count by 48% and significantly improved reconstruction accuracy when transferred to a physical system.

Original authors: D. Weston, X. Kong, G. S. D. Gordon, S. Piano

Published 2026-05-19

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: D. Weston, X. Kong, G. S. D. Gordon, S. Piano

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to take a perfect 3D photo of a complex object, like a car part or a sculpture, using a special camera and a projector. This technique is called Fringe Projection Profilometry (FPP). Think of the projector as a flashlight that shines a pattern of wavy lines (like a barcode) onto the object. The camera looks at how those lines bend and distort over the bumps and curves of the object. By mathematically analyzing those bends, the computer can build a precise 3D map of the surface.

However, getting this system to work perfectly is tricky. You have to tune many knobs: how many lines to project, how far apart the camera and projector should sit, and how to process the images. In the real world, testing every possible combination is a nightmare. It's like trying to find the perfect recipe for a cake by baking a new one every time you change the amount of sugar, flour, or oven temperature. It takes forever, costs a fortune in ingredients, and you might burn your kitchen down.

The Solution: A "Digital Twin"

This paper introduces a clever shortcut: a Digital Twin.

Think of a Digital Twin as a hyper-realistic video game simulation of your actual lab. The researchers used a piece of free software called Blender (usually used for making movies and animations) to build a virtual copy of their camera, projector, and the objects they want to measure.

Here is how they made it work, step-by-step:

Building the Ghost: They didn't just guess what their equipment looked like. They measured their real camera and projector very carefully and then programmed the virtual ones in Blender to act exactly the same way. They even matched the "gamma" (how bright or dark the images look) so that a photo taken in the real world looked identical to one generated in the computer.
The Virtual Playground: Once the virtual system was a perfect mirror of the real one, they could start playing "what-if" games. They could move the virtual camera closer to the projector, change the number of lines projected, or tweak the software algorithms—all inside the computer.
The Magic Optimization: Because the computer is fast and doesn't need to buy new parts, it tested thousands of combinations instantly to find the "sweet spot" where the 3D map was most accurate.

The Results: Doing More with Less

The researchers tested this by measuring three different 3D-printed objects (a pyramid, some pillars, and a stepped block). Here is what happened when they took the settings found in the "Digital Twin" and applied them to the "Real World":

Fewer Photos Needed: In the real world, they used to need to project 36 different patterns to get a good 3D scan. After optimizing in the digital twin, they only needed 21. That's a 48% reduction in the number of photos required. It's like getting the same high-quality cake recipe but using half the ingredients.
Better Accuracy: When they changed the number of lines (stripes) in the pattern based on the digital twin's advice, the error in the 3D measurement dropped by 74%.
Smarter Spacing: They also figured out the perfect distance to place the camera and projector. In the real world, moving these heavy devices is hard and risky. In the digital twin, they just slid them closer together virtually, found the best spot, and then moved the real equipment to match. This improved the measurement quality by nearly 37%.

Why This Matters

The main takeaway is that you don't need to break your equipment or waste time guessing to get the best results. By building a "ghost" version of your system in a computer, you can experiment freely, find the perfect settings, and then simply copy those settings to your real machine.

The paper shows that this method works: the virtual system predicted the best settings, and when those settings were used on the real hardware, the measurements became faster (fewer images) and more accurate (less error). It turns the slow, expensive process of trial-and-error in a physics lab into a fast, free game of optimization in a computer.

Technical Summary: Using a Digital Twin for Fringe Projection Profilometry Optimisation

Problem Statement
Fringe Projection Profilometry (FPP) is a high-precision optical metrology technique used to reconstruct 3D surface geometry. However, optimizing FPP systems for real-world deployment is challenging. Determining the optimal trade-off between computational complexity, measurement resolution, and system configuration (e.g., camera-projector spacing, fringe density, phase-shift count) typically requires exhaustive physical testing. This process is time-consuming, financially costly, and often impractical due to mechanical constraints that fix device positions after fabrication. Furthermore, existing simulation tools often lack the physical fidelity required for metrology or are not easily customizable.

Methodology
The authors propose an automated "digital twin" framework implemented in Blender, an open-source, ray-traced 3D rendering environment. The methodology follows a "real-to-digital" refinement pipeline to align the virtual simulation with a physical FPP system, followed by an optimization phase where parameters are tuned in the virtual domain and transferred back to the physical system.

System Characterisation and Matching:
- Physical Setup: A DLP4500 projector and Basler Ace camera were characterised using Zhang's method to obtain intrinsic and extrinsic parameters. Gamma correction parameters ( $a, b, g$ ) were also derived to model non-linear radiometric responses.
- Digital Twin Construction: A custom Python add-on was developed to emulate projector behavior within Blender's Cycles rendering engine. The virtual system was initialized with the physical system's characterisation data.
- Refinement Loop: To ensure high fidelity, the digital twin underwent a refinement process involving two stages:
  - Gamma Matching: A non-linear least-squares optimisation minimised the mean DC intensity difference between physical and virtual images, adjusting virtual projector radiant intensity and gamma parameters.
  - Characterisation Matching: The pose of the characterisation board in the simulation was adjusted to match physical images. The objective function combined the Structural Similarity Index (SSIM) with a geometric consistency term based on the difference in reprojection errors ( $\epsilon_d$ ) between physical and virtual systems. This prevented overfitting to low-intensity regions and ensured geometric alignment.
Optimisation Framework:
Once the digital twin was validated, an optimisation pipeline was employed to improve system performance. The objective was to minimize the Symmetric Mean Chamfer Distance (SMCD) between reconstructed meshes and ground-truth artefacts. The framework iteratively adjusted three key parameters:
- Phase-shift count: Reducing the number of projected images to lower computational load.
- Fringe stripe count: Altering the number of stripes in the temporal multi-frequency unwrapping scheme.
- Camera-projector spacing: Adjusting the baseline distance (a parameter difficult to test physically due to hardware constraints).

Key Contributions

Blender-based Digital Twin: The paper demonstrates the use of Blender, typically used for animation, as a high-fidelity simulation environment for optical metrology. It leverages ray-tracing (Cycles) and Python scripting to create a fully automated, end-to-end FPP system.
Closed-Loop Refinement: The authors introduce a specific refinement routine that matches the digital twin to the physical system not just via geometric parameters, but also via radiometric (gamma) and image-similarity metrics (SSIM + reprojection error).
Virtual Prototyping: The framework enables the exploration of system configurations (specifically camera-projector spacing) that are physically inaccessible or prohibitively expensive to test.

Results
The framework was validated using three 3D-printed artefacts (pyramid, pillars, and steps).

Twin Accuracy: After refinement, the digital twin achieved a high degree of alignment with the physical system. The average SMCD difference between the physical and virtual reconstructions was 13 µm, with the virtual system performing approximately 19% better on average, likely due to the absence of physical noise and infinite depth of field in the simulation.
Optimisation Outcomes:
- Image Reduction: Optimising the phase-shift count reduced the number of required images per measurement from 36 to 21, a 48% reduction.
- Fringe Density: Altering the fringe pattern stripe count resulted in a 74.0% reduction in mean SMCD.
- Baseline Spacing: Adjusting the camera-projector spacing purely within the digital twin yielded a 36.9% reduction in mean SMCD.
- Physical Transfer: Transferring the optimal parameters derived from the digital twin to the physical system successfully reduced the measurement effort (image count) while maintaining or improving reconstruction quality.

Significance and Claims
The paper claims that optimising an FPP system using a digital twin can significantly improve reconstruction quality while reducing measurement effort in real-world metrology applications. The authors emphasize that this approach allows for "what-if" analyses and continuous performance optimisation without risking downtime or damaging physical hardware.

The study modestly acknowledges limitations, noting that the current optimisation relies on single-objective gradient descent and that the digital twin's accuracy depends on the fidelity of the environmental modelling (lighting, geometry). However, the authors posit that this framework provides a strong foundation for future integration with AI-driven FPP algorithms, potentially addressing the scarcity of ground-truth 3D data by generating high-quality synthetic datasets for training deep learning models. The work demonstrates that virtual prototyping is a viable alternative to exhaustive physical experimentation for system configuration and optimisation.

More like this