📄 radiology and imaging

Image transmission through a multimode fibre in reflection mode with physics-guided deep learning towards ultrathin endoscopy

This paper presents a physics-guided deep learning framework that combines a reflected real-valued intensity transmission matrix with image restoration networks to enable single-shot, phase-retrieval-free image recovery through multimode fibres in reflection mode, significantly improving image quality and generalizability for ultrathin endoscopy.

Original authors: Ye, Z., He, F., Zhao, T., Xia, W.

Published 2026-08-31

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Ye, Z., He, F., Zhao, T., Xia, W.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine trying to see inside the human body without making a large incision. Doctors have long relied on endoscopes—flexible tubes with cameras—to peer into narrow passages like airways or blood vessels. However, getting a clear picture through a tube that is both incredibly thin and flexible is a major engineering challenge. Traditional thin scopes often use bundles of thousands of tiny glass fibers, but these can only produce images with a honeycomb-like texture and limited sharpness. A more promising alternative is a single, ultra-thin strand of glass called a multimode fiber. While this fiber is flexible enough to twist and turn through the body's most difficult corners, it scrambles light so thoroughly that if you shine a picture through it, the light that comes out the other side looks like a random, glittering mess of dots. For years, scientists have struggled to unscramble this mess to recover the original image, especially when the fiber is used in a "reflection" mode, where the same strand sends light out and catches the light bouncing back from tissue.

A team of researchers at King's College London has now developed a new way to solve this puzzle, turning a chaotic speckle of light into a clear picture in a single instant. Their approach, detailed in a recent study, combines a clever physics-based calculation with a type of artificial intelligence designed to clean up images. Instead of trying to reverse-engineer the complex scrambling of light all at once, they first use a mathematical map to create a rough, blurry sketch of the object. Then, a specialized computer network takes that sketch and refines it, filling in the missing details and removing the noise. This two-step process allows them to see through the fiber without needing complex equipment to measure the phase of light waves, a requirement that has made previous methods difficult to use in real medical settings.

The researchers built a system where a laser beam is sent down a 2-meter-long multimode fiber with a core diameter of 600 micrometers. At the far end, the light hits an object and bounces back up the same fiber. Because the light travels down and back up, it gets scrambled twice, creating a highly complex pattern of bright and dark spots on a camera at the near end. To make sense of this, the team first performed a calibration process. They projected a series of known patterns onto a digital mirror at the tip of the fiber and recorded how the light scrambled on its way back. From these measurements, they built a "reflected real-valued intensity transmission matrix." In plain terms, this is a massive lookup table that predicts what a blurry, scrambled pattern should look like for any given object, based purely on how bright the light is, without needing to know the invisible phase of the light waves.

Using this lookup table, the system can instantly generate a first guess of the image from a single snapshot of the scrambled light. This initial image is not perfect; it is often grainy and low in contrast, but it captures the basic shape and layout of the object. The researchers then fed this rough image into four different types of deep learning networks, which are computer programs trained to recognize and restore visual details. These networks act like a digital editor, taking the grainy starting point and sharpening the edges, boosting the contrast, and removing the random speckles to reveal a clear picture. The team tested this method on three different sets of images: handwritten digits, pictures of clothing, and natural scenes. In every case, the combination of the physics-based lookup table and the AI cleaner produced significantly better results than trying to use the AI alone to guess the image from the raw scrambled light.

The true test of this method came when the researchers asked if it could handle images it had never seen before. They trained their AI networks using only pictures of clothing, but then asked them to reconstruct images of natural scenes, like animals and landscapes, which look very different from clothes. When the AI tried to do this on its own, without the help of the physics-based lookup table, the results were poor; the system failed to recognize the new shapes and produced blurry, unrecognizable blobs. However, when the AI was given the rough sketch from the lookup table first, it performed remarkably well. It successfully reconstructed the natural scenes, preserving the structure of the objects even though it had never been trained on them. This suggests that the physics-based step provides a solid foundation that helps the AI generalize to new situations, rather than just memorizing specific examples.

To prove the system works in the real world, the researchers placed actual physical objects at the tip of the fiber, including a metal ventilation grille, a sewing needle, and a printed letter. These objects were not part of any training data. The system captured the light bouncing off these items and, using the same calibration and AI process, reconstructed clear images of the grille's holes, the needle's shaft, and the strokes of the letter. The results were sharp enough to distinguish fine details, demonstrating that the method can move beyond computer simulations and handle the unpredictable nature of real-world light scattering. The researchers noted that while the system is currently limited by the need to recalibrate if the fiber bends significantly or the temperature changes, the ability to generate a clear image from a single snapshot without phase measurements is a major step forward.

This work offers a promising path toward ultrathin endoscopy, where a probe as thin as a hair could be guided into the deepest, narrowest parts of the body to provide real-time, high-resolution images. By separating the problem into a physics-guided first step and a data-driven cleanup step, the researchers have created a system that is both robust and adaptable. It does not require the complex interferometry setups that have hindered previous attempts at reflection-mode imaging, making it more practical for clinical use. The findings suggest that by combining the predictability of physics with the adaptability of artificial intelligence, scientists can unlock the potential of single-fiber endoscopes, potentially allowing doctors to see inside the body with a level of detail and flexibility that was previously impossible.

Technical Summary: Image Transmission through a Multimode Fibre in Reflection Mode with Physics-Guided Deep Learning

Problem Statement
Ultrathin endoscopy is critical for imaging narrow, hard-to-reach anatomical regions (e.g., pancreaticobiliary ducts, deep brain regions). While single multimode fibres (MMFs) offer a compelling alternative to coherent fibre bundles due to their small diameter, flexibility, and high spatial resolution, they suffer from severe mode coupling and dispersion. This scrambles input light into a random speckle pattern, necessitating computational reconstruction.

While transmission-mode imaging (where the object is beyond the fibre tip) has been explored, reflection-mode imaging is more practical for clinical endoscopy where the distal tip is inaccessible. In this mode, a single MMF delivers illumination and collects backscattered light. However, reconstruction is significantly more challenging due to:

Double-pass propagation: Light traverses the fibre twice, compounding mode coupling and dispersion.
Weak signals: The backscattered signal is inherently weak.
Phase loss: Cameras record only intensity, lacking phase information required for traditional phase-conjugation methods.
Generalization limits: Existing deep learning approaches often require large, representative datasets and struggle to generalize to unseen image types or physical objects outside the training distribution.

Methodology
The authors propose a single-shot reflection-mode imaging framework that decouples the optical inversion from the image restoration using a hybrid physics-guided deep learning approach.

Optical System:
- A 532 nm continuous-wave laser illuminates the proximal facet of a step-index MMF (600 µm core, 2 m length).
- At the distal end, a lens projects the output onto a Digital Micromirror Device (DMD) for controlled pattern generation or onto physical objects.
- Backscattered light travels back through the same MMF, is separated from the illumination path, and recorded as an intensity speckle pattern by a CMOS camera.
Reflected Real-Valued Intensity Transmission Matrix (Reflected-RVITM):
- Instead of measuring complex fields (requiring interferometry), the system calibrates an intensity-only mapping.
- The double-pass optical channel is approximated as a linear intensity-to-intensity relationship: $s = R_{ref} p + n$ , where $s$ is the measured speckle intensity, $p$ is the target intensity reflectance, and $R_{ref}$ is the reflected-RVITM.
- Calibration: $R_{ref}$ is recovered using Hadamard patterns displayed on the DMD. By measuring responses to complementary binary patterns and subtracting a reference, the matrix is solved using orthogonality relations. This requires no phase retrieval.
Hybrid Reconstruction Pipeline:
- Step 1 (Physics-Guided Initial Reconstruction): For an unknown target, the raw speckle $s$ is inverted using the pseudoinverse of the calibrated matrix ( $\hat{o}_{RVITM} = R_{ref}^+ s$ ). This yields a coarse, physics-guided initial image containing the target's spatial structure but corrupted by speckle artifacts.
- Step 2 (Data-Driven Restoration): A deep learning network takes $\hat{o}_{RVITM}$ as input and refines it to produce the final image. The network is tasked with suppressing residual artifacts and enhancing details, rather than learning the full complex speckle-to-image mapping from scratch.
Network Architectures:
Four restoration backbones were evaluated:
- CICPNet: Convolutional attention U-Net.
- HPM-Attention-UNet: Hierarchical parallel multi-scale attention U-Net.
- GAM: Residual U-Net with global attention.
- MambaIRv2: State-space model with local attention.
Experimental Protocol:
- Datasets: MNIST, Fashion-MNIST, and CIFAR (resized to 32×32).
- Strategies: Compared "Hybrid" (RVITM + Network) vs. "Direct Learning" (Network mapping raw speckle to image).
- Settings: Matched-dataset (train/test on same data) and Cross-dataset (train on Fashion-MNIST, test on CIFAR) to evaluate domain shift.
- Physical Validation: Tested on unseen physical objects (metal grille, needle, printed letter) at the distal tip without recalibration.

Key Contributions

Reflected-RVITM Framework: Extension of the intensity-only transmission matrix concept to reflection-mode MMF imaging, enabling calibration without interferometry or phase retrieval.
Decoupled Architecture: A novel pipeline separating the physics-based inversion (RVITM) from the data-driven refinement (Deep Learning). This shifts the learning burden from the complex speckle domain to the more intuitive image domain.
Robustness to Domain Shift: Demonstration that hybrid models maintain performance when trained on one dataset (Fashion-MNIST) and tested on a completely different domain (CIFAR), whereas direct-learning models fail significantly.
Physical Object Generalization: Successful reconstruction of real-world physical objects using models trained only on digital patterns, without target-specific fine-tuning.

Results

Matched Datasets: Hybrid models consistently outperformed direct-learning baselines.
- Example: On MNIST, HPM-Attention-UNet improved mean Pearson Correlation Coefficient (PCC) from 0.572 (direct) to 0.944 (hybrid), a +65.1% gain.
- Even for strong backbones where direct learning performed well, the hybrid approach provided statistically significant improvements.
Cross-Dataset Generalization:
- Direct learning suffered a severe domain shift penalty when moving from Fashion-MNIST to CIFAR (PCC dropped from ~0.83–0.91 to 0.36–0.50).
- Hybrid models were substantially more robust, maintaining PCC in the range of 0.61–0.65 on unseen CIFAR scenes.
- Qualitatively, direct models produced garment-like textures on natural scenes, while hybrid models preserved the coarse spatial layout of the objects.
Real-Object Imaging:
- Hybrid models successfully reconstructed physical objects (grille, needle, letter) with PCC values around 0.70–0.77, whereas direct-learning models often yielded near-zero or negative correlations.
- The initial RVITM reconstruction provided the necessary structural alignment, which the network refined.

Significance and Claims
The paper claims that combining a reflected-RVITM physics prior with an image-domain restoration network offers a practical, phase-retrieval-free route to reflection-mode MMF endoscopy.

Generalizability: The primary advantage is not necessarily a marginal increase in in-distribution performance, but a substantial improvement in robustness to domain shifts and transfer to physical objects. By providing a physics-guided initial reconstruction, the network is relieved of the task of learning the full optical inverse mapping, reducing its dependence on specific training image statistics.
Practicality: The method requires only intensity-only calibration (no interferometry), making it more feasible for clinical translation where phase stability is difficult to maintain.
Modularity: The framework allows for the integration of various restoration backbones and potential future improvements (e.g., perturbation-tolerant training) without altering the core physics-guided front end.

The authors conclude that this approach enables single-shot image recovery and represents a step toward minimally invasive, in vivo single-fibre endoscopy, though they acknowledge limitations regarding calibration stability under fibre bending and the need for further advances in resolution and speed for clinical deployment.

More like this