💻 computer science

Trustworthy Endoscopic Super-Resolution

This paper proposes a model-agnostic, theoretically grounded framework that integrates a lightweight error-prediction network with Conformal Failure Masks to provide real-time, safety-guaranteed detection of unreliable super-resolution reconstructions in endoscopic and robotic surgery.

Original authors: Julio Silva-Rodríguez, Ender Konukoglu

Published 2026-04-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Julio Silva-Rodríguez, Ender Konukoglu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a surgeon performing a delicate operation using a tiny camera (an endoscope) inside a patient's body. The camera is small and cheap, so the images it sends back are blurry and grainy. To help you see better, you use a special AI tool called Super-Resolution (SR). This tool acts like a magical photo editor, taking the blurry image and "guessing" the missing details to make it look sharp and high-definition.

The Problem:
The problem is that this AI is a bit of a daydreamer. Sometimes, when it tries to fill in the missing details, it doesn't just guess; it hallucinates. It might invent a blood vessel that isn't there, smooth over a tear in the tissue, or turn a shadow into a tumor. In a surgery, seeing something that isn't real can be dangerous. Doctors need to know: "Can I trust this sharp image, or is the AI making things up?"

The Solution: The "Trustworthy AI" Framework
The authors of this paper built a new system to solve this. Think of it as a two-person team working together in real-time:

The Artist (The SR Model): This is the AI that makes the blurry image sharp.
The Inspector (The Error Network): This is a new, lightweight AI that watches the Artist work. Its only job is to look at the image and say, "Hey, I think this part looks suspicious," or "This part looks safe."

How It Works (The Creative Analogy)

Imagine the Inspector is a quality control manager at a bakery.

The Artist bakes a cake (the high-res image).
The Inspector tastes a tiny crumb from every part of the cake to guess how bad it might taste if it's burnt or undercooked.
The Inspector doesn't just say "It's good" or "It's bad." It gives a score for every single bite.

The "Conformal Failure Mask" (The Red Sticker)
This is the paper's biggest innovation. Instead of just giving a score, the system uses a mathematical rule (called Conformal Prediction) to decide exactly where to put a Red Sticker.

The Rule: The doctor can set a safety limit. For example, "I want to be 95% sure that if I see a Red Sticker, it means there is a problem. And I want to make sure I don't miss more than 5% of the bad spots."
The Result: The system looks at the Inspector's scores and places Red Stickers on the blurry, hallucinated, or noisy parts of the image.
The Outcome: The doctor sees the sharp image, but the dangerous parts are covered by a semi-transparent blue mask (the "Failure Mask"). They know: "I can trust the clear parts, but I should ignore the parts under the blue mask."

Why This is Special

It's Fast: The Inspector is very lightweight. It doesn't slow down the surgery. It works in real-time, like a co-pilot.
It's Honest: Unlike other AI safety tools that might just say "I'm 80% confident," this system gives a mathematical guarantee. It promises, "If you set the safety level to X, we will catch at least Y% of the errors." It's like a contract between the AI and the doctor.
It's Flexible: The doctor can choose how strict they want to be.
- Super Strict: "Mark everything that looks even slightly weird." (Lots of blue masks, very safe, but you see less of the image).
- Relaxed: "Only mark the really obvious mistakes." (Fewer blue masks, more image visible, but slightly higher risk).

The Bottom Line

This paper introduces a way to make AI image enhancers safe for surgery. It doesn't just make the picture prettier; it acts as a safety net, highlighting the areas where the AI might be lying or guessing. It turns a "black box" AI into a trustworthy partner that tells the surgeon exactly where to look and where to be careful.

In short: It's the difference between a magic trick that might fool you, and a high-tech microscope that tells you, "Here is what is real, and here is where I'm not sure."

1. Problem Statement

Minimally invasive and robotic surgeries rely heavily on high-quality imaging for safety and precision. However, clinical hardware constraints often limit the resolution of endoscopic cameras. While Deep Learning-based Super-Resolution (SR) models can enhance these images, they introduce significant safety risks:

Hallucinations: SR models may generate plausible but non-existent anatomical structures.
Amplified Noise: Artifacts and noise can be exaggerated, leading to misdiagnosis.
Lack of Reliability Metrics: Current uncertainty quantification methods (e.g., Monte Carlo dropout, ensembles) are often too computationally heavy for real-time surgery or do not provide rigorous, distribution-free guarantees suitable for clinical decision-making.
The Gap: There is a need for a framework that not only enhances resolution but also identifies where the reconstruction is likely to fail, providing theoretical guarantees on safety.

2. Methodology

The authors propose a model-agnostic, bi-level conformal risk control framework that integrates a lightweight error-prediction network with Conformal Prediction (CP) theory to generate Conformal Failure Masks (CFMs).

A. Reconstruction Error Network (REN)

Architecture: A lightweight auxiliary network (2 blocks, 64 features, ~70k parameters) that operates in parallel with the main SR model.
Input: It takes intermediate feature representations ( $F$ ) from the encoder of a frozen pre-trained SR model.
Output: A pixel-wise continuous error map ( $\hat{E}$ ) estimating the squared error between the SR output and the ground truth.
Efficiency: Designed to be low-latency, allowing real-time deployment without significantly impacting the SR inference time.

B. Bi-Level Conformal Risk Control

The core innovation is transforming continuous error estimates into binary "trust/don't trust" decisions with theoretical guarantees.

Defining Failure Levels ( $\tau^{fail}$ ): The user defines a specific quality threshold (e.g., PSNR $\le$ 22 dB) that constitutes a "failure." This converts the continuous regression problem into a binary classification sub-task.
Operational Thresholding: The REN's error scores are binarized using a threshold $\tau$ to create a preliminary mask.
Calibration & FNR Control: Using a separate calibration dataset, the system calculates the False Negative Rate (FNR). The goal is to ensure that the rate of unflagged failures (pixels where the SR is bad but the mask says "trust") does not exceed a user-defined risk level ( $\alpha$ ).
Conformal Failure Mask (CFM): The final mask is generated by rejecting pixels where the estimated error exceeds the calibrated threshold $\tilde{\tau}$ $\tilde{τ}$ .
- Guarantee: The method provides a theoretical guarantee that for any new data exchangeable with the calibration set, the miscoverage (unflagged failures) will be $\le \alpha$ .

3. Key Contributions

First Theoretically Grounded SR Safety Framework: This is the first study to provide a model-agnostic approach with distribution-free guarantees for real-time endoscopic SR.
Bi-Level Control: Unlike previous methods that only control the maximum error or the coverage of the mask, this framework allows clinicians to independently control:
- The permissible error level (e.g., "I only trust pixels with PSNR > 24 dB").
- The miscoverage rate (e.g., "I accept that at most 5% of bad pixels will be missed").
Operational Failure Masks (CFM): Instead of vague heatmaps, the system produces binary masks explicitly highlighting regions that should not be trusted, directly supporting clinical decision-making.
Lightweight Design: The error prediction module is computationally negligible, making it suitable for real-time surgical pipelines.

4. Experimental Results

The framework was evaluated on two distinct datasets:

SurgiSR4K: Robotic surgery images (2× and 4× upscaling).
HyperKvasir: Gastrointestinal diagnostic videos (4× upscaling with noise).
Models Tested: SwinIR (Single-image) and BasicVSR (Video).

Key Findings:

Error Detection: The Reconstruction Error Network achieved low False Positive Rates at 95% True Positive Rate (FPR95), effectively distinguishing between good and bad reconstructions.
Quality Improvement: By masking out unreliable regions, the effective PSNR of the remaining (trusted) image increased significantly (e.g., +7.0 dB improvement on HKvasir).
Risk Control: The empirical False Negative Rates closely matched the target $\alpha$ values (0.05 and 0.10), validating the theoretical guarantees.
Comparison: Compared to a recent state-of-the-art method (Adame et al.), the proposed CFM is less conservative. While the competitor produces large masks to guarantee safety (often masking the whole image), the proposed method allows for flexible trade-offs between mask size and safety guarantees.
Data Efficiency: The error network requires very little training data (a few videos) to perform optimally, even when reusing data from the SR model's validation set.

5. Significance and Impact

Clinical Safety: This work bridges the gap between high-performance AI and clinical safety requirements. It moves SR from a "black box" enhancement tool to a "trustworthy" system where clinicians know exactly which parts of the enhanced image are reliable.
Regulatory Readiness: The use of Conformal Prediction provides the mathematical rigor often required for the regulatory approval of AI in medical devices.
Practical Deployment: By ensuring the method is lightweight and model-agnostic, it can be integrated into existing surgical workflows without requiring hardware upgrades or massive computational resources.
Future Direction: The authors note that while the framework guarantees control over missed failures, the quality of the mask still depends on the accuracy of the error predictor. Future work will focus on improving these predictors under real-time constraints and relaxing the exchangeability assumption.

In summary, this paper presents a robust, mathematically rigorous solution to the "hallucination problem" in medical image super-resolution, enabling safer adoption of AI in minimally invasive surgery.