⚡ electrical engineering

TRUE-Colon: Exposing a Consistent Transfer Asymmetry in Real-Time Polyp Detection

The paper introduces TRUE-Colon, a benchmarking protocol demonstrating that real-time polyp detection models trained on curated, lesion-centric clips suffer severe performance collapse in real-world full-procedure settings, whereas models trained on full procedures maintain high accuracy while effectively rejecting non-polyp content, thereby advocating for a shift toward full-procedure data and deployment-relevant metrics in CADe development.

Original authors: Sebastian Doerrich, Andreas Franz Schwab, Francesco Di Salvo, Shyam Nandan Rai, Hanh Huyen My Nguyen, Christian Ledig

Published 2026-08-17

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Sebastian Doerrich, Andreas Franz Schwab, Francesco Di Salvo, Shyam Nandan Rai, Hanh Huyen My Nguyen, Christian Ledig

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to find a specific, rare type of lost toy hidden inside a massive, chaotic warehouse. The warehouse is full of moving conveyor belts, flickering lights, piles of junk, and long stretches of empty floor where nothing interesting is happening. Now, imagine that to train your detective, you only show them a photo album. But here's the catch: the photo album only contains perfect, zoomed-in pictures of the lost toys, surrounded by clean white backgrounds. There are no pictures of the conveyor belts, no shadows, and no empty floor. You might think your detective is a genius because they can spot the toy in the photo album instantly. But the moment you send them into the real, messy warehouse, they might get overwhelmed by the junk, miss the toy because it's partially hidden, or start screaming "Found it!" every time they see a piece of trash that looks vaguely like a toy.

This is exactly the problem facing a field called "Computer-Aided Detection" (CADe) in medicine, specifically for colonoscopies. A colonoscopy is a procedure where a doctor uses a camera to look inside a person's colon to find and remove small growths called polyps before they turn into cancer. The goal is to find these polyps in real-time while the camera is moving through a long, twisting, and often dirty tunnel. For years, scientists have been building AI "detectives" to help doctors. They train these AIs on datasets—collections of images and videos—that are carefully curated. This means the data is "cleaned up" to focus mostly on the polyps, often cutting out the boring parts where nothing is there. The big question this paper asks is: If we train our AI detectives only on these clean, perfect photo albums, will they actually work when they have to patrol the messy, real-life warehouse of a full medical procedure?

The Great "Photo Album" vs. "Real Warehouse" Experiment

In this study, a team of researchers from the University of Bamberg decided to test this exact scenario. They called their new testing ground TRUE-Colon. Think of TRUE-Colon as a standardized "stress test" for these AI detectives. Instead of just checking if the AI can find a polyp in a single, perfect picture, TRUE-Colon measures four things that actually matter in the real world:

Can it find the polyp? (Localization accuracy)
Does it cry wolf too often? (False-alert burden: how many times does it scream "Polyp!" when it's just seeing a fold in the skin or a speck of dirt?)
How fast does it react? (Detection latency: does it spot the polyp the second it appears, or does it take a few seconds to catch up?)
Does it keep watching? (Temporal reliability: once it sees a polyp, does it keep tracking it, or does it blink and lose it?)

The researchers took four popular AI models (Faster R-CNN, YOLOv8, YOLOv11, and RT-DETR) and put them through two different training camps. One camp used the old-school "photo albums" (curated datasets like SUN and PICCOLO), which are full of polyp pictures but very few "empty" frames. The other camp used the "real warehouse" (REAL-Colon), which consists of 60 full, unedited colonoscopy videos. These real videos are mostly empty space (over 85% of the frames have no polyps) and are full of messy artifacts like motion blur, bright reflections, and surgical tools.

The Shocking Discovery: The "Transfer Asymmetry"

Here is the big reveal, and it's a bit of a plot twist. The researchers found a consistent transfer asymmetry. It's like a one-way street.

When they trained the AI models on the clean photo albums (curated data) and then tested them on the messy real videos, the models completely collapsed. They got terrible at finding polyps and, worse, they started hallucinating. They would scream "Polyp!" at almost every piece of debris or shadow. For example, a model called YOLOv11, which was a star performer on the clean photo albums (getting a score of 0.724), dropped to a dismal 0.164 when faced with the real videos. It was as if the detective, trained only on clean photos, couldn't handle the chaos of the real world at all.

However, the reverse was true! When they trained the models on the messy real videos (REAL-Colon) and then tested them back on the clean photo albums, the models didn't just survive; they thrived. They kept their high accuracy on the clean pictures but, crucially, they learned how to ignore the junk in the real videos. They became much better at saying "No, that's just a shadow" instead of "Polyp!"

This proves that training on the "messy" full procedures is actually the better way to prepare an AI for the real world. The "clean" photo albums were giving the AI a false sense of security, creating an "illusion of success" that vanished the moment the real work began.

The Race Between the Detectives

Once they fixed the training method, the researchers compared the four AI models head-to-head on the real videos, but with a fair rule: they adjusted the settings so that every model was allowed to make the same number of "false alarms" (about 4–5% of the time). This ensured they were comparing their actual skill, not just how "loud" their confidence settings were.

The Transformer Detective (RT-DETR): This model was the most sensitive and the fastest. It found polyps earlier than the others and kept tracking them for longer. It was like a detective with eagle eyes who never blinked. However, it required more computing power to do this.
The Convolutional Detectives (YOLOv8 and YOLOv11): These models were slightly slower to spot the polyp and didn't track them quite as persistently, but they were incredibly fast at processing the video. They were like a team of detectives who could scan the room much quicker, trading a tiny bit of tracking perfection for raw speed.

The paper suggests that there isn't one single "winner." Instead, it's a trade-off: if you need the absolute earliest detection and don't mind using more computer power, the Transformer (RT-DETR) is the best. If you need to process video very quickly and can tolerate a tiny bit of delay, the YOLO models are great competitors.

The Bottom Line

The main takeaway from this research is a warning to the medical AI community: Stop training and testing your AI only on clean, curated clips. If you do, you are building a detective that only works in a photo studio, not in a hospital. To build a system that actually helps doctors and saves lives, you must train and test on full, unedited procedures that include all the boring, messy, and empty parts of the exam. Only then can you truly know if your AI is ready to be a safety net for patients, or if it's just going to cause a panic by crying wolf at every speck of dust.

The authors are quite sure about this asymmetry because they tested it across multiple models and datasets, but they also note that their study has limits. For instance, they only looked at medium-to-large polyps (the AI struggled with tiny ones), and their test group of polyps was relatively small. They suggest that the next step is to test this on even more diverse groups of patients and different hospital settings to make sure this "messy training" rule holds up everywhere. But for now, the message is clear: real-world messiness is not a bug; it's the most important feature for training a reliable AI.

Technical Summary: TRUE-Colon: Exposing a Consistent Transfer Asymmetry in Real-Time Polyp Detection

Problem Statement
Despite the promise of Computer-Aided Detection (CADe) systems to reduce clinical miss rates in colonoscopy, reliable real-world deployment remains elusive. The authors identify a structural flaw in current model development: the reliance on curated, lesion-centric datasets (e.g., SUN, PICCOLO) that under-represent the long stretches of negative frames and procedure-related artifacts (specular highlights, motion blur, debris) characteristic of routine clinical examinations. Evaluating architectures strictly on these benchmarks creates an "illusion of success" by concealing the false-alert burden triggered in continuous video. Furthermore, the academic literature suffers from methodological inconsistencies, utilizing disjoint protocols and non-standardized data splits that complicate the comparison of clinically relevant behaviors beyond simple localization accuracy.

Methodology
To address these gaps, the authors establish TRUE-Colon, a standardized benchmarking protocol evaluated on REAL-Colon, a dataset comprising 60 full, unedited colonoscopy procedures. The study evaluates four real-time object detection architectures: Faster R-CNN, YOLOv8, YOLOv11, and RT-DETR.

The evaluation framework moves beyond standard instance-level metrics to define four complementary characteristics critical for deployment:

Localization Accuracy: Measured via mAP50 and mAP50:95 using standard Intersection-over-Union (IoU) criteria.
False-Alert Burden: Quantified via the False Positive Rate (FPR) at the frame level, collapsing detector outputs into binary alert decisions.
Detection Latency: Measured as the number of frames from a lesion's annotated onset to the first valid detection.
Temporal Reliability: Assessed by the persistence of detections (valid detections occurring in at least 50% of a lesion's visible frames).

The study employs a cross-distribution transfer analysis across four settings: in-domain curated, curated-to-procedure, in-domain procedure, and procedure-to-curated. Crucially, to ensure fair comparison across architectures with different score calibrations, the authors evaluate models at a matched false-alert operating point ( $\tau^*$ ), where thresholds are adjusted post-hoc to achieve a comparable frame-level FPR (approximately 4–5%) on the REAL-Colon test set.

Key Results
The experiments reveal a consistent transfer asymmetry:

Curated-to-Procedure Failure: Models trained strictly on curated clips (SUN or PICCOLO) suffer a severe performance collapse when evaluated on full procedures. For instance, YOLOv11's mAP50 drops from 0.724 on SUN to 0.164 on REAL-Colon. Even more critically, models trained on the highly curated PICCOLO dataset degrade further (mAP50 = 0.059), indicating that lesion-centric benchmarks fail to provide sufficient exposure to non-polyp content.
Procedure-to-Curated Robustness: Conversely, models trained on full procedures (REAL-Colon) substantially improve their rejection of non-polyp content and largely retain their accuracy when transferred back to curated benchmarks (e.g., YOLOv11 trained on REAL-Colon achieves mAP50 = 0.705 on SUN, matching its in-domain performance).
Architecture Comparison at Matched Operating Points: When evaluated at the matched false-alert operating point ( $\tau^*$ $τ^{*}$ ):
- RT-DETR (Transformer-based) demonstrates the strongest sensitivity, the earliest detection (lowest mean latency of 38.3 frames), and the highest temporal persistence.
- Convolutional detectors (YOLOv8, YOLOv11) remain competitive in sensitivity but offer substantially higher throughput, highlighting an accuracy-versus-compute trade-off that fixed-threshold comparisons obscure.

Significance and Claims
The paper claims that the current paradigm of training and benchmarking CADe systems on curated, lesion-centric clips is insufficient for deployment. The authors argue that full-procedure data are the more informative training signal, as they capture the variability and prevalence of negative frames necessary for robust clinical performance.

The study concludes that deployment-relevant evaluation must shift from isolated localization accuracy to metrics that account for false-alert burden, detection latency, and temporal reliability. The authors emphasize that score calibration varies by architecture; therefore, comparing detectors at a fixed confidence threshold is misleading. Instead, comparisons must be made at a matched false-alert operating point to isolate true performance differences. Finally, the paper posits that while current detectors are memoryless and frame-based, the next step toward lower latency and steadier detections lies in incorporating temporal modeling to exploit inter-frame consistency.

Limitations
The authors modestly acknowledge several constraints: the transfer asymmetry may be confounded by differences in dataset scale and annotation protocols rather than realism alone; cross-architecture comparisons reflect default framework recipes rather than a controlled architectural study; and the lesion-level evidence is derived from a single cohort ( $n=21$ test lesions). They also note that their frame-level false-alert metric weights every alerting negative frame equally, potentially differing from the clinical experience of event-based interruptions.

The Great "Photo Album" vs. "Real Warehouse" Experiment

The Shocking Discovery: The "Transfer Asymmetry"

The Race Between the Detectives

The Bottom Line

Technical Summary: TRUE-Colon: Exposing a Consistent Transfer Asymmetry in Real-Time Polyp Detection

More like this