RARE disease detection from Capsule Endoscopic Videos based on Vision Transformers
This paper presents a fine-tuned Google Vision Transformer (ViT) model for the multi-label classification of 17 gastrointestinal conditions in capsule endoscopic videos as part of the Gastro Competition, though it achieved a low overall mAP of approximately 0.02 on the test dataset.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine your digestive system as a 9-meter-long, narrow tunnel (about the width of a soda can) that food travels through. For years, doctors have had a hard time inspecting this tunnel because it's so long and cramped.
To solve this, patients now swallow a tiny, pill-sized camera (a Capsule Endoscope) that takes about 50,000 photos as it crawls from your mouth to your exit. It's like sending a spy drone through a dark cave, but the photos are often blurry, covered in bubbles, or blocked by food debris.
The Challenge: Finding the Needle in the Haystack
A doctor looking at these 50,000 photos would need to spot two things:
- Where are we? (Is the camera in the stomach? The small intestine? The colon?)
- What's wrong? (Is there bleeding? A polyp? An ulcer?)
The problem is that some things (like the "colon") appear in thousands of photos, while rare, dangerous things (like a specific type of bleeding or a rare valve) might only appear in a handful of photos. It's like trying to teach a student to recognize a rare bird that only shows up once a year, while they are flooded with pictures of pigeons.
The Solution: The "Super-Brain" AI
The team MDXBrain (from Middlesex University) built an AI to help doctors. Instead of looking at photos one by one like a human, they used a Vision Transformer (ViT).
Think of a Vision Transformer as a super-organized librarian who doesn't just read one book at a time. Instead, it looks at the whole library, understands how different books relate to each other, and instantly spots patterns. They "fine-tuned" this librarian to look at the capsule camera photos.
The Big Hurdle: The "Unbalanced Class" Problem
The biggest issue was that the data was heavily unbalanced.
- The Pigeon Problem: There were nearly 2 million photos of the "colon."
- The Rare Bird Problem: There were only 122 photos of the "z-line" (a specific part of the esophagus).
If they trained the AI on all the data, the AI would just learn to say "Colon" every time because it's the most common thing. It would ignore the rare, dangerous stuff.
How they fixed it: The "Under-Sampling" Strategy
Imagine you are making a soup. You have a giant bucket of potatoes (common data) and a tiny jar of truffles (rare data). If you dump the whole bucket in, the soup tastes like potatoes.
- What they did: They threw away most of the potatoes and kept only a few, so the number of potatoes matched the number of truffles.
- The Twist: They were very careful to keep the "special" potatoes—the ones that had both potatoes and truffles (images with multiple labels). They kept every single one of those rare, complex images to teach the AI how to spot multiple problems at once.
The Result: A Learning Curve
After training this AI, they tested it on three new video capsules.
- The Score: The AI achieved a score of roughly 0.02 (on a scale where 1.0 is perfect).
- What this means: In the world of AI competitions, this is a very low score. It's like a student who studied hard but still got a 2% on the final exam.
Why was the score so low?
The paper admits the results were poor. This is likely because:
- The images are terrible: The capsule photos are low-resolution and often blurry.
- The task is incredibly hard: Finding a tiny ulcer in a blurry, moving video is like finding a specific grain of sand on a beach while wearing foggy goggles.
- The "Rare" nature: By definition, rare diseases are hard to find because there is so little data to learn from, even with their balancing tricks.
The Takeaway
This paper is a "work in progress" story. The team built a sophisticated AI system (the Vision Transformer) and used clever math to balance the data (the under-sampling). While the final score wasn't a winner yet, the methodology is solid. They proved that you can build a system to look for rare diseases in these difficult videos, even if the system needs more training and better data to become truly helpful for doctors.
It's a reminder that in medicine, AI is a powerful assistant, but it's still learning the ropes.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.