Discrimination transfers but calibration does not:1prevalence-dominated miscalibration of diabetic2retinopathy screening across fundus cameras

Imagine you are teaching a robot to spot a specific type of cloud in the sky. You show it thousands of photos taken from a high-tech weather station with a perfect lens. The robot learns to say, "That's a storm cloud!" with great confidence. But here is the tricky part: being able to spot the cloud is different from knowing how likely it is to rain. If the robot says, "I am 90% sure this is a storm," it needs to be right 90% of the time. If it's wrong, and you trust it too much, you might forget your umbrella on a rainy day, or you might carry one when the sky is perfectly clear.

This paper dives into a world where computers help doctors find eye diseases called diabetic retinopathy. It asks a very practical question: If we train a computer on photos taken with big, expensive cameras in a hospital, will it still be trustworthy when we take it to a village and use a small, handheld camera? The answer isn't just about whether the computer can see the disease (which it can), but whether its confidence makes sense in the new environment. The researchers found that while the computer's eyes stay sharp, its "gut feeling" about how sure it is gets completely scrambled, mostly because the disease is more common in the new place than in the old one.

---

The Story of the Confused Eye-Doctor Robot

Imagine you have a brilliant robot eye-doctor named "Dr. AI." You trained Dr. AI in a fancy, climate-controlled hospital using a giant, expensive tabletop camera. Dr. AI learned to look at photos of eyes and spot a disease called diabetic retinopathy. When Dr. AI sees a sick eye, it gives a score: "I'm 95% sure this is sick!" When it sees a healthy eye, it says, "I'm 99% sure this is fine."

Now, you want to take Dr. AI to a remote village to help people who don't have access to big hospitals. But in the village, you can't carry the giant camera. You have to use a tiny, handheld camera that fits in a backpack. The photos from this little camera look a bit different—they might be brighter, darker, or slightly blurry compared to the hospital photos.

The Big Question: When Dr. AI looks at these new village photos, will it still be a good doctor?

The researchers in this paper put Dr. AI to the test. They found two very different things happened:

1. The Eyes Stayed Sharp (Discrimination): Dr. AI was still excellent at telling the difference between a sick eye and a healthy one. Even without any extra training, it could still rank the sick eyes higher than the healthy ones almost as well as a doctor trained specifically on village photos. It didn't lose its ability to see the problem.
2. The Confidence Got Scrambled (Calibration): This is where things went wrong. Dr. AI started lying about how sure it was. If it said, "I'm 90% sure this eye is sick," it wasn't actually right 90% of the time. Its confidence scores were broken. In the village, the robot might be way too confident about healthy eyes or not confident enough about sick ones. This is dangerous because doctors rely on these confidence numbers to decide who needs to see a human specialist immediately.

Why Did the Confidence Break?

You might think, "Oh, the handheld camera must be the problem! The photos look so different." The researchers actually tested this idea, and they found something surprising.

They discovered that the camera itself was only a small part of the problem (about 20%). The real villain was the crowd.

Think of it like this: In the hospital where Dr. AI was trained, only 1 out of every 20 people had the disease (5.7%). But in the village, 1 out of every 6 people had it (17.6%). In another test group, nearly half the people had it (40.6%).

Dr. AI was trained in a world where the disease was rare. It learned to be very cautious. When it moved to a village where the disease was common, it didn't know how to adjust its "gut feeling." It's like a weather forecaster who grew up in a desert. If they move to a rainforest and say, "It's only a 10% chance of rain," they are wrong, not because their eyes are bad, but because they are used to a world where rain is rare. The researchers showed that about 80% of the confusion came from this change in how common the disease was, not from the camera taking the picture.

How to Fix the Robot

So, how do we fix Dr. AI's broken confidence? The researchers tried a few tricks:

• The "Temperature" Trick: They tried a simple math tweak called "temperature scaling." This is like turning a dial to make the robot's answers a bit softer or harder. It didn't work well because the robot's problem wasn't just about how "soft" its answers were; it needed to shift its whole baseline because the disease was more common.
• The "Re-Training" Fix: They tried two other methods called Platt scaling and isotonic scaling. These are like giving the robot a tiny, quick lesson. They showed the robot just 100 to 200 new photos from the village (labeled by humans) and asked it to adjust its confidence numbers.
  • The Result: It worked! With just a tiny bit of new data, Dr. AI's confidence became trustworthy again. It went from being wildly wrong to being almost as good as a robot trained from scratch on village photos.

The "Tiny Brain" Problem

There was one more twist. To run Dr. AI on a cheap, handheld device, you often have to shrink the computer program to make it fit. This is called "quantization."

• Half-Precision (FP16): This is like compressing a photo slightly. It made the program smaller but didn't hurt Dr. AI's eyes or its confidence. It was safe.
• 8-Bit Integer (INT8): This is like compressing the photo so much it becomes pixelated. The researchers found that this broke Dr. AI's eyes and its confidence. Even if they tried to fix the confidence with the "Re-Training" trick, they couldn't fully fix the damage to the eyes. The robot became less accurate, and no amount of quick math could bring it back.

The Takeaway

The main lesson from this study is that for AI to be safe in the real world, we can't just look at whether it can spot the disease. We have to check if its confidence makes sense in the new place.

If you are taking a medical AI from a rich hospital to a poor village, don't just assume it will work because it's smart. The disease might be more common there, and that will confuse the robot's confidence. But the good news is, you don't need a million new photos to fix it. You just need a small, cheap lesson (about 100–200 examples) and the right math trick to teach the robot how to trust its own feelings again. And if you are squeezing the program to fit on a tiny device, be careful: shrinking it too much might break the robot in ways you can't easily fix.