💻 computer science

CUPID-Med: Conformal Uncertainty-aware Prototype Discovery for Clinically Interpretable Deep Medical Clustering

CUPID-Med is a novel framework that leverages conformal uncertainty-aware prototype discovery to generate clinically interpretable, set-valued cluster predictions with calibrated confidence scores, significantly outperforming deterministic baselines in structuring unlabeled medical image data.

Original authors: Mojtaba Jahanian, Abbas Karimi, Nafiseh Osati Eraghi, Faraneh Zarafshan

Published 2026-07-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mojtaba Jahanian, Abbas Karimi, Nafiseh Osati Eraghi, Faraneh Zarafshan

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a librarian trying to organize a massive, chaotic library of 5,000 medical X-ray images. The problem? None of the books have titles or labels on their spines. You don't know which X-ray shows a broken bone, which shows pneumonia, and which is perfectly healthy. You just have a giant pile of images.

Most computer programs trying to solve this would act like a very rigid librarian. They would look at an image, force it into one single shelf, and say, "This goes in the 'Pneumonia' bin." But here's the catch: in medicine, things are often messy. An X-ray might look a little bit like pneumonia, a little bit like fluid, and a little bit like a normal heart. A rigid computer might get it wrong and be very confident about being wrong.

Enter CUPID-Med.

Think of CUPID-Med not as a librarian, but as a skeptical, highly organized detective who uses a special set of tools to sort these images. Here is how it works, broken down into simple concepts:

1. The "Smart Eye" (Medical Foundation Encoder)

First, the system looks at the X-rays using a "smart eye" that has already been trained on millions of medical images. It's like giving the detective a pair of glasses that have seen every type of chest X-ray before. This helps the system understand the basic shapes and patterns without needing to be told what they are.

2. The "Fuzzy Cloud" (Distributional Learning)

Here is where CUPID-Med is different from the old, rigid methods.

Old Method: It treats every X-ray as a single, sharp dot on a map. "This dot is definitely in the 'Pneumonia' group."
CUPID-Med: It treats every X-ray as a fuzzy cloud. It realizes that some images are very clear (a small, tight cloud), while others are blurry and confusing (a huge, spread-out cloud).
- If the cloud is tight, the system is confident.
- If the cloud is huge and spread out, the system says, "I'm not sure what this is. It could be a few different things."

3. The "Representative Faces" (Prototype Discovery)

Instead of just creating abstract groups (like "Group A" and "Group B"), CUPID-Med creates Prototypes. Think of these as the "Face of the Group."

For every group it finds, it picks the best, most "average" looking X-ray to represent that group.
This is like a detective saying, "This group of patients all look like this specific person."
Doctors can look at these "Face of the Group" images and say, "Ah, yes, I see what you mean. These patients all have that specific shadow in their lungs." This makes the computer's work interpretable (easy for humans to understand).

4. The "Safety Net" (Conformal Uncertainty)

This is the most important safety feature. Because the system knows when it is confused (the "fuzzy cloud"), it has a rule: "If I'm not 100% sure, I won't force a decision."

Instead of saying, "This is Group A," it might say, "This could be Group A, OR it could be Group B."
It creates a shortlist of possibilities.
If the list is too long or the system is too unsure, it puts a red flag on that image and says, "Hey, a human doctor needs to look at this one." It refuses to guess when the stakes are high.

5. The Results: Sorting the Chaos

The researchers tested this on a dataset of 5,000 chest X-rays from the CheXpert database. They didn't tell the computer what the diseases were during the training; they just let it sort the images on its own.

The Old Way: When they tried to sort these images with standard computer methods, the groups were messy. The images in each group didn't really look alike. It was like trying to sort a deck of cards where the "Hearts" pile had some "Spades" mixed in. The score for how well they were sorted was very low (0.05).
The CUPID-Med Way: When they used their new detective system, the groups became very distinct. The images in each pile looked very similar to each other. The score jumped up significantly (to 0.51).
The Bonus: The system also successfully identified which images were "fuzzy" (uncertain) and flagged them for review, just as it promised.

The Bottom Line

CUPID-Med is a new way to organize medical images that admits when it doesn't know the answer. Instead of forcing a "best guess" that might be wrong, it:

Groups similar images together clearly.
Shows doctors a "face" of each group so they can understand why the computer grouped them.
Flags the confusing cases so a human expert can step in.

The paper concludes that this approach creates much cleaner, more understandable groups than previous methods, but notes that the system still needs some fine-tuning to make sure the groups are all roughly the same size before it can be used in a real hospital.

Technical Summary of CUPID-Med: Conformal Uncertainty-aware Prototype Discovery for Clinically Interpretable Deep Medical Clustering

Problem Statement
Deep medical clustering aims to uncover latent disease subgroups from unlabeled medical data, a critical task given the high cost and variability of expert annotation. However, existing deep clustering approaches suffer from three primary limitations:

Deterministic Assignments: Most methods force samples into single, hard cluster labels, failing to account for the continuous, overlapping, and heterogeneous nature of clinical phenotypes.
Lack of Calibrated Uncertainty: Standard pipelines rarely provide calibrated uncertainty estimates or formal mechanisms to identify ambiguous cases that require expert review.
Limited Interpretability: Clustering results often lack clinically meaningful summaries, such as representative prototypes or visual explanations, making them difficult for clinicians to inspect and trust.

Furthermore, while medical foundation models provide powerful representations, their fixed embeddings may encode non-clinical artifacts (e.g., acquisition biases) and do not inherently support uncertainty-aware subgroup discovery.

Methodology: CUPID-Med
The authors propose CUPID-Med, a framework designed to transform fixed medical embeddings into calibrated, interpretable, and uncertainty-aware subgroups. The architecture consists of five core stages:

Medical Foundation Encoding: The framework utilizes medical foundation encoders (or pre-trained backbones like DenseNet121) to map raw inputs (images or multimodal data) into a unified representation. In the image-only setting (CheXpert), this involves patch embedding and Transformer-based processing or CNN feature extraction.
Distributional Latent Representation Learning: Instead of mapping samples to deterministic points, CUPID-Med represents each sample as a Gaussian distribution $N(\mu_i, \text{diag}(\sigma_i^2))$ . This is achieved via a projection head that outputs mean vectors and log-variances. The model employs self-supervised consistency losses (KL divergence between augmented views) and contrastive objectives to learn robust latent distributions. A scalar uncertainty score $u_i$ is derived from the latent variance.
Prototype-Based Deep Clustering: The framework maintains a learnable memory bank of $K$ $K$ prototypes representing latent disease subgroups. It optimizes a clustering objective that combines:
- Target distribution refinement (aligning soft assignments with a refined target).
- Prototype compactness (minimizing distance between samples and their assigned prototypes).
- Cluster balance (encouraging uniform prototype utilization).
- Prototype separation (maximizing distance between distinct prototypes).
Conformal Uncertainty Calibration: To address the need for reliable decision boundaries, CUPID-Med adapts conformal prediction to the unsupervised setting. It converts soft cluster assignments into set-valued predictions.
- A nonconformity score is calculated based on low assignment confidence and latent uncertainty.
- A threshold is estimated on a calibration set to guarantee distribution-free coverage.
- For a test sample, the output is a set of candidate clusters $\Gamma_\alpha(x)$ . If the set contains multiple clusters or the uncertainty is high, the sample is flagged for expert review rather than being forced into a single label.
Clinically Interpretable Outputs: The system generates "cluster cards" containing representative images (prototype-nearest samples), clinical enrichment profiles (if labels are available post-hoc), and uncertainty distributions. Visual explanations (e.g., Grad-CAM) are noted as future extensions, with the current focus on prototype retrieval.

Key Contributions

Framework Integration: The proposal of a unified framework combining medical foundation encoding, distributional latent learning, prototype discovery, and conformal calibration for medical clustering.
Uncertainty-Aware Assignment: The introduction of a conformal strategy that produces set-valued predictions, explicitly identifying ambiguous cases for human-in-the-loop review rather than providing deterministic labels.
Prototype-Driven Interpretability: A mechanism to retrieve representative samples for each cluster, enabling case-based reasoning and visual inspection of discovered subgroups.
No-Label Training Protocol: The definition of a rigorous evaluation protocol on the CheXpert dataset where disease labels are strictly excluded from training and used solely for post-hoc clinical enrichment and interpretability analysis.

Experimental Results
The framework was evaluated on a 5,000-image frontal subset of the CheXpert dataset using a no-label training protocol.

Baseline Performance: Preliminary clustering on fixed DenseNet121 embeddings using standard methods (K-means, GMM, Agglomerative) yielded weak separation. The best baseline (K-means, $K=5$ ) achieved a Silhouette score of 0.0510 and a Davies–Bouldin index of 3.6950.
CUPID-Med Performance: The primary CUPID-Med run ( $K=14$ ) achieved a Silhouette score of 0.5175, a Davies–Bouldin index of 2.7809, and a Calinski–Harabasz score of 29,733.16.
Secondary Run: A secondary 50-epoch run showed even higher internal metrics (Silhouette: 0.6216) but exhibited an empty-cluster warning, leading the authors to treat the 80-epoch run as the official reproducible result.
Observations: While CUPID-Med significantly improved latent separation compared to deterministic baselines, the primary run exhibited cluster-size imbalance (ranging from 2 to 1,132 samples per cluster), indicating that prototype balance regularization requires further refinement.

Significance and Claims
The paper claims that distributional prototype learning can produce more structured and interpretable medical image subgroups than deterministic clustering of fixed deep embeddings.

Shift in Paradigm: The work argues for moving from "opaque hard clusters" to "calibrated, interpretable, and uncertainty-aware subgroup discovery."
Clinical Utility: By generating set-valued predictions and flagging uncertain cases, the framework aligns better with clinical reality, where overlapping phenotypes and ambiguous findings are common.
Modesty on Deployment: The authors explicitly state that while the method demonstrates superior internal separation and interpretability, the observed cluster imbalance and the need for stronger balance regularization mean the method is not yet ready for stable clinical deployment as a phenotype discovery tool. The results are presented as a proof-of-concept for a more trustworthy, human-centric clustering workflow.