🤖 machine learning

Conformal Risk Minimization for Semi-Supervised Domain Adaptation via Optimal Transport

This paper proposes an end-to-end framework for Semi-Supervised Domain Adaptation that integrates Conformal Risk Minimization with Optimal Transport to generate pseudolabels, enabling the training of models that produce compact, coverage-valid prediction sets despite limited labeled target data.

Original authors: Manos Giannopoulos, Yi Shen, Michael M. Zavlanos

Published 2026-08-25

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Manos Giannopoulos, Yi Shen, Michael M. Zavlanos

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the high-stakes world of medical diagnosis, a machine learning model is often trained on data from one group of patients but must be used to diagnose a different group. This shift in population creates a hidden gap: the model might recognize patterns it has never seen before, leading to errors that could harm a patient. To build trust, doctors need more than just a single guess; they need to know how sure the model is. A method called conformal prediction offers a solution by providing a list of possible diagnoses rather than a single one. This list is guaranteed to contain the correct answer a specific percentage of the time, giving clinicians a safety net. However, a major problem arises when these lists are generated after the model has already been trained. Because the training process did not consider how these lists would be formed, the resulting lists are often far too large to be useful, containing so many possibilities that a doctor cannot make a decision.

Researchers at Duke University have developed a new approach to fix this issue, specifically for situations where labeled data for the new patient group is scarce. They created a system that teaches the model to think about the size of its future diagnosis lists while it is still learning. By combining a technique that aligns different patient populations with a method that optimizes for smaller, more precise lists, they built a model that produces compact, reliable sets of potential diagnoses. Their work suggests that by planning for uncertainty during the training phase rather than adding it on afterward, medical AI can become both more accurate and more practical for real-world use.

The core challenge the team addressed is known as semi-supervised domain adaptation. Imagine a doctor who has studied thousands of skin images from one hospital but must now diagnose patients at a different hospital where only a handful of images are labeled. The new hospital's patients might have different skin tones, lighting conditions, or equipment, creating a "distribution shift" that confuses the model. Traditional methods try to bridge this gap by teaching the model to ignore the differences between the two hospitals and focus only on the features that matter for diagnosis. However, these methods usually aim only for the single most likely answer. They do not care about the size of the safety net the model will eventually provide.

When researchers apply conformal prediction to these pre-trained models as a final step, the safety net often becomes a net so wide it catches everything. This happens because the model was never trained to minimize the number of options it offers. The Duke team realized that to get a useful list, the model must be trained with the list in mind from the very beginning. They proposed a framework called CP-JDOT, which integrates the goal of creating small, efficient lists directly into the training process. This allows the model to learn a representation of the data that naturally leads to tighter, more confident predictions.

To make this work with very few labeled examples from the target hospital, the researchers used a mathematical tool called optimal transport. Think of this tool as a way to match every unlabeled patient from the new hospital with a similar patient from the old hospital, effectively borrowing the old patient's diagnosis to guide the new one. This creates a set of "pseudolabels" that act as a bridge, allowing the model to learn from the abundant data of the source hospital while adapting to the specific needs of the target hospital. The system then uses the small number of actual labeled examples from the target hospital to set a threshold for how large the prediction lists should be.

The researchers tested their method on two different scenarios. First, they used a standard benchmark for object recognition involving images from four different domains: art, clipart, products, and real-world photos. They simulated the difficult condition where only one or three labeled examples per category were available for the target domain. In these tests, their method consistently produced smaller prediction lists than the best existing methods, reducing the average number of options by roughly five to ten percent. This improvement was significant even in cases where the transfer between domains was difficult, suggesting that the approach is robust.

They also applied the method to a more critical medical task: classifying skin lesions. Using data from the HAM10000 dataset as the source, they tested the model on five different target datasets representing various skin imaging conditions. Here, the results were even more striking. The new method reduced the average size of the prediction lists by up to twelve percent compared to previous approaches. This means that for a doctor looking at a skin lesion, the model would offer a much shorter list of potential conditions, making the diagnosis faster and more actionable, while still maintaining the rigorous guarantee that the correct diagnosis is included in the list.

Beyond simply making the lists smaller, the researchers showed that their framework could be tuned to respect specific medical rules. In skin cancer diagnosis, it is dangerous to have a list that includes both benign and malignant possibilities simultaneously, as this creates a confusing signal for the doctor. The team modified their training loss to penalize such contradictory lists. When they did this, the number of confusing, mixed lists dropped dramatically, by sixty to seventy-five percent across the different datasets. The trade-off was a very slight increase in the overall size of the lists, but the result was a set of predictions that were not only statistically valid but also semantically coherent and clinically useful.

The findings suggest that the traditional way of training AI models—optimizing for a single correct answer and then trying to fix the uncertainty afterward—is fundamentally limited. By integrating the goal of efficient uncertainty quantification directly into the learning process, the model learns to produce outputs that are inherently better suited for high-stakes decision-making. The researchers demonstrated that it is possible to train a model that is aware of its own limitations and the structure of its future predictions, even when data is scarce. This approach offers a promising path toward medical AI that is not only accurate but also trustworthy and aligned with the practical needs of clinicians.

Technical Summary: Conformal Risk Minimization for Semi-Supervised Domain Adaptation via Optimal Transport

1. Problem Definition

In high-stakes clinical applications, machine learning models often face a distribution shift when trained on a source patient population and deployed on a target population. While Semi-Supervised Domain Adaptation (SSDA) addresses this by leveraging labeled source data and limited labeled target data to improve point-prediction accuracy, existing SSDA methods fail to provide principled uncertainty quantification.

Conventional Conformal Prediction (CP) offers distribution-free coverage guarantees but is typically applied post-hoc to pre-trained models. This approach often yields prohibitively large prediction sets because the underlying model was optimized for point-accuracy, not for the geometry of nonconformity scores that dictates set size. While Conformal Risk Minimization (CRM) integrates set-size optimization directly into training, it traditionally requires large labeled datasets to compute nonconformity thresholds during training—a resource scarce in the SSDA regime.

The core problem addressed is: How can we optimally train conformal predictors for semi-supervised domain adaptation tasks where labeled data is limited in the target domain?

2. Methodology: CP-JDOT

The authors propose CP-JDOT, an end-to-end framework that integrates Conformal Risk Minimization into a Joint Domain Adaptation (JDOT) framework using Optimal Transport (OT). The method is designed to optimize a predictive model $f_\theta$ (comprising an embedding function $g$ and a classifier $h$ ) to produce compact, coverage-valid prediction sets on the target domain.

2.1 Core Components

The training objective minimizes a composite loss function:
$\min_{\theta} L_{cls}(f_\theta) + \lambda_1 L_{DA}(f_\theta) + \lambda_2 L_c(f_\theta)$
Where:

$L_{cls}$ : Classification loss (adapted for cross-domain learning).
$L_{DA}$ : Domain alignment loss.
$L_c$ : Conformal alignment loss (penalizing large prediction sets).

2.2 The Training Loop

The algorithm operates in an iterative stochastic manner using two minibatches: one from the labeled source set ( $S$ ) and one from the unlabeled target set ( $T_u$ ).

Threshold Estimation: At each iteration, a differentiable $(1-\alpha)$ -quantile $\tau$ of the nonconformity scores is computed using the small labeled target set ( $T_l$ ). This $\tau$ serves as the conformal threshold.
Optimal Transport Coupling: The framework uses Joint Domain Adaptation to map source and target domains into a shared latent space. It employs Optimal Transport (OT) to compute a coupling $\gamma$ between source samples and unlabeled target samples. This coupling transfers source labels to target instances as pseudolabels, providing the necessary supervision signal for the unlabeled target data.
Loss Optimization: The model is updated via stochastic gradient descent to minimize a total loss that includes:
- Source ERM: Standard cross-entropy on labeled source data to stabilize training.
- Domain Alignment ( $L_{DA}$ ): Uses Geometric Optimal Transport (GOT) to align feature distributions between source and target.
- Conformal Classification ( $L_{cls}$ ): A differentiable penalty ensuring that the conformal set for a target sample includes the OT-propagated source label.
- Conformal Efficiency ( $L_c$ ): A penalty term minimizing the expected cardinality (size) of the prediction sets for unlabeled target samples.

2.3 Domain-Specific Constraints

The framework supports the incorporation of domain knowledge by modifying the classification loss to a control loss ( $L_{control}$ ). This allows for penalties on specific set compositions. For example, in skin lesion classification, the loss can penalize prediction sets that simultaneously include mutually exclusive classes (e.g., benign and malignant diagnoses), ensuring semantic coherence even if it slightly increases set size.

3. Key Contributions

CP-JDOT Framework: A novel SSDA training framework that integrates CRM within a Joint Domain Adaptation with Optimal Transport setting. It bridges the gap between supervised CRM and SSDA by leveraging OT-derived pseudolabels to calibrate conformal predictors in a globally aligned feature space using only a small labeled target set.
Controlled Set Composition: The method demonstrates that incorporating the conformal objective directly into the training loop enables the use of penalty terms tailored to domain-specific requirements. This allows for the control of set composition (e.g., mitigating "coverage confusion" by excluding conflicting classes), yielding prediction sets that are statistically valid and semantically meaningful.
Empirical Superiority: Rigorous analysis shows that the end-to-end approach consistently outperforms state-of-the-art SSDA methods followed by post-hoc conformal wrappers. The method achieves smaller average prediction set sizes while preserving coverage guarantees.

4. Experimental Results

The authors evaluated CP-JDOT on two benchmark settings:

Office-Home: A standard domain adaptation benchmark with 12 pairwise transfer directions (Art, Clipart, Product, Real World) using ResNet-34 in 1-shot and 3-shot settings.
Skin Lesion Classification: A medical imaging setting transferring from HAM10000 (source) to five target domains (PH2, Derm7pt, MSK, SONIC, UDA) using ResNet-18 with 10 labeled target samples per class.

Key Findings:

Set Size Reduction: CP-JDOT consistently achieved the smallest average prediction set size across all transfer directions, outperforming the best-performing baselines (e.g., MME, CDAC, DECOTA) by approximately 5–10% on Office-Home and up to 12% on skin lesion datasets.
Accuracy vs. Efficiency: Notably, CP-JDOT achieved these improvements even when it had up to 10% lower top-1 accuracy than the strongest SSDA baselines. This supports the hypothesis that conventional accuracy objectives are misaligned with downstream conformal set-size efficiency.
Semantic Control: In the skin lesion ablation study, replacing the standard loss with $L_{control}$ reduced the proportion of "conflicting sets" (containing both benign and malignant classes) by 60–75% across datasets, with only a modest increase in average set size.

5. Significance and Claims

The paper claims that CP-JDOT provides a useful foundation for developing domain-adaptive models where uncertainty quantification is not treated as a separate post-processing step. By jointly optimizing domain adaptation and conformal objectives, the framework produces prediction sets that are:

Compact: Significantly smaller than those generated by post-hoc calibration of accuracy-optimized models.
Coverage-Valid: Maintaining rigorous distribution-free guarantees.
Clinically Actionable: Capable of incorporating domain-specific constraints (e.g., excluding contradictory diagnoses) directly into the training process.

The authors emphasize that this approach is particularly critical in high-stakes settings like healthcare, where the composition of a prediction set (e.g., avoiding a mix of benign and malignant diagnoses) is as important as its size for clinical decision-making.