📊 statistics

Domain Adaptation Targeting Heterogeneous and Imbalanced Subgroups

This paper proposes a novel domain adaptation framework that simultaneously addresses high-dimensionality, covariate shift, and outcome model heterogeneity to effectively handle target data composed of heterogeneous, data-sparse, and imbalanced subgroups without gold-standard labels, thereby mitigating biases and unfairness in real-world applications like genetic risk modeling.

Original authors: Doudou Zhou, Mengyan Li, Yun Wang, Tianxi Cai, Molei Liu

Published 2026-07-14

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Doudou Zhou, Mengyan Li, Yun Wang, Tianxi Cai, Molei Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to predict the future, but you have a massive problem: the robot has never seen the specific type of future it needs to predict, and the data it does have is wildly unbalanced.

This is the story of a new method called AIMS (Adaptation to targets with Heterogeneous and Imbalanced Subgroups), developed by a team of statisticians. They built a clever framework to help machines learn from a "source" group of data to make predictions about a "target" group, even when the target group is a messy mix of different subgroups, and the most important subgroup is tiny and has no labels at all.

The Problem: The "Majority Rules" Trap

Think of the data as a giant classroom.

The Source: You have a textbook full of answers (labeled data) from a group of students who are mostly tall and wear blue shirts (the "majority").
The Target: You need to predict the test scores for a new class of students (the "target"). But this new class is a mix: most are tall and wear blue, but a small, rare group wears red and is very short.
The Catch: You have the textbook for the blue-shirted students, but for the red-shirted students in the new class, you have zero answer keys. You can't check their work.

If you just take the textbook from the blue-shirted students and apply it to the whole new class, the robot learns to be great at predicting for the blue shirts but fails miserably for the red shirts. Why? Because the red shirts might have different rules for how they learn, and they are so few in number that the robot ignores them. This is called negative transfer: borrowing too much from the majority actually hurts the minority.

The Solution: A Three-Step Detective Kit

The authors, Doudou Zhou, Mengyan Li, Yun Wang, Tianxi Cai, and Molei Liu, propose a three-step detective kit to solve this without needing to see the red-shirted students' answers first.

Step 1: The Double-Check (Covariate Shift Correction)
First, the robot notices that the blue-shirted students in the textbook (Source) look slightly different from the blue-shirted students in the new class (Target). Maybe the textbook is from 2015 and the new class is from 2024, or they use different language.
The method uses a "double-check" system. It tries to fix the differences in two ways at once:

Weighting: It gives more importance to the textbook examples that look like the new students.
Imputation: It guesses what the answers might be based on the patterns it sees.
Crucially, this step is doubly robust. This means if one of the guesses is wrong, the other one saves the day. The robot doesn't need both to be perfect; it just needs one to be right to get a solid starting point.

Step 2: The "Offset" Trick (Knowledge Transfer)
Now, the robot has a decent guess for the red-shirted students, but it's shaky because there are so few of them. The robot looks at the blue-shirted students in the new class (the majority). It knows the blue shirts are well-understood.
The robot asks: "How different are the red shirts from the blue shirts?"
Instead of trying to learn the red shirts from scratch, it assumes the red shirts are mostly like the blue shirts, with just a few small differences (a "sparse" difference). It learns the blue shirt pattern first, then only tries to learn the tiny differences for the red shirts. This is like learning a new language by starting with a language you already know and only memorizing the few words that are different.

Step 3: The Safety Net (Negative Transfer Protection)
Here is the tricky part. What if the red shirts are not like the blue shirts at all? What if the "difference" is huge? If the robot blindly copies the blue shirt rules, it will fail.
Usually, robots check their work by looking at the answer key. But remember, the red-shirted students have no answer key!
AIMS solves this by creating a "safety net." It splits the data and uses a clever mathematical trick to compare two versions of the prediction:

The "Red Only" guess (ignoring the blue shirts).
The "Red + Blue" guess (borrowing from the blue shirts).
The method calculates which one is likely better without seeing the real answers. If borrowing from the blue shirts makes things worse, the method automatically switches back to the "Red Only" guess. It protects the minority from being dragged down by the majority.

Does it actually work?

The authors didn't just dream this up; they put it to the test.

In the Lab (Simulations):
They ran thousands of computer simulations where they knew the "true" answer. They tested the method when the data was messy, the dimensions were high (lots of variables), and the minority group was tiny.

The Result: In these simulations, AIMS consistently beat other methods. It handled the "messy" data better and didn't crash when the minority group was small. The paper shows that when the difference between groups is small, AIMS learns faster. When the difference is big, AIMS safely ignores the majority and sticks to the minority, avoiding the "negative transfer" trap.

In the Real World:
The team tested AIMS on two real-world problems:

Type II Diabetes Risk: They tried to predict diabetes risk for non-White patients (the minority) using data from White patients (the majority). The data came from hospital records where the coding systems had changed over time.
- The Outcome: AIMS was the best at predicting risk. It didn't just rank patients correctly (which many methods did); it was also much better at predicting the actual probability of getting sick. Other methods were confident but wrong; AIMS was calibrated and reliable.
Protein Stability: They tried to predict how stable a protein would be after a mutation. The "majority" was proteins tested in neutral pH (6–7), and the "minority" was proteins tested in acidic pH (1–5).
- The Outcome: AIMS achieved the lowest error rates (RMSE of 2.67) compared to other methods. While other methods collapsed and just guessed the same number for everyone, AIMS found a pattern and made meaningful predictions.

What the Paper Rules Out

The authors are very clear about what their method is not.

It is not a magic wand that works if the minority group is completely unrelated to the majority. If the difference between the groups is too huge (dense, not sparse), the method has a safety switch to stop borrowing, but it can't magically invent a connection that doesn't exist.
It is not a method that requires you to have answer keys for the target group. If you have labels for the target minority, you don't need this complex setup; simpler methods work fine. The whole point is that you don't have those labels.
It is not a method that assumes the data is simple. It specifically handles high-dimensional data (where there are more variables than data points), which breaks many standard tools.

How Sure Are We?

The paper is quite confident, but it keeps its feet on the ground.

The Theory: They have mathematical proofs showing that under certain conditions (like the data being "sparse" enough), the method converges to the right answer. They proved it is "doubly robust," meaning it survives if one part of the model is wrong.
The Evidence: The confidence comes from the simulations and the two real-world case studies. In the simulations, the method consistently outperformed the competition. In the real-world tests, it showed clear improvements in accuracy and calibration.
The Caveat: The authors note that the method is computationally expensive (it takes a lot of computer power) and requires careful tuning of some settings. They also mention that while it works well for two groups (majority/minority), extending it to many different groups is a challenge for the future.

In short, AIMS is a smart, safety-conscious way to teach a robot to care about the small, rare groups in a crowd, using a big group as a guide but knowing exactly when to stop listening. It's a tool for fairness in data science, ensuring that the "minority" doesn't get lost in the noise of the "majority."

Technical Summary: Domain Adaptation Targeting Heterogeneous and Imbalanced Subgroups

Problem Formulation
The paper addresses a complex domain adaptation scenario where a target population contains heterogeneous subgroups with imbalanced sample sizes, and gold-standard labels are unavailable for the target domain during training. Specifically, the authors consider a setting with a source population ( $S$ ) and a target population ( $T$ ), where both populations are divided into a majority subgroup ( $R=1$ ) and a minority subgroup ( $R=0$ ). The goal is to estimate a high-dimensional, sparse coefficient vector $\bar{\beta}[0]$ for the minority subgroup in the target population, defined by the population parameter $E_T[Y | X, R=0] = g(X^T\bar{\beta}[0])$ .

The problem is characterized by three simultaneous challenges:

Covariate Shift: The distribution of covariates $Z = (X^T, W^T)^T$ differs between the source and target domains within each subgroup, while the conditional outcome mechanism $Y|Z, R$ remains invariant.
Sampling Disparity and Model Heterogeneity: The minority subgroup has significantly fewer samples than the majority ( $n_{T,0} \ll n_{T,1}$ ), and the outcome models between subgroups may differ ( $\bar{\beta}[0] \neq \bar{\beta}[1]$ ).
High-Dimensionality and Label Scarcity: The target model involves high-dimensional risk factors $X$ , and no labeled outcomes are available in the target domain for training or model selection.

A critical issue is negative transfer: borrowing information from the majority subgroup to aid the minority estimation may worsen performance if the subgroup-specific models differ significantly (i.e., the difference $\bar{\delta} = \bar{\beta}[1] - \bar{\beta}[0]$ is not sparse). Existing methods typically assume homogeneous targets or require labeled target data for selection, failing to address this specific combination of constraints.

Methodology: AIMS Framework
The authors propose AIMS (Adaptation to targets with heterogeneous and IMbalanced Subgroups), a unified framework comprising three algorithmic components:

Doubly Robust Covariate Shift Correction with Bifold Debiasing (Algorithm 1):
- To correct for covariate shift without target labels, the method constructs a doubly robust estimating equation combining importance weighting (using a density ratio model) and outcome imputation (using a conditional mean model).
- To handle high-dimensionality and regularization bias, a bifold bias correction is applied:
  - Fold 1: Nodewise Lasso debiasing removes regularization bias from the target coefficient estimator.
  - Fold 2: Moment calibration adjusts the nuisance estimators (density ratio and imputation models) to satisfy orthogonality conditions, removing first-order bias from nuisance estimation errors.
- A thresholding step is subsequently applied to the debiased dense vector to recover an $\ell_2$ -consistent sparse estimator.
Knowledge Transfer via Thresholded Contrast Estimation (Algorithm 2):
- This component leverages the majority subgroup's estimator ( $\hat{\beta}[1]_{Thr}$ ) as an "offset" to estimate the minority model.
- Instead of estimating $\bar{\beta}[0]$ directly, the method estimates the sparse contrast $\bar{\delta} = \bar{\beta}[0] - \bar{\beta}[1]$ .
- If the contrast is sparse, this approach borrows strength from the larger majority group, improving estimation efficiency for the minority subgroup without requiring target labels.
Label-Free Negative Transfer Protection via Model Ensemble (Algorithm 3):
- To guard against negative transfer (when the contrast is dense), the method employs a model selection strategy that does not rely on target labels.
- It uses cross-fitting to split the minority data into two folds. One fold generates candidate estimators (minority-only and knowledge-transfer), while the other generates a dense, debiased reference vector.
- A surrogate loss function, defined as the squared distance between a candidate estimator and the dense reference vector from the independent fold, is used to weight and aggregate the estimators via exponential weighting. This ensures the final estimator performs no worse than the minority-only baseline.

Key Contributions

Methodological: The paper introduces the first framework to simultaneously handle high-dimensional covariate shift, subgroup heterogeneity, and sampling disparity in a label-free target setting. It integrates doubly robust estimation with high-dimensional debiasing and adaptive knowledge transfer.
Theoretical:
- Model Doubly Robustness: The calibrated estimator is shown to be consistent if either the density ratio model or the conditional mean model is correctly specified, even in high dimensions.
- Convergence Rates: The method achieves convergence rates for the minority model that match supervised transfer learning methods (which use target labels) under mild nuisance error conditions.
- Adaptive Optimality: The final estimator adapts to the unknown sparsity of the subgroup difference, achieving the faster rate between the minority-only and knowledge-transfer estimators, up to a mild detection error cost.
Practical: The framework provides a solution for fair and equitable domain adaptation in scenarios where minority subgroups lack labeled outcomes, a common issue in biomedical and industrial applications.

Results

Simulations: Extensive simulations across three data-generating mechanisms (correctly specified, misspecified outcome model, and misspecified density ratio model) demonstrate that AIMS consistently outperforms or matches state-of-the-art baselines (including IW, IM, TransFusion, and TransGLM variants). AIMS shows particular robustness in high-dimensional settings and when the transferability between subgroups varies.
Real-World Applications:
1. Type II Diabetes (T2D) Risk Modeling: Using MGB biobank data, AIMS was applied to predict T2D risk for underrepresented (non-White) populations using labeled data from the majority (White) population. AIMS achieved the best calibration (Brier Skill Score: 0.36 vs. 0.23 for the next best) and AUC (0.89), significantly outperforming baselines that failed to correct for EHR coding shifts and subgroup imbalance.
2. Protein Stability Prediction ( $\Delta\Delta G$ ): In a study predicting mutation-induced stability changes, AIMS was used to transfer knowledge from a near-neutral pH assay (majority) to an acidic pH assay (minority). AIMS achieved the lowest RMSE (2.67) and highest Spearman correlation (0.27) on the target minority subgroup, whereas several baselines collapsed to constant predictions due to the small sample size and distributional shift.

Significance
The paper claims that AIMS fills a critical methodological gap by addressing covariate shift and sampling disparity simultaneously in high-dimensional, label-free settings. Unlike existing methods that treat the target as homogeneous or require target labels for selection, AIMS explicitly protects minority subgroups from bias and negative transfer. The authors position this work as a step toward responsible and generalizable statistical learning in biomedical and other fields where data scarcity and heterogeneity are prevalent. The theoretical justification of model double robustness and adaptive convergence rates in high dimensions is presented as a significant advancement over prior low-dimensional or single-challenge approaches.