🤖 machine learning

ADAPTOOD: Uncertainty-Aware Fine-Tuning for Out-of-Distribution ECG Time Series Models

The paper proposes ADAPTOOD, an uncertainty-aware fine-tuning framework that quantifies out-of-distribution shift severity in ECG time series data to guide adaptive hyperparameter optimization and low-rank model updates, thereby significantly improving robustness and performance compared to existing methods.

Original authors: Sotirios Vavaroutas, Yu Yvonne Wu, Ali Etemad, Cecilia Mascolo

Published 2026-06-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sotirios Vavaroutas, Yu Yvonne Wu, Ali Etemad, Cecilia Mascolo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "One-Size-Fits-All" Trap

Imagine you hire a master chef who spent 10 years perfecting a recipe for Italian pasta using high-end ingredients in a professional kitchen (this is the pre-trained model).

Now, you want to use this chef to cook for a new group of people in a different setting. Maybe they are in a rustic cabin with a campfire, or maybe they only have canned beans and a tiny stove.

The Issue: If you just tell the chef, "Cook pasta," they might try to use their expensive kitchen tools on a campfire (it won't work) or try to cook a fancy dish with canned beans (it will taste bad).
The Current Solution: Usually, we try to "fine-tune" the chef by letting them practice on the new ingredients. But existing methods treat every new situation the same way. Whether the new kitchen is slightly different or completely alien, they apply the same amount of training. This is inefficient and often leads to bad results.

The Solution: ADAPTOOD (The "Smart Sensing" Chef)

The authors created a new system called ADAPTOOD. Instead of blindly retraining the chef, ADAPTOOD acts like a smart sensor that first checks how different the new situation is from the old one.

Here is how it works, step-by-step:

1. The "Uncertainty Meter" (Checking the Gap)

Before the chef starts cooking, ADAPTOOD measures the "distance" between the old kitchen and the new one.

The Analogy: Imagine the chef has a "familiarity meter."
- If the new kitchen is just a different color but has the same stove, the meter says, "Low uncertainty. This is easy; we only need to tweak a few things."
- If the new kitchen is a campfire with no stove, the meter says, "High uncertainty! This is totally different. We need to change our whole approach."
How it works: The system uses math (called Mahalanobis and Hellinger distances) to calculate exactly how "out of place" the new data feels compared to what the model learned originally.

2. Selective Unfreezing (Deciding What to Change)

Once the system knows how different the new situation is, it decides how much of the chef's brain to "retrain."

The Analogy:
- Low Uncertainty (Mild Shift): The system says, "The stove is just a bit smaller." It only unlocks the chef's hands to adjust the heat. It keeps the rest of the chef's knowledge frozen so they don't forget how to make pasta.
- High Uncertainty (Severe Shift): The system says, "This is a campfire!" It unlocks the chef's entire brain, allowing them to learn entirely new techniques, but it does so carefully to avoid forgetting the basics.
The Paper's Claim: This prevents the model from overthinking simple changes or under-thinking complex ones.

3. The "Low-Rank" Shortcut (Efficiency)

The system uses a technique called LoRA (Low-Rank Adaptation).

The Analogy: Instead of rewriting the chef's entire 500-page cookbook, the system writes a small, sticky-note addendum (a few pages) that tells the chef how to adapt to the new ingredients.
The Benefit: This makes the process much faster and requires less computer memory, which is great for running on small devices like smartwatches.

4. The "Auto-Tuner" (Finding the Perfect Settings)

Finally, the system automatically adjusts the "knobs" (hyperparameters) for the training.

The Analogy: It's like a car with an automatic transmission that shifts gears perfectly for the terrain, rather than a driver manually guessing which gear to use. It tries different settings to find the one that works best for this specific new dataset.

What Did They Find? (The Results)

The researchers tested this on ECG data (heart rate monitors). They simulated many different "new kitchens":

Different People: Young adults vs. elderly patients.
Different Devices: High-quality hospital machines vs. cheap wearable sensors.
Different Signals: Heartbeats (ECG) vs. blood flow (PPG).
Different Data Sizes: Huge datasets vs. tiny datasets.

The Outcome:
ADAPTOOD consistently beat the other methods.

It was up to 7% more accurate and 12.9% more precise than the best existing methods.
Crucially, as the "new kitchen" got more different (more severe shifts), ADAPTOOD got better at handling it, while other methods struggled.
It also used significantly less computer memory (up to 3x less) because of the "sticky-note" (LoRA) approach.

Summary

ADAPTOOD is a smart way to teach an AI model new tasks without starting from scratch. Instead of forcing the model to learn everything the same way every time, it first measures how different the new data is, then only changes the parts of the model that need changing, and automatically finds the best settings to do it. This makes AI models for heart monitoring (and other time-series data) much more robust and efficient when they encounter real-world surprises.

Technical Summary: ADAPTOOD – Uncertainty-Aware Fine-Tuning for Out-of-Distribution ECG Time Series Models

Problem Statement

Deploying machine learning models trained on large-scale sensor data into real-world clinical settings faces significant hurdles due to distribution shifts. While pre-training offers a foundation, models often struggle when fine-tuned on small, annotated datasets that differ from the pre-training distribution. These discrepancies arise from diverse sources, including population heterogeneity, varying sensor types, differing collection protocols, and temporal drifts.

Existing adaptation methods, such as transfer learning and domain adaptation, typically assume fixed distribution shifts or treat Out-of-Distribution (OOD) cases as a binary state (in-domain vs. out-of-domain). They fail to account for the severity or granularity of these shifts. Consequently, they often apply uniform adaptation strategies regardless of whether the shift is mild (e.g., a slightly different population) or severe (e.g., a new sensor modality), leading to suboptimal generalization, overfitting on small datasets, or catastrophic forgetting.

Methodology: ADAPTOOD Framework

The authors propose ADAPTOOD, a framework designed to quantify distribution shift severity and guide fine-tuning dynamically. The system operates through four interconnected modules:

1. Uncertainty-Guided Severity Quantification

The core innovation is using data uncertainty to measure how much a target dataset ( $D_t$ ) deviates from the pre-training source distribution ( $D_s$ ).

Dimensionality Reduction: High-dimensional ECG signals are reduced to a dominant component using Principal Component Analysis (PCA) to improve density estimation.
Distance Metrics: The framework calculates distribution-level divergence using two complementary metrics:
- Mahalanobis Distance ( $d_m$ ): Measures the distance of new batches from the center of the known distribution, accounting for feature correlations and scale.
- Hellinger Distance ( $d_h$ ): Quantifies the similarity between the overall probability distributions of the source and target.
Combined Metric: These distances are normalized and combined into a weighted uncertainty score, $\sigma(D_t | P(D_s))$ , where a higher score indicates a more severe OOD shift.

2. Selective Layer Unfreezing

Instead of fine-tuning the entire model or a fixed subset, ADAPTOOD dynamically determines how many layers to unfreeze based on the calculated uncertainty score.

Strategy: It employs a combination of linear and exponential decay functions.
- Linear: The number of unfrozen layers is proportional to the inverse of the uncertainty.
- Exponential: Uses a decay factor to increase sensitivity to uncertainty levels.
Execution: The final number of trainable layers is the mean of these two calculations. Layers are unfrozen starting from the top (closest to the output) down to the bottom. This ensures that mild shifts require minimal architectural changes, while severe shifts trigger more extensive adaptation, balancing the risk of overfitting and underfitting.

3. Low-Rank Adaptation (LoRA)

To maintain computational efficiency and prevent overfitting on limited target data, the framework integrates LoRA into all Conv1D layers.

Mechanism: Pre-trained weights are frozen, and lightweight, trainable low-rank matrices are introduced.
Benefit: This significantly reduces the parameter count and memory footprint while focusing the model's capacity on learning local temporal patterns where it is most needed.

4. Adaptive Hyperparameter Optimisation

Recognizing that hyperparameters optimal for pre-training may not suit OOD tasks, the framework utilizes Bayesian Optimization.

Process: It iteratively searches for optimal configurations for the optimizer (e.g., Adam, RMSprop) and learning rate over a defined search space.
Goal: To identify configurations that best accommodate the specific distribution shift without manual tuning.

Key Contributions

Novel Severity Quantification: Introduction of a mechanism that leverages data uncertainty (via Mahalanobis and Hellinger distances) to assess the severity of OOD shifts relative to pre-training data, moving beyond binary in/out-of-domain assumptions.
Severity-Flexible Adaptation: Development of an approach that uses uncertainty scores to guide selective layer unfreezing and hyperparameter tuning, ensuring the adaptation strategy matches the complexity of the shift.
Efficiency and Robustness: Integration of LoRA and Bayesian optimization to achieve parameter-efficient fine-tuning that maintains strong performance across diverse metrics even as distribution shift severity increases.

Experimental Results

The authors evaluated ADAPTOOD on ECG time series datasets involving various shifts (population, sensor, temporal, label, modality, and dimensionality) compared against baselines including standard Transfer Learning, Supervised Learning, Feature-based Domain Adaptation, and Instance-based Domain Adaptation.

Performance Gains: ADAPTOOD consistently outperformed all baselines.
- In the MIT-BIH dataset (sensor/population shift), it achieved 0.953 accuracy and 0.930 precision, outperforming the best baseline (Feature-based DA) by 1.3% in accuracy and 0.9% in precision.
- In the PTB-DB dataset, it showed a 11.9% improvement in accuracy over Transfer Learning (0.922 vs. 0.803) and a 12.9% improvement in precision.
- In the MIMICPERform PPG dataset (a severe modality shift from ECG to PPG), ADAPTOOD reached 0.975 accuracy, surpassing the best baseline by approximately 3% (0.945).
- In the CODEtest dataset (dimensionality shift with 12-lead ECG), it achieved 0.922 accuracy, outperforming the next best method (Feature-based DA at 0.894).
Ablation Studies: Removing components (uncertainty metrics, hyperparameter tuning, or LoRA) consistently degraded performance, confirming the necessity of the full framework. Specifically, the combination of both distance metrics provided more consistent results than using either alone.
Efficiency: By utilizing LoRA and selective unfreezing, ADAPTOOD reduced the total parameter count and memory footprint significantly. For example, on the MIT-BIH dataset, the model size was reduced from ~29.17 MB (without LoRA) to 8.61 MB.

Significance and Claims

The paper claims that ADAPTOOD addresses a critical gap in biosignal interpretation: the lack of methods that can adapt to varying severities of distribution shifts. By treating OOD adaptation as a severity-dependent process rather than a fixed protocol, the framework achieves:

Robust Generalization: Maintaining high accuracy and precision even when test data diverges significantly from training data due to multiple overlapping shift types.
Resource Efficiency: Achieving these gains with fewer parameters and lower computational costs, making it suitable for deployment in resource-constrained environments.
Calibration: Providing a more balanced performance across metrics (accuracy, precision, recall, F1) compared to baselines that often suffer from class imbalance issues (e.g., low sensitivity).

The authors position ADAPTOOD as a reliable, data-driven approach for fine-tuning pre-trained models in real-world, out-of-distribution scenarios, particularly where labeled data is scarce and shift characteristics are unknown or variable.