💻 computer science

Test-Time Adaptation for EEG Foundation Models: A Systematic Study under Real-World Distribution Shifts

This paper introduces NeuroAdapt-Bench, a systematic benchmark revealing that while test-time adaptation offers a privacy-preserving solution for EEG foundation models facing real-world distribution shifts, standard gradient-based methods often degrade performance whereas optimization-free approaches provide more stable and reliable improvements.

Original authors: Gabriel Jason Lee, Jathurshan Pradeepkumar, Jimeng Sun

Published 2026-04-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Gabriel Jason Lee, Jathurshan Pradeepkumar, Jimeng Sun

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you've trained a brilliant, world-class chef (the EEG Foundation Model) in a specific kitchen using a specific set of ingredients, recipes, and a particular stove. This chef is amazing at making omelets when the eggs are fresh, the pan is non-stick, and the heat is exactly 350°F.

Now, imagine you send this chef to a new restaurant (a new hospital or device) to cook. But here's the catch:

The eggs are slightly different (different patients).
The stove is a different brand (different EEG machine).
The kitchen layout is weird (different recording setup).

In the real world, this chef often burns the omelet or makes a mess because the conditions have changed. This is what scientists call a "distribution shift."

The paper you shared is a systematic study asking a crucial question: Can we teach this chef to adapt instantly while they are cooking, without sending them back to the original kitchen for training?

This process is called Test-Time Adaptation (TTA). The researchers built a "gym" called NeuroAdapt-Bench to test different ways to help the chef adjust on the fly.

Here is the breakdown of their findings using simple analogies:

1. The Problem: The Chef is Confused

When the chef moves to the new kitchen, the brain signals (EEG) look different. They might be noisier, come from different sensors (like ear-EEG vs. scalp-EEG), or come from a different type of patient. The chef's original training doesn't fit perfectly anymore, and their performance drops.

2. The Solutions: Three Ways to Adapt

The researchers tested three different "coaching strategies" to help the chef adjust in real-time:

Strategy A: The "Entropy Minimizer" (Tent)
- The Analogy: This coach tells the chef, "Stop guessing! Just pick the option you feel most confident about, and tweak your grip on the spatula slightly to make that feeling stronger."
- The Result: This is like trying to force a square peg into a round hole by twisting it hard. In the study, this method often made the chef worse. It messed up the chef's original, well-trained instincts, causing them to burn the food even more.
Strategy B: The "Pseudo-Labeler" (SHOT)
- The Analogy: This coach says, "Look at all the eggs in the new kitchen first. Guess which ones are good, write down a new recipe based on those guesses, and then retrain your whole brain."
- The Result: This requires stopping to look at the whole batch of eggs before cooking. While it worked okay in some specific cases, it often caused the chef to overthink and lose their rhythm, leading to a drop in performance.
Strategy C: The "Prototype Refiner" (T3A)
- The Analogy: This coach says, "Don't change your cooking style or your grip. Just keep a mental list of what a 'perfect omelet' looks like. As you cook, if you see an egg that looks a bit like a perfect omelet, update your mental list slightly. Keep your hands steady, just update your memory."
- The Result: This was the winner. Because it didn't try to rewrite the chef's entire brain (no heavy math/gradient updates), it was stable. It helped the chef adjust to the new eggs without forgetting how to cook.

3. The Big Findings

Don't Over-Engineer It: The most complex, math-heavy methods (like Tent and SHOT) often broke the models. In healthcare, where mistakes can be dangerous, stability is more important than trying to squeeze out a tiny bit more accuracy.
The "Ear" Problem: They tested the chef on a completely weird setup: cooking with sensors in the ear (Ear-EEG) instead of the scalp. This is like asking the chef to cook with a toaster instead of a stove. The complex methods failed miserably. The simple "Prototype Refiner" (T3A) was the only one that didn't completely crash and actually improved the results slightly.
One Size Does Not Fit All: What works for natural images (like recognizing cats in photos) doesn't work for brain waves. Brain waves are messy, change from person to person, and are very sensitive.

The Takeaway for the Real World

If you are building AI for hospitals, don't just copy-paste methods from image recognition.

The paper suggests that when deploying AI for brain data:

Keep it simple: Don't try to retrain the whole model on the fly.
Prioritize stability: It's better to have a model that performs consistently well than one that occasionally gets perfect but usually crashes.
Use the "T3A" approach: Adjusting the "mental list" of what the model expects (prototypes) is safer and more effective than trying to rewrite the model's internal math.

In short: When sending a brain-AI chef to a new kitchen, give them a simple notepad to update their expectations (T3A), rather than forcing them to rewrite their entire cookbook while they are cooking (Tent/SHOT).

1. Problem Statement

Electroencephalography (EEG) foundation models have demonstrated strong potential in learning generalizable representations from large-scale neural data. However, their clinical deployment is severely hindered by distribution shifts. Unlike natural images where domain gaps are often stylistic, EEG signals exhibit complex, patient-specific dynamics and vary significantly across acquisition devices, clinical sites, and recording protocols (e.g., scalp vs. ear-EEG).

Traditional domain adaptation often requires access to source data or labeled target data, which is frequently unavailable in healthcare due to privacy regulations (HIPAA/GDPR) and the scarcity of labeled clinical data. Test-Time Adaptation (TTA) offers a solution by allowing models to adapt to unlabeled target data during inference without source data access. Despite its success in computer vision and speech, the effectiveness of existing TTA methods for EEG foundation models remains largely unexplored and unverified under realistic, heterogeneous clinical conditions.

2. Methodology: NeuroAdapt-Bench

The authors introduce NeuroAdapt-Bench, a systematic benchmark designed to evaluate TTA methods on EEG foundation models. The benchmark follows a three-stage pipeline:

Classifier Fine-Tuning: Pretrained EEG foundation models are paired with a unified, lightweight classification head. The encoder backbone is frozen, and only the head is trained on labeled source data to ensure fair cross-model comparison.
Test-Time Adaptation: The model is deployed on unlabeled target data. The benchmark evaluates three representative TTA methods against a "No-TTA" baseline:
- Tent (Online, Gradient-based): Minimizes prediction entropy by updating the affine parameters of normalization layers.
- SHOT (Offline, Gradient-based): Uses mutual information maximization and pseudo-labeling to adapt the feature extractor via gradient descent, requiring the full target batch upfront.
- T3A (Online, Optimization-free): Adapts the classifier by refining class prototypes (support sets) based on low-entropy target features without updating network weights.
Evaluation: Performance is measured using standard metrics (Balanced Accuracy, ROC-AUC, PR-AUC, Cohen's $\kappa$ , Weighted F1) across diverse distribution shifts.

Experimental Scope:

Foundation Models: CBraMod, TFM-Tokenizer, and REVE (Base and Large variants).
Datasets & Shifts:
- In-Distribution: TUEV, TUAB (subject-level variability).
- Out-of-Distribution (Cross-Dataset): CHB-MIT (seizure detection), SleepEDF-78 (sleep staging).
- Extreme Modality Shift: Ear-EEG (wearable, different channel configuration).
Variables: Adaptation regimes (online vs. offline), batch sizes, and random seeds.

3. Key Contributions

NeuroAdapt-Bench: The first unified benchmark for evaluating TTA on EEG foundation models across realistic distribution shifts, including extreme modality changes (Ear-EEG).
Comprehensive Evaluation: A systematic study covering multiple foundation models, diverse downstream tasks (seizure detection, sleep staging, abnormality screening), and heterogeneous deployment settings.
Critical Insights: The paper challenges the assumption that TTA methods successful in vision/speech will transfer to EEG, providing empirical evidence on when adaptation fails and succeeds.
Open-Source Framework: Release of code and evaluation pipelines to facilitate reproducibility and future research in healthcare AI.

4. Key Results

The study yields several counter-intuitive and significant findings:

Inconsistency of Standard TTA: Standard TTA methods (Tent, SHOT) often yield inconsistent gains and frequently degrade performance compared to the No-TTA baseline, particularly under distribution shifts.
Gradient-Based Instability: Gradient-based approaches (Tent and SHOT) are particularly prone to heavy degradation. They often disrupt well-calibrated pretrained representations, leading to "negative transfer."
- Example: On the TUAB dataset, all TTA methods degraded performance, with Tent showing the largest drop.
- Example: On SleepEDF-78, gradient-based methods caused substantial performance drops across nearly all models.
Superiority of Optimization-Free Methods: T3A demonstrated the greatest stability and reliability.
- It is the only method with a positive mean balanced-accuracy improvement across in-distribution, out-of-distribution, and extreme-shift (Ear-EEG) settings.
- It achieved a massive +18.9 percentage-point improvement in balanced accuracy for REVE-Base on the CHB-MIT dataset.
Robustness to Modality Shift: In the extreme Ear-EEG setting, gradient-based methods consistently failed, while T3A provided stable improvements (e.g., +4.8% balanced accuracy for CBraMod).
Batch Size Sensitivity: Increasing batch size did not consistently improve performance for gradient-based methods and was irrelevant for T3A, suggesting that simply scaling batch size is insufficient to stabilize TTA in EEG.
Representation Impact: Models using discrete tokenization (TFM-Tokenizer) showed different resilience patterns compared to continuous embedding models, indicating that TTA effectiveness is tied to the underlying representation paradigm.

5. Significance and Implications

Clinical Deployment Reality: The findings highlight that "plug-and-play" TTA is not currently viable for EEG. Deploying models without careful selection of adaptation strategies can actively harm clinical performance.
Stability over Accuracy: For clinical applications, stability and robustness are as critical as peak accuracy. Optimization-free methods (like T3A) that preserve pretrained representations are shown to be safer and more reliable than gradient-based fine-tuning.
Need for Domain-Specific Solutions: Existing TTA techniques from computer vision do not transfer well to the non-stationary, artifact-prone nature of EEG signals. This underscores the urgent need for EEG-specific adaptation strategies that account for signal dynamics and acquisition variability.
Future Directions: The paper advocates for research into adaptation methods that minimize disruptive updates and are tailored to the specific representation types (continuous vs. discrete) of EEG foundation models.

In conclusion, this study establishes that while TTA is a promising concept for healthcare AI, current off-the-shelf methods are often detrimental for EEG. The authors provide a rigorous framework (NeuroAdapt-Bench) and empirical evidence guiding the community toward more stable, optimization-free adaptation strategies for real-world clinical deployment.