Parameter-Efficient Self-Supervised Adaptation for EEG-FM under Fixed Computational Budgets

Technical Summary: Parameter-Efficient Self-Supervised Adaptation for EEG-FM under Fixed Computational Budgets

Problem Statement
Electroencephalography (EEG) foundation models (EEG-FMs), pretrained via self-supervised learning (SSL) on massive unlabeled corpora, offer transferable representations for neurological monitoring. However, their direct deployment in clinical settings is hindered by distribution shifts arising from differences in recording hardware, channel montages, and sampling rates. While full fine-tuning is an option, it is often impractical for resource-constrained clinical environments due to high computational costs and the scarcity of annotated data. Furthermore, it remains unclear how much unlabeled target-domain data is necessary for reliable adaptation and whether adaptation benefits more from patient diversity or temporal data volume.

Methodology
The authors propose a Parameter-Efficient Self-Supervised Adaptation (SA) strategy designed to align pretrained EEG-FMs to target tasks using only unlabeled target-domain data. The methodology operates under a normalized fixed-compute protocol to isolate the effects of data diversity from computational cost.

1. Adaptation Strategy: The approach partitions the model parameters into frozen lower layers ($\theta_{frozen}$) and a trainable subset ($\theta_{adapt}$) corresponding to the final encoder layer. The model preserves its original SSL objective (either contrastive learning for BIOT or masked reconstruction for CBraMod) and optimizes the loss with respect to $\theta_{adapt}$ on the unlabeled target dataset. This updates only approximately 9% of the total parameters.
2. Evaluation: Downstream performance is assessed via linear probing on labeled datasets. A linear classifier is trained on the representations extracted from the adapted encoder, while the encoder weights remain frozen.
3. Experimental Design:
   • Models: Two state-of-the-art EEG-FMs with distinct pretraining objectives: BIOT (contrastive) and CBraMod (masked reconstruction).
   • Datasets: Three clinical datasets covering abnormality detection (TUAB), event classification (TUEV), and seizure detection (CHB-MIT), evaluated under both in-distribution (ID) and out-of-distribution (OOD) conditions.
   • Fixed Compute Protocol: The total computational budget ($C$) is defined as the product of the number of unique windows ($N$) and the number of epochs/repetition factor ($E$), i.e., $C = N \times E$. Experiments vary $N$ (data diversity) while adjusting $E$ to keep $C$ constant.
   • Patient vs. Window Diversity: To determine the impact of patient count ($P$) versus temporal coverage, the authors fix the total number of windows ($N$) and compute budget ($C$) while varying $P$.

Key Contributions and Results

• Superiority over Linear Probing: Parameter-efficient SSL adaptation yields consistent and substantial performance gains over standard linear probing (using frozen pretrained features). Improvements range from +1.79 Weighted F1 to +36.2 AUCROC. Notably, on the rare-event seizure detection task (CHB-MIT), where frozen features performed near-randomly (AUCPR $\approx$ 1.5%), adaptation recovered clinically meaningful performance (AUCPR $\approx$ 31.5%), representing a gain of up to 20$\times$.
• Data Efficiency under Fixed Compute: The study demonstrates that peak performance is achieved using only 20–50% of the available unlabeled target data. Beyond this threshold, increasing unique data samples under a fixed compute budget yields diminishing returns, as the model receives insufficient repetition to consolidate learned features. The optimal repetition factor ($r$) was found to be approximately 12.
• Temporal Diversity over Patient Count: Under a fixed computational budget and total window count, the number of unique patients has no significant impact on adaptation performance. The results suggest that for FM-initialized SSL adaptation, the total diversity of temporal windows is the critical factor, rather than the diversity of patient identities.
• Parameter Efficiency: Updating only the final encoder layer (approx. 9% of parameters) provides an optimal cost-performance balance. Ablation studies showed that unfreezing additional layers offered marginal or negative gains relative to the increased parameter count.

Significance and Claims
The paper claims that parameter-efficient adaptation enables the effective deployment of EEG Foundation Models with minimal computational overhead and data collection burdens. By demonstrating that:

1. Only a small fraction of parameters (final layer) needs updating to align representations.
2. Peak performance requires only a fraction (20–50%) of available unlabeled data when repetition is optimized.
3. Clinical sites can adapt models effectively using longer recordings from smaller patient cohorts rather than requiring large numbers of unique patients.

The authors conclude that this approach bridges the gap between large-scale pretrained models and resource-constrained clinical deployment, particularly for tasks involving rare events or significant distribution shifts. The work suggests that future clinical adaptation efforts should prioritize temporal window coverage and repetition over maximizing patient diversity, provided the total compute budget is constrained.