Technical Summary: MtReNIRS

Problem Statement

Functional near-infrared spectroscopy (fNIRS) offers a portable, non-invasive method for monitoring hemodynamic correlates of neural activity, yet its adoption in scalable brain-computer interfaces (BCIs) is hindered by significant heterogeneity. Existing modeling approaches are largely confined to small-scale, single-session datasets with low sampling rates. The field faces three primary challenges:

1. Data Heterogeneity: Acquisition devices, sampling rates, signal representations, and source–detector (S–D) geometries vary substantially across studies, preventing models trained on one setup from generalizing to others.
2. Scalability and Calibration: Computational modeling often requires subject-specific calibration, which is costly and time-consuming, limiting the deployment of fNIRS in everyday scenarios.
3. Physiological Mismatch: fNIRS signals exhibit slow, delayed, and highly autocorrelated hemodynamic responses (HbO/HbR), which differ fundamentally from the fast electrophysiology targeted by many existing brain-signal encoders (e.g., those designed for EEG).

Consequently, it remains unclear whether a single pretrained fNIRS encoder can learn representations that are transferable across subjects, tasks, devices, and sensor montages.

Methodology

The authors introduce MtReNIRS (Montage-aligned transferable Representations for fNIRS), a self-supervised foundation model designed to learn generalizable neurovascular representations.

1. Dataset Curation and Preprocessing

The model was pretrained on 12 public fNIRS datasets spanning 268 subjects and 567 recording hours. These datasets cover diverse tasks (cognitive, motor, auditory, affective, clinical) and cortical coverage.

• Signal Processing: A dataset-adapted pipeline converts raw intensity or optical density to HbO/HbR concentrations using the modified Beer–Lambert law. Signals are band-pass filtered (0.01–0.5 Hz), resampled to 10 Hz, and z-scored.
• Montage Harmonization: A core component of the framework maps heterogeneous S–D pair geometries to a shared canonical 10–05 reference system (chosen over 10–20 for denser coverage).
  • For 3-D geometries, raw S–D midpoints are aligned via Root Mean Square (RMS) scaling.
  • For 2-D schematics, probes are placed on anatomically supported local tangent planes.
  • For known placements without geometry, direct label assignment is used.
  • This process ensures that signals from different devices are consistently related to underlying cortical locations.

2. Model Architecture

MtReNIRS employs a montage-aware masked autoencoder with a compact encoder of 1.23 million parameters.

• Tokenization: Data is segmented into non-overlapping 10-second windows. Each token represents a chromophore channel (HbO or HbR) from a specific S–D pair.
• Embeddings:
  • Signal Embedding: 10-second signal patches are projected to 128 dimensions.
  • Spatio-Chromophore Embedding: Canonical 3-D coordinates and binary chromophore indicators are encoded using Fourier features and projected via an MLP.
  • Positional Encoding: Rotary Positional Encoding (RoPE) preserves temporal order across 60-second crops (six 10-second windows) without conflating time with spatial identity.
• Training Objective: The model acts as a masked autoencoder. For each 60-second crop, 25% of tokens are masked. A lightweight Transformer decoder reconstructs the masked 10-second waveforms from the visible tokens, minimizing Mean Squared Error (MSE) only over the masked real tokens.

3. Evaluation Protocols

The model was evaluated on three unseen downstream datasets (Gunther2014, Shader2021, SujinBak2019) using two benchmarks:

• Cross-Subject Transfer (LOSO): Leave-one-subject-out classification to test generalization to new users.
• Within-Subject Few-Shot Calibration: Testing performance with only 2–8 labeled examples per class to assess label efficiency.
• Baselines: Comparisons included fully supervised Transformers, CNNs, LSTMs, and statistical SVMs, as well as ablation studies on pretraining duration, token duration, and anatomical alignment.

Key Contributions

1. Montage Harmonization Framework: A novel method to map heterogeneous fNIRS recordings to a shared anatomical representation, enabling joint pretraining across datasets with different sensor layouts.
2. Physiologically Grounded Foundation Model: A compact encoder that jointly represents signal dynamics, anatomical location, and chromophore identity, specifically designed for the slow, delayed nature of hemodynamic responses.
3. Systematic Transfer Evaluation: Comprehensive evaluation demonstrating that the model achieves cross-subject performance comparable to fully supervised baselines and excels in few-shot within-subject calibration, outperforming prior state-of-the-art fNIRS models.

Results

• Cross-Subject Transfer: MtReNIRS adaptation strategies (specifically Linear Probe and Full Fine-Tuning) matched or exceeded fully supervised baselines across all three downstream datasets. While no single adaptation strategy was uniformly superior, pretraining provided competitive transfer performance.
• Few-Shot Calibration: MtReNIRS demonstrated significant gains in few-shot settings. On the Gunther2014 dataset, it outperformed existing fNIRS-only pretrained models and fully supervised baselines across 2–8 shot conditions. On the SujinBak2019 dataset, gains over random initialization were substantial (up to 11 percentage points) and statistically significant.
• Ablation Studies:
  • Pretraining Duration: Significant performance gains were observed after pretraining, particularly on the Shader and SujinBak datasets.
  • Temporal Context: A 10-second token duration yielded the best results, consistent with the slow nature of hemodynamic responses.
  • Anatomical Alignment: Shuffling spatial coordinates during pretraining significantly degraded performance, confirming that learning anatomical correspondence is critical for transferability.
  • Robustness: The model remained robust to reduced sensor coverage, maintaining high performance even with as few as 16 S–D pairs, approaching full-montage performance.

Significance and Claims

The paper claims that montage-aware self-supervision yields physiologically grounded, transferable fNIRS representations. The primary significance lies in demonstrating that a single foundation model can learn reusable neurovascular features that generalize across subjects, tasks, and devices.

The authors emphasize that the largest and most consistent benefits of pretraining emerge in personalized adaptation (few-shot within-subject calibration) rather than pure zero-calibration cross-subject transfer. This suggests that while subject-specific physiological variability remains a dominant source of error, a small number of subject-specific labels are sufficient to adapt a generic, pretrained representation to a new user.

The work provides a scalable foundation for label-efficient BCIs, addressing the bottleneck of data annotation and device heterogeneity. The authors note limitations, including the need for larger and more diverse downstream benchmarks, the challenge of isolating device-specific effects, and the inherent difficulty of cross-subject transfer due to anatomical and vascular variability. Future work is suggested to explore multimodal pretraining (e.g., EEG-fNIRS fusion) and controlled cross-device studies.