Technical Summary: CaMBRAIN - Real-time, Continuous EEG Inference with Causal State Space Models

Problem Statement

Electroencephalography (EEG) signals are critical for monitoring brain activity in clinical and biomechanical settings, yet they present unique challenges for deep learning due to their limitless length, often spanning minutes to hours. Existing deep learning approaches, particularly those based on Transformers, face two primary hurdles when applied to long-duration EEG:

1. Computational Inefficiency: Attention mechanisms incur quadratic scaling ($O(T^2)$) with sequence length, making them inefficient for long signals.
2. Context Limitations: To manage computational costs, current models typically process EEG via sliding windows with fixed input lengths. This approach forces the model to recompute features over overlapping regions and prevents the maintenance of a persistent representation of the entire signal history. Consequently, these models lack global temporal understanding and cannot perform true real-time, continuous inference where each new observation builds upon prior context without revisiting past inputs.

Furthermore, existing self-supervised pretraining objectives for EEG often rely on signal reconstruction (e.g., masked autoencoders). While effective for learning local signal fidelity, these objectives do not explicitly train the hidden state to retain salient long-range context, which is essential for streaming inference where the model must compress history into a compact memory.

Methodology

The authors propose CaMBRAIN, the first causal, Mamba-based EEG model designed for real-time, continuous streaming inference. The methodology is built on two core pillars: a causal architecture and a specialized multi-stage self-supervised training pipeline.

1. Architecture: Causal Mamba-3

CaMBRAIN utilizes a unidirectional, causal State Space Model (SSM), specifically the Mamba-3 architecture, to process EEG signals.

• Input Processing: Raw EEG signals are partitioned into non-overlapping patches of 62.5 ms (16 samples at 256 Hz) across all channels. These patches are embedded into tokens using a transformer-based channel embedder.
• Persistent Hidden State: Unlike sliding-window approaches, CaMBRAIN maintains a persistent hidden state that incrementally compresses all past context. As new patches arrive, the hidden state is updated recurrently ($h_t = \bar{A}h_{t-1} + \bar{B}x_t$) without revisiting previous inputs.
• Causal Constraint: The model operates strictly in a forward-only manner, ensuring that predictions at time $t$ depend only on past and current inputs ($x_{\le t}$), aligning with the physical reality of EEG monitoring.
• Complex-Valued Transitions: Leveraging Mamba-3, the model employs complex-valued state transitions to better capture the oscillatory dynamics and phase relationships inherent in multi-band EEG signals.

2. Training Pipeline

To address the limitations of reconstruction-based objectives for streaming, the authors introduce a three-stage training pipeline:

• Stage 1: Causal Predictive Pretraining:
  The encoder is pretrained using a hybrid causal loss to instill general temporal structure. This combines:

  • Autoregressive Modeling (ARM): Predicting the next patch embedding from the current hidden state.
  • Masked Reconstruction: Reconstructing masked raw EEG tokens from the causal context provided by the encoder.
    This stage ensures the model learns local temporal dynamics and signal fidelity.

• Stage 2: Reconstruction-Free Latent Representation Refinement (JEPA-style):
  To encourage the retention of long-range, predictive information, the model undergoes a decoder-free training phase inspired by Joint-Embedding Predictive Architectures (JEPA).

  • Student-Teacher Framework: A student network processes a masked view of the input, while a teacher network (an Exponential Moving Average of the student) processes the unmasked input.
  • Objectives: The student is trained to predict:
    1. Masked Latents: The teacher's representations for masked positions.
    2. Future Latents: Multi-step future representations ($K$ steps ahead) from the current hidden state.
       This stage explicitly forces the hidden state to encode informative, invariant, and temporally relevant features necessary for continuous inference, rather than just reconstructing local signal details.

• Stage 3: Downstream Fine-Tuning:
  The pretrained model is fine-tuned on specific tasks (e.g., seizure detection, artifact classification) using standard supervised learning. Notably, CaMBRAIN utilizes only the final hidden state for classification, avoiding the need for pooling or aggregating tokens across fixed segments.

Key Contributions

1. Causal Inference Formulation: The paper proposes a unidirectional EEG inference formulation that departs from sliding-window and bidirectional transformer approaches, treating signals as continuous streams rather than independent segments.
2. CaMBRAIN Architecture: A causal Mamba-3 encoder that replaces redundant sliding-window recomputation with a persistent hidden state, enabling efficient, scalable inference over arbitrarily long signals with linear-time complexity.
3. Representation-Level Training: A novel multi-stage self-supervised framework that moves beyond signal reconstruction. By combining autoregressive prediction with masked and future latent prediction, it explicitly trains the hidden state to retain salient long-range context.

Results

CaMBRAIN was evaluated on four diverse EEG datasets: TUAR (artifact detection), TUAB (abnormal EEG detection), MAT (mental stress detection), and CHB-MIT (seizure detection).

• Performance: CaMBRAIN achieved state-of-the-art (SOTA) or competitive results across all benchmarks.
  • TUAR: Achieved an AUROC of 0.936, outperforming larger foundation models like LUNA-H (0.921).
  • CHB-MIT: Achieved an AUROC of 0.921, demonstrating strong performance on highly imbalanced seizure data.
  • MAT: Achieved the best balanced accuracy (0.778) and AUROC (0.876) among compared methods.
• Computational Efficiency: The model demonstrated significant efficiency gains in streaming scenarios.
  • At a 16 Hz streaming update rate, CaMBRAIN required 1.23 GFLOPs/s.
  • This represents a 6.1× to 158× reduction in sustained compute compared to other foundation model baselines (e.g., REVE required 194.79 GFLOPs/s), while matching or exceeding their accuracy.
• Ablation Studies:
  • Persistent State: Using a persistent hidden state (vs. resetting every 5 seconds) improved AUROC by +0.015 on CHB-MIT and significantly improved seizure onset localization.
  • Training Stages: The full two-stage pretraining (Stage 1 + Stage 2) yielded substantially larger gains (+2.30 AUROC) over standard fine-tuning or Stage 1 alone, confirming the necessity of representation-level refinement for streaming tasks.

Significance and Claims

The paper claims that CaMBRAIN represents a paradigm shift in EEG modeling by aligning both architecture and training objectives with the inherent causal and continuous nature of EEG signals.

• Real-Time Capability: It enables the first model capable of performing long-range, continuous inference on variable-length EEG signals without the computational penalty of quadratic scaling or the redundancy of sliding windows.
• Clinical Relevance: The approach mirrors how clinicians monitor EEG at the bedside, maintaining a compact memory of the entire signal history to detect events that may be separated by long intervals.
• Efficiency: By leveraging the linear-time scaling of State Space Models and a representation-level training objective, CaMBRAIN achieves high throughput and low latency, making it suitable for real-time clinical and wearable applications where computational resources are constrained.

The authors acknowledge that while CaMBRAIN sets new benchmarks on several datasets, it remains competitive rather than superior on TUAB abnormal EEG detection compared to the strongest foundation models, suggesting that scaling the pretraining corpus to match larger, more heterogeneous datasets (like REVE) is a necessary future direction.