🤖 machine learning

DIVER-1: Scaling Intracranial EEG Foundation Models for Transferable Representations

The paper introduces DIVER-1, a self-supervised intracranial EEG foundation model trained on a massive, diverse dataset that achieves state-of-the-art transferable representations for cognitive decoding and seizure detection while demonstrating that data scale and training duration are more critical than parameter count for effective scaling.

Original authors: Danny Dongyeop Han, Yonghyeon Gwon, Ahhyun Lucy Lee, Taeyang Lee, Seong Jin Lee, Jubin Choi, Sebin Lee, Jihyun Bang, Seungju Lee, David Keetae Park, Shinjae Yoo, Chun Kee Chung, Jiook Cha

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Danny Dongyeop Han, Yonghyeon Gwon, Ahhyun Lucy Lee, Taeyang Lee, Seong Jin Lee, Jubin Choi, Sebin Lee, Jihyun Bang, Seungju Lee, David Keetae Park, Shinjae Yoo, Chun Kee Chung, Jiook Cha

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine your brain is a massive, bustling city. When doctors want to understand what's happening in that city, they sometimes place tiny microphones (electrodes) directly on the streets or inside the buildings. This is called intracranial EEG (iEEG). It gives a crystal-clear, millisecond-by-millisecond recording of the city's activity.

However, there's a huge problem: every city looks different. One patient might have microphones on the north side, another on the south; one might have 10 microphones, another 100. Because of this, it's been incredibly hard to build a "universal translator" (an AI) that can listen to any patient's brain and understand it. Previous attempts at these translators were like trying to teach a student using only one specific textbook, only to find they can't read a different one.

Enter DIVER-1. Think of DIVER-1 as a super-smart, adaptable student who has read every brain recording available, not just one specific type. Here is how it works, using simple analogies:

1. The Problem: The "One-Size-Fits-None" Puzzle

Previous AI models were like rigid Lego sets. If you tried to build a castle with a set designed for a spaceship, it fell apart.

The Issue: Brain recordings vary wildly. Some have electrodes in a grid, some in a line. Some record for a split second, others for minutes.
The Old Way: Previous models forced all data into a single, fixed shape, losing important details or failing to connect the dots between different parts of the brain.

2. The Solution: The "Shape-Shifting" Brain

DIVER-1 is built differently. It doesn't force the data into a box; it molds itself to the data.

The "Any-Variable" Attention: Imagine a group of people in a room where everyone can talk to everyone else instantly, regardless of where they are standing or what time it is. DIVER-1 does this with brain signals. It lets every "microphone" talk to every other "microphone" at every moment in time, figuring out how they work together without needing a fixed map.
The "Adaptive" Input: If you give DIVER-1 a recording with 5 electrodes, it works. If you give it 50, it works. It doesn't care about the layout; it learns the relationships between the signals, not just their positions.

3. The Training: The "Marathon" vs. The "Sprint"

The researchers didn't just train DIVER-1 on a small dataset. They fed it a massive amount of data: 5,310 hours of brain recordings from 37 different people.

The Scale: This is about 54 times more data than what previous models used. It's the difference between reading a single comic book and reading the entire library of Congress.
The Method: Instead of just listening to the raw sound, DIVER-1 was trained to "fill in the blanks." The researchers would hide parts of the recording (like muting a song for a few seconds) and ask the AI to guess what was missing based on the rest of the song. This forced the AI to learn the deep structure of brain activity.

4. The Results: Beating the Experts

The team tested DIVER-1 on two very different challenges:

The "Movie Watcher" Test (Neuroprobe): They asked the AI to guess what a person was thinking about while watching a movie (e.g., "Is the volume loud?" "Is a word being spoken?").
- The Result: DIVER-1 was the first model to beat a simple, standard baseline. Even though it was trained on different people than the ones being tested, it understood the brain signals better than models trained on the exact same people.
The "Seizure" Test (MAYO): They asked the AI to detect seizures (sudden electrical storms in the brain).
- The Result: DIVER-1 was the best at spotting these events, outperforming all previous models.

5. The Big Discovery: Data Over Size

The most surprising finding was about how to build a better AI.

The Old Belief: "If you want a smarter AI, just make it bigger (add more parameters/brain cells)."
The DIVER-1 Discovery: "No, make the AI read more books first."
- The researchers found that for brain data, more data and longer training are far more important than making the model huge.
- The Analogy: Imagine training a chef. You can give them a giant, expensive kitchen (a huge model), but if they only practice on one recipe for one day, they won't be a master. It's better to give them a small kitchen but let them cook thousands of different meals for a long time. DIVER-1 proved that for brain signals, variety and repetition matter more than raw size.

Summary

DIVER-1 is a new type of AI that can listen to any brain, no matter how the electrodes are arranged. By training on a massive, diverse library of brain recordings and focusing on learning the patterns rather than just memorizing positions, it has become the best tool yet for decoding thoughts and detecting seizures. The key lesson? In the world of brain AI, quantity and diversity of data beat model size every time.

Technical Summary: DIVER-1

Problem Statement

Intracranial EEG (iEEG) offers direct, millisecond-scale recordings of human neural activity, making it valuable for cognitive decoding, brain-computer interfaces, and precision neuromedicine. However, developing reusable foundation models for iEEG has been hindered by three primary bottlenecks:

Restricted Attention Architectures: Existing models often treat electrodes independently or separate spatial and temporal processing, failing to capture direct cross-channel and cross-time interactions essential for modeling the brain's delayed inter-regional coordination.
Fixed Pretraining Interfaces: Clinical iEEG data varies significantly across patients in channel count, anatomical coverage, electrode type (SEEG vs. ECoG), and referencing schemes. Current models assume fixed input windows and channel layouts, limiting their ability to generalize to variable clinical scenarios or different temporal contexts (e.g., sub-second decoding vs. multi-second seizure detection).
Under-Scaled Pretraining: Previous iEEG foundation models (e.g., BrainBERT, PopT, BaRISTA) have been trained on relatively small datasets (e.g., the ~40–43 hour BrainTreeBank), often restricted to pediatric movie-watching recordings. This limits the scale and diversity of pretraining compared to the vast, heterogeneous data available in clinical monitoring. Furthermore, the field lacks controlled, compute-aware scaling laws to determine whether performance gains should come from increasing data volume, model parameters, or training duration.

Methodology: DIVER-1 Architecture and Training

The authors introduce DIVER-1, a self-supervised iEEG foundation model designed to address these bottlenecks through a novel architecture and large-scale pretraining strategy.

Core Architectural Components

Any-Variate Electrode–Time Attention: Unlike standard Transformers that restrict attention to temporal or spatial axes, DIVER-1 employs full self-attention over all electrode–time tokens. To make this expressive without assuming a fixed montage, it injects spatio-temporal structure directly into the attention mechanism using:
- RoPE (Rotary Positional Embeddings): Encodes temporal offsets.
- Learnable Biases: Distinguishes between same-channel and cross-channel interactions, preserving channel permutation equivariance even without explicit spatial metadata.
Input-Conditioned Encoding: To handle variable electrode layouts and referencing schemes, DIVER-1 utilizes:
- Spatio-Temporal Conditional Positional Embedding (STCPE): A sliding-window mechanism based on the MOIRAI architecture that provides local spatio-temporal bias without relying on fixed channel order or absolute coordinates.
- Spatio-Temporal Resampling (STR): During pretraining, the model samples random subsets of channels and temporal patches from 30-second segments. This forces the model to learn robust representations from variable input sizes and contexts.
- Multi-Domain Reconstruction (MDRO): The model is trained to reconstruct masked patches across multiple signal domains (raw time-domain, FFT frequency spectra, and STFT time-frequency representations), rather than just raw waveforms.
Positional and Spectral Embeddings: The model incorporates channel metadata (3D MNI coordinates, electrode type) and spectral features (FFT-transformed patches) as input embeddings.

Pretraining and Scaling Study

Dataset: DIVER-1 was pretrained on 5,310 hours of iEEG data (352k channel-hours) spanning 37 subjects, comprising both ECoG and SEEG recordings from multiple centers. This volume is approximately 54× larger than the BrainTreeBank corpus used for prior models.
Model Variants: Two variants were instantiated to match different temporal granularities: DIVER-1-0.1s (50-sample patches) and DIVER-1-1s (500-sample patches).
Scaling Law Analysis: The authors conducted the first controlled, compute-aware scaling study for self-supervised iEEG pretraining. They swept data scale, subject count, training duration, and model size (up to 1.83B parameters) using maximal update parameterization (µP) to ensure stable hyperparameter transfer across model widths.

Key Results

Downstream Performance

DIVER-1 was evaluated on two held-out benchmarks: Neuroprobe (15-task naturalistic cognitive decoding) and MAYO (seizure detection).

Neuroprobe: DIVER-1-0.1s achieved the best mean AUROC among all evaluated iEEG foundation models and supervised baselines. Notably, it surpassed the linear spectrogram decoder, reversing the previous finding that pretrained iEEG models underperform simple linear baselines under leakage-aware evaluation. It achieved this despite being pretrained on external adult data, while Neuroprobe consists of pediatric data, demonstrating strong cross-dataset transfer.
MAYO: DIVER-1-1s achieved the top AUROC for seizure detection, outperforming BrainBERT and Brant.
Linear Probing: The frozen DIVER-1-0.1s model was highly competitive, often outperforming its fully fine-tuned counterpart on Neuroprobe, indicating that the pretrained representations are already linearly useful.

Scaling Law Findings

The scaling study revealed a data-constrained regime for iEEG foundation models:

Data vs. Parameters: Increasing unique data recordings and training duration yielded more reliable performance gains than increasing parameter count alone.
Compute-Optimal Allocation:
- When new data is available, the optimal strategy is to scale data first, then training duration, and finally model parameters.
- When the corpus is fixed, the optimal strategy is to train longer (more passes) on smaller-to-mid-sized models rather than training very large models briefly.
Diminishing Returns: The study quantified diminishing returns for repeated data ( $R^*_D \approx 9.5$ epochs) and excessive parameters ( $R^*_N \approx 3.4$ ), suggesting that gains are best pursued by scaling dataset repetition while keeping model sizes moderate.

Significance and Claims

The paper claims DIVER-1 represents a significant step forward in iEEG foundation modeling by:

Realizing Transferable Representations: Demonstrating that large-scale, heterogeneous pretraining can produce representations that transfer across subjects, centers, and clinical tasks, overcoming the limitations of previous models trained on narrow, small-scale datasets.
Establishing a New Scaling Paradigm: Providing the first controlled evidence that iEEG pretraining is data-constrained. The authors argue that the community should prioritize expanding unique recording volumes and training duration over simply increasing model parameter counts, a finding that contrasts with the parameter-heavy scaling laws often observed in language models.
Architectural Innovation: Introducing an architecture that natively handles the variability of clinical iEEG (variable channels, referencing, and temporal contexts) through any-variate attention and input-conditioned positional embeddings, enabling robust modeling of spatio-temporal neural dynamics without fixed montages.

The authors conclude that the path forward for iEEG foundation models involves "scaling data first, training longer second, and scaling parameters last."