⚡ electrical engineering

Cross-Subject Intracranial EEG Reconstruction from Scalp Recordings Using Multi-Scale Cross-Attention Transformers

This paper introduces CAST, a multi-scale cross-attention transformer framework that enables the reconstruction of intracranial EEG signals for unseen subjects from non-invasive scalp recordings using a two-stage transfer learning strategy, achieving high correlations in cortical regions with only minimal subject-specific calibration.

Original authors: Tien-Dat Pham, Xuan-The Tran

Published 2026-05-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Tien-Dat Pham, Xuan-The Tran

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Hearing the Whisper from the Street

Imagine you are standing on a busy street (the scalp). You want to know exactly what a specific conversation is happening inside a house down the street (the brain).

Usually, to hear that conversation clearly, you have to break into the house and put a microphone right next to the speakers. This is like Intracranial EEG (iEEG). It gives you perfect, high-quality audio, but it requires dangerous surgery to get there.

The problem is that we want to hear these conversations without breaking in. We want to use a microphone on the street (Scalp EEG) to guess what's happening inside. The catch? The sound gets muffled, distorted, and quiet as it travels through the walls, the roof, and the air.

The Problem: The "One-Size-Fits-None" Trap

Scientists have tried to build a machine that translates street noise into house conversations. But most previous machines had a fatal flaw: they needed to be "trained" on the specific house they were listening to.

To train the machine, you had to break into the house first to record the real conversation, then teach the machine how to translate the street noise. This creates a silly loop: You need surgery to train the machine that is supposed to help you avoid surgery.

The Solution: The "CAST" Translator

The authors of this paper built a new system called CAST (Cross-Attention Spatial-Temporal Transformer). Think of CAST as a super-smart translator that learns the general rules of how sound travels through different types of houses, so it can guess the conversation in a new house it has never seen before.

Here is how it works, step-by-step:

1. The Universal Listener (The Encoder)

Imagine a detective who has listened to thousands of different houses. They don't know the specific layout of your house yet, but they know the general rules: "If the street noise sounds like a drumbeat, it's probably a party in the living room," or "If it's a low hum, it's the fridge in the kitchen."

CAST's Encoder does this. It looks at the scalp EEG (street noise) and learns universal patterns of brain activity. It doesn't care who the patient is; it just learns how brain signals generally look.

2. The Custom Adapter (The Decoder)

Now, imagine you bring this detective to a new house. Even though the detective knows the rules, every house has a different floor plan. The kitchen might be on the left in one house and the right in another.

To fix this, CAST uses a Decoder that acts like a custom adapter. It takes a tiny bit of data from the new patient (just 2 to 12 minutes of recording) to "calibrate" itself. It's like the detective saying, "Okay, I know the rules, but in this house, the kitchen is on the left, so I'll adjust my guess accordingly."

Once calibrated, the system can predict the inside conversation for the rest of the time without needing more surgery.

What Did They Find?

1. The "Wall Thickness" Rule

The paper discovered something very logical: The system works best when the "walls" between the street and the room are thin.

Surface Rooms (Cortex): If the brain activity is happening near the surface of the brain (close to the skull), the signal is strong. The system was incredibly accurate here. For example, in the area controlling hand movement (the precentral gyrus), it matched the real signal with 86% accuracy.
Basement Rooms (Deep Structures): If the activity is deep inside the brain (like the hippocampus or amygdala), the signal has to travel through a lot of tissue and bone. By the time it reaches the scalp, it's very faint. The system struggled here, matching only about 20% of the signal.

The Analogy: It's like trying to hear a whisper from the basement. Even with a super-microphone on the street, the walls are too thick. The fact that the system failed more on deep signals actually proves it's working correctly—it's respecting the laws of physics!

2. The "Good Enough" Filter

The researchers realized that not every room in the house is worth guessing. Some rooms are just too far away to hear.
They created a strategy to pick only the "observable" channels—the brain areas where the signal is strong enough to be heard clearly. When they focused only on these areas, the system's accuracy jumped significantly, reaching 54% correlation on average for patients who had enough surface brain data.

The Bottom Line

This paper shows that we can finally build a system that translates scalp EEG into intracranial EEG for new patients without needing to do surgery on them first to train the model.

It works best for brain areas near the surface (like the motor cortex).
It struggles with deep brain areas (due to physics, not bad math).
It needs a very short "warm-up" period (2–12 minutes) to adjust to the new patient's specific brain shape.

In short, CAST is like a universal translator that learns the general language of the brain, then quickly learns the specific accent of a new person, allowing us to "hear" the brain's surface conversations without ever needing to break the skull.

Technical Summary: Cross-Subject Intracranial EEG Reconstruction from Scalp Recordings Using Multi-Scale Cross-Attention Transformers

Problem Statement
Intracranial EEG (iEEG) offers high-fidelity neural recordings with millimeter spatial resolution and millisecond temporal precision, essential for clinical epilepsy monitoring and brain-computer interface (BCI) applications. However, acquiring iEEG requires invasive neurosurgery, limiting its use to specific clinical populations and restricting the study of brain regions to those targeted for implantation. While recent efforts have attempted to estimate iEEG from non-invasive scalp EEG, existing methods predominantly rely on patient-specific (within-subject) training. This creates a circular dependency: if a patient requires invasive surgery to generate the training data necessary for a non-invasive model, the non-invasive approach offers little practical advantage over direct recording. Furthermore, existing cross-subject approaches often focus on classification tasks rather than waveform-level reconstruction, or they require individual structural MRI data to account for anatomical variability.

Methodology: The CAST Framework
The authors propose CAST (Cross-Attention Spatial-Temporal Transformer), a machine learning framework designed to translate scalp EEG into multi-channel iEEG waveforms for new, unseen subjects using a two-stage transfer learning approach.

Architecture:
- Multi-Scale Encoder: The model processes input scalp EEG signals by slicing them into non-overlapping patches at three distinct time scales (8, 16, and 32 samples). These patches are projected into a latent space ( $d_{model}=128$ ) via MLPs and combined with scale-specific and positional embeddings. A 4-layer Transformer encoder with multi-head self-attention processes these concatenated tokens to capture broad neural representations across high-frequency transients and slower oscillations.
- Channel-Aware Decoder: To address the variability in electrode placement and count across patients, the decoder utilizes a cross-attention mechanism with shared temporal query tokens ( $M=25$ ) and learnable per-channel embeddings. This allows the model to reconstruct specific target iEEG channels independently based on the encoded scalp features.
Training Protocol (Leave-One-Subject-Out):
- Universal Encoder Pre-training: To prevent catastrophic forgetting and overfitting to specific anatomies, the encoder is pre-trained sequentially on data from all subjects except the test subject.
- Decoder Calibration: Recognizing that hardware setups vary, the decoder is treated as a personalized projection tool. For a new patient, the decoder is initialized and fine-tuned using a brief calibration phase (20% of the patient's data, approximately 2–12 minutes).
- Channel Selection Strategy: The authors introduce a data-driven strategy to identify "observable" channels. Channels achieving a Pearson correlation $r \ge 0.15$ during a warm-up phase on calibration data are selected for final reconstruction, filtering out channels where the signal is too attenuated to be recovered.
Loss Function: The model is optimized using a composite loss function balancing amplitude accuracy (MSE), waveform shape (Pearson correlation penalty), and frequency content (Spectral loss).

Datasets and Evaluation
The study was evaluated on two independent public datasets:

GIN Cohort: 9 patients with drug-resistant focal epilepsy performing a verbal working memory task (17–21 scalp channels, 32–64 iEEG contacts).
OpenNeuro ds004752: 14 patients (originally 15) performing a visual categorization task (6–21 scalp channels, 32–80 iEEG contacts).
Evaluation utilized a Leave-One-Subject-Out (LOSO) cross-validation strategy across a total of 1,282 iEEG channels.

Key Results

Cross-Subject Generalization: CAST successfully reconstructed cortical iEEG signals for new subjects without extensive patient-specific training. In highly observable sensorimotor regions, the model achieved peak correlations of $r = 0.864$ in the precentral gyrus and $r = 0.732$ in the postcentral gyrus.
Depth Gradient Validation: The study confirmed a statistically significant depth gradient ( $p = 0.014$ via Linear Mixed-Effects Models). Cortical contacts ( $N=313$ , mean $r = 0.221$ ) were reconstructed with significantly higher fidelity than deep subcortical structures ( $N=507$ , mean $r = 0.182$ ), such as the hippocampus ( $r=0.197$ ) and amygdala ( $r=0.233$ ). This aligns with volume conduction theory, where signals from deep structures attenuate significantly before reaching the scalp.
Performance on Viable Subjects: By applying the channel selection strategy, the model achieved a mean correlation of $r = 0.545$ ( $R^2 = 0.333$ ) on "viable" subjects (those with sufficient observable channels) in the ds004752 dataset, surpassing previous within-subject baselines and other cross-subject methods that either require MRI or focus on classification.
Ablation Studies: The multi-scale temporal modeling and the inclusion of the Pearson correlation penalty in the loss function were identified as critical components for maximizing waveform fidelity.

Significance and Claims
The paper claims that CAST demonstrates the feasibility of reconstructing cortical iEEG signals from scalp EEG for new subjects without the need for extensive patient-specific training data or individual structural MRI scans. The authors argue that:

Breaking the Circular Dependency: By using a universal encoder and a brief calibration phase, the model offers a practical alternative to invasive data collection for cortical signal estimation.
Physiological Consistency: The observed depth gradient serves as independent physiological validation, suggesting the model learns real neural signal relationships governed by volume conduction physics rather than spurious correlations.
Clinical and BCI Utility: The ability to accurately reconstruct signals in sensorimotor regions (where correlations exceeded 0.7) suggests potential applications in non-invasive BCIs and as a screening tool to guide electrode placement in epilepsy surgery planning.

The authors remain modest regarding deep brain structures, acknowledging that reconstruction accuracy for deep regions (e.g., hippocampus) remains limited by fundamental physical constraints of signal attenuation, noting that even advanced within-subject models with MRI data struggle to achieve high correlations in these areas. They conclude that while a brief calibration phase is necessary to adapt to new hardware, the approach represents a significant step toward hardware-agnostic neural translation.