🤖 AI

Benchmarking Positional Encoding Strategies for Transformer-Based EEG Foundation Models

This paper benchmarks five positional encoding strategies within the CBraMod transformer backbone for EEG foundation models, revealing that no single strategy universally outperforms others across tasks, with Spherical Positional Encoding excelling in motor imagery and Asymmetric Conditional Positional Encoding offering more consistent performance across diverse decoding scenarios.

Original authors: Ayse Betul Yuce, Sebastian Stober

Published 2026-05-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ayse Betul Yuce, Sebastian Stober

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine your brain is a massive orchestra, and EEG (Electroencephalography) is the technique we use to record the music it plays. To understand this music, scientists use powerful AI models called Transformers. These models are like super-smart conductors who can listen to the recording and figure out what the musicians (your brain cells) are doing.

However, there's a catch: Transformers are naturally "tone-deaf" to where things are happening. If you shuffle the order of the musicians in the orchestra, the Transformer hears the exact same song. But in the real world, the position of an electrode on your head matters immensely. An electrode on your forehead hears different "music" than one on the back of your head.

This paper is a taste test to see which method works best for teaching these AI conductors where the electrodes are located. The researchers took a standard AI model (called CBraMod) and swapped out its "location guide" with five different strategies to see which one helped the AI understand brain signals better.

Here is a breakdown of the five "location guides" they tested:

1. The "No Map" Strategy (NoPE)

This is the control group. The AI is given the brain signals but no information at all about where the electrodes are. It's like giving a conductor a sheet of music but telling them, "You don't need to know which instrument is playing which note; just guess."

Result: The AI struggled the most. It proved that knowing where the signals come from is crucial for learning.

2. The "Flexible Neighborhood" Guide (ACPE)

This method is like a smart map that looks at the immediate neighborhood of each signal. Instead of looking at a fixed map of the whole head, it asks, "Who is sitting next to this electrode right now?" It uses a special math trick (convolutions) to figure out the layout dynamically.

Result: This was the most consistent performer. It did well on both "Motor Imagery" (imagining moving your hand) and "Emotion Recognition" (feeling happy or sad). It's like a conductor who adapts to the room every time they walk in.

3. The "Globe" Guide (SPE - Spherical Positional Encoding)

The authors invented this one. Since the human head is round, they treated the electrodes like points on a globe. They mapped every electrode using angles (like latitude and longitude) rather than a flat grid.

Result: This was a specialist.
- For Motor Imagery: It was the champion. Imagining movement seems to rely heavily on the specific shape of the head, so this "globe" map worked perfectly.
- For Emotions: It stumbled. Emotions might not care as much about the exact geometric shape of the head, so this rigid map wasn't as helpful.

4. The "Globe with a Translator" (SPE+Proj)

This is the "Globe" guide, but with a twist: they added a small, learnable layer that tries to "translate" the globe map into something the AI likes better.

Result: It didn't help much. It seems the original "Globe" map was already good enough, and the translator just made things slightly messy.

5. The "Blank Slate" Guide (Learnable PE)

Here, the AI is given a blank map and told, "You figure out the locations yourself while you learn." It starts with no knowledge and learns the electrode positions from scratch for every new task.

Result: It was a surprise winner for Emotions. When the AI was allowed to relearn the map specifically for the emotion task, it performed the best. However, this is a heavy lift; it requires the AI to relearn everything every time it switches tasks, which isn't always efficient.

The Big Takeaway

The researchers found that there is no single "perfect" map for all brain tasks.

If you want to detect movement thoughts, a geometric map that respects the round shape of the head (SPE) is excellent.
If you want to detect emotions, a flexible, neighborhood-based map (ACPE) or letting the AI learn the map from scratch (Learnable PE) works better.
If you need a solution that works consistently well across different tasks without needing to retrain everything, the Flexible Neighborhood guide (ACPE) is the most reliable choice.

In short, just like you wouldn't use a flat street map to navigate a mountain range, you shouldn't use one single "location guide" for every type of brain activity. The best strategy depends entirely on what the brain is doing at that moment.

Technical Summary: Benchmarking Positional Encoding Strategies for Transformer-Based EEG Foundation Models

Problem Statement

Electroencephalography (EEG) is a critical non-invasive technique for brain–computer interface (BCI) applications, yet supervised decoding models struggle to generalize across tasks, subjects, and datasets due to low signal-to-noise ratios and high inter-subject variability. To address this, transformer-based foundation models trained with self-supervised learning have emerged. However, transformers are inherently permutation-invariant and require explicit positional encoding (PE) to model structural dependencies.

Unlike textual tokens, EEG electrodes possess a specific spatial distribution across the scalp. Current transformer-based EEG foundation models employ diverse and often uncomparable mechanisms to encode this spatial-temporal structure, including channel-specific embeddings, relative temporal encoding, and learned embeddings. There is a lack of systematic evaluation regarding which positional encoding strategy is optimal for EEG decoding, particularly given the anatomical meaningfulness of electrode placement and the variability in channel montages across datasets.

Methodology

The authors conducted a controlled benchmark within the CBraMod backbone, a self-supervised EEG foundation model utilizing a criss-cross attention mechanism to capture spatio-temporal dependencies. To isolate the impact of positional design, the study retained the original transformer architecture, training protocols, and hyperparameters, replacing only the positional encoding module.

Positional Encoding Strategies Evaluated

Five strategies were compared:

No Positional Encoding (NoPE): A baseline with no explicit positional information.
Asymmetric Conditional Positional Encoding (ACPE): A method where positional representations are dynamically generated from spatio-temporal neighborhoods of EEG patches using depthwise 2D convolutions with asymmetric kernels ( $k_s > k_t$ ). This captures long-range inter-channel dependencies and short-range temporal context without relying on absolute coordinates.
Spherical Positional Encoding (SPE): A novel strategy proposed in this work that leverages the spherical geometry of electrode placement. It computes azimuth ( $\theta$ ) and inclination ( $\phi$ ) angles for each electrode and maps them to fixed-length representations using multi-frequency sinusoidal functions. This encodes angular relationships invariant to head size and scaling, without learnable parameters.
SPE with Projection (SPE+Proj): An extension of SPE where channel and patch embeddings are passed through separate learnable linear projections before summation.
Learnable PE: Fully learnable channel and patch embeddings summed to form the final encoding. These are reinitialized and learned from scratch during fine-tuning if the downstream montage differs from pretraining.

Experimental Setup

Pretraining: Conducted on the Healthy Brain Network EEG (HBN-EEG) dataset (>3,000 participants, 128 channels) using a masked patch reconstruction objective.
Downstream Tasks:
- Motor Imagery (MI): Classification on the PhysioNet MI dataset (64 channels, 4 classes).
- Emotion Recognition: Classification on the FACED dataset (32 channels, 9 classes).
Evaluation Protocols: Models were evaluated under linear probing (frozen backbone, trained classifier) and fine-tuning (all weights updated).
Metrics: Balanced Accuracy, Cohen's Kappa, and Weighted F1 score.

Key Results

The study found that no single positional encoding strategy consistently outperformed others across all tasks and protocols. Performance was highly task-dependent:

Motor Imagery (PhysioNet-MI):
- SPE achieved the best performance in the linear probing protocol, suggesting that explicit geometric priors are beneficial when spatial activation patterns are distinct.
- ACPE achieved the best performance in the fine-tuning protocol.
- NoPE was consistently the weakest in linear probing but remained competitive after fine-tuning, indicating the backbone can partially recover positional structure through end-to-end optimization.
Emotion Recognition (FACED):
- ACPE and Learnable PE yielded the best results.
- SPE underperformed significantly compared to other strategies in both protocols, suggesting that strict geometric priors may not benefit emotion-related representations as much as motor imagery.
Pretraining Loss: The absence of positional encoding resulted in the highest masked reconstruction loss, confirming that positional information is crucial for pretraining representation quality. Learnable PE and SPE+Proj achieved the lowest validation losses.

Significance and Conclusions

The paper concludes that the optimal positional encoding strategy for EEG foundation models is task-dependent, with no universal solution.

Task-Specificity: Strategies that encode explicit anatomical geometry (SPE) are advantageous for tasks with strong spatial distinguishability (e.g., motor imagery) but may be suboptimal for tasks where spatial patterns are less rigid (e.g., emotion recognition).
Generalizability vs. Adaptability: ACPE demonstrates consistent performance by learning local neighborhood patterns from patch content, making it robust across diverse tasks without requiring explicit coordinate knowledge. Conversely, Learnable PE, while requiring re-initialization for different montages, can outperform fixed strategies when the model is allowed to learn task-specific spatial structures from scratch on the target dataset.
Practical Implications: SPE offers a lightweight, parameter-free, and montage-agnostic alternative that is precomputed from electrode angles, making it a viable option when spatial structure is relevant and retraining is undesirable.

The authors emphasize that while performance differences were relatively small, they are measurable and significant given that only the positional encoding module was altered. Future work is suggested to explore these strategies across more diverse EEG tasks, electrode montages, and larger pretraining datasets to further assess the generalizability of topology-aware encoding.