🤖 machine learning

RECTOR: Masked Region-Channel-Temporal Modeling for Affective and Cognitive Representation Learning

RECTOR is a novel self-supervised framework that employs masked region-channel-temporal modeling and adaptive functional partitioning to achieve state-of-the-art performance in EEG/sEEG-based affective and cognitive disorder diagnosis while offering robustness to missing channels and interpretable insights.

Original authors: Jinhan Liu, Mahsa Shoaran

Published 2026-06-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jinhan Liu, Mahsa Shoaran

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine your brain is a massive, bustling city with thousands of neighborhoods (regions) and millions of individual citizens (channels) sending messages to each other every second. To understand how this city feels (emotions) or how hard it's working (cognitive tasks), we listen to its radio signals (EEG/sEEG).

The problem is that the city is chaotic. The neighborhoods change their roles depending on what's happening, and sometimes the radio stations go silent (missing channels). Traditional methods try to listen to this city using a rigid, pre-drawn map that never changes. If a neighborhood changes its function, the old map fails. If a radio station goes silent, the map breaks.

Enter RECTOR, a new AI system that acts like a super-smart, adaptive city planner. Here is how it works, broken down into simple concepts:

1. The Flexible Map (Adaptive Functional Partitioning)

Most AI models look at the brain using a fixed map based on anatomy (e.g., "This area is always the 'Frontal Lobe'").

The Old Way: Like trying to navigate a city using a map from 1950. It doesn't know about new neighborhoods or changed traffic patterns.
The RECTOR Way: RECTOR starts with a basic map but learns to redraw it in real-time. It realizes that during a specific task, a group of "citizens" (channels) from different parts of the city might suddenly start working together as a new, temporary team. It groups them dynamically based on what they are actually doing, not just where they are sitting. This allows it to understand the brain's fluid, changing nature.

2. The "Blindfold" Training Game (Masked Topology and Representation Learning)

To teach this AI without needing a million labeled doctors' notes (which are rare), the researchers use a game called "Fill in the Blanks."

The Game: The AI is shown a brain signal, but a big chunk of it is covered with a blindfold (masked).
The Challenge: The AI must guess what's under the blindfold. But it's not just guessing random noise; it has to guess three things at once:
1. The Signal: What did the missing radio waves sound like?
2. The Map: Which neighborhoods were those missing signals part of? (Did they belong to the "Emotion Team" or the "Focus Team"?)
3. The Consistency: If we look at the same brain signal from a slightly different angle (a different view), does the story still make sense?

By playing this game millions of times, the AI learns the deep, hidden rules of how the brain's city operates, rather than just memorizing surface patterns.

3. The Efficient Detective (Hierarchical Attention)

Looking at every single citizen in a city of millions is slow and confusing.

The Old Way: Trying to listen to every single person in the city at once. It's too much data, and you miss the big picture.
The RECTOR Way: RECTOR acts like a smart detective who uses layers of focus.
- First, it looks at the Neighborhoods (Regions) to get the big picture.
- Then, it zooms in on specific Citizens (Channels) within those neighborhoods to get the fine details.
- Crucially, it uses a "Top-p Gating" mechanism, which is like a bouncer at a club. It instantly ignores the noisy, unimportant conversations and only lets the most important signals through. This makes it incredibly fast and efficient.

Why It Matters (According to the Paper)

The paper claims RECTOR is a huge leap forward because:

It's Robust: If some radio stations (channels) go silent or are missing, RECTOR doesn't crash. Because it understands the relationships between neighborhoods, it can fill in the gaps using the rest of the city's data.
It Generalizes: It works well even when the "city layout" changes (different electrode setups). It doesn't get confused by new hardware; it adapts its flexible map.
It's Accurate: On tests involving emotion recognition and task engagement, it beat all previous state-of-the-art models.
It's Explainable: Unlike a "black box" that gives an answer without explaining why, RECTOR can show you which neighborhoods it thought were most important for a specific emotion or task. This gives doctors a clear view of the brain's activity.

In short, RECTOR is a brain-signal AI that doesn't just memorize a static map; it learns to dynamically reorganize the city as it listens, making it a powerful tool for understanding the complex, shifting dynamics of the human mind.

Technical Summary: RECTOR - Masked Region-Channel-Temporal Modeling for Affective and Cognitive Representation Learning

1. Problem Statement

Affective and cognitive disorders manifest as distributed, time-varying brain network dynamics across regions, channels, and time. Current approaches to learning robust representations from EEG and stereo-EEG (sEEG) for clinical diagnosis face three primary challenges:

Rigid Anatomical Priors: Existing methods (e.g., Graph Neural Networks, Transformers) often rely on fixed anatomical definitions (e.g., standard lobes) to model regional interactions. These fail to capture the adaptive functional structures that evolve during cognitive processing, limiting robustness against missing channels and preventing cross-montage generalization.
Superficial Learning: Standard self-supervised learning (SSL) with random masking often encourages models to learn spatio-temporal shortcuts rather than generalizable features. Furthermore, many methods overlook topological structure modeling, leading to representations that lack neurophysiological plausibility.
Computational and Structural Inefficiency: Full spatio-temporal self-attention is computationally prohibitive for high-density neural data and risks amplifying uninformative relationships. Conversely, decoupled spatial and temporal modeling fails to capture joint interactions, while population-level spatial encoding ignores established functional network architectures.

2. Methodology

The authors propose RECTOR (Masked Region-Channel-Temporal Modeling), an end-to-end self-supervised framework designed to unify joint region-channel-temporal representation learning.

2.1. Core Architecture: RECTOR-SA

The backbone of the framework is RECTOR-SA, a hierarchical, block-sparse self-attention mechanism driven by Adaptive Functional Partitioning (AFP).

Hierarchical Tokenization: The model explicitly models the brain's hierarchy by introducing region tokens ( $X_R$ ) alongside channel tokens ( $X_C$ ).
Adaptive Functional Partitioning (AFP): Unlike fixed priors, AFP learns a soft channel-to-region assignment matrix ( $\Pi_X$ ) that evolves from an initial anatomical prior (e.g., 10-20 system or electrode labeling algorithms) into task-specific functional regions during training. This is achieved via a learnable assignment head using Gumbel-Softmax for discrete topology.
Block-Sparse Attention: Instead of dense attention over all tokens, RECTOR-SA factorizes attention into three parallel, sparse mechanisms:
1. Topological Attention: Models interactions between channels and their assigned regions.
2. Local Functional Attention: Captures fine-grained interactions within assigned regions over time.
3. Global Functional Attention: Models high-level coordination among regions.
Top-p Gating: A dynamic gating operator prunes weak connections, acting as a learned denoiser to focus on salient topological interactions.

2.2. Learning Framework: Masked Topology and Representation Learning (MTRL)

RECTOR employs a unified SSL paradigm that optimizes three complementary objectives in a single forward pass:

Masked Predictive Modeling (MPM): The model reconstructs masked input signals and representations in both input and latent spaces. This couples local signal reconstruction with representation alignment to prevent representational collapse.
Topological Structure Modeling (TSM): This objective explicitly regularizes the learned region-channel assignments. It forces the model to predict masked topological assignments in the latent space and aligns them with the input-space assignments, ensuring the learned topology remains faithful to the data structure.
Cross-View Consistency (CVC): Using Topology-Aware Multi-View Masking, the model generates multiple target views from distinct topological domains (region, channel, time, region-time, channel-time). CVC enforces invariance across these diverse views to learn robust semantic representations.

The total pre-training objective combines these losses with a Region-Channel Regularization term to disentangle region-common and channel-specific information.

3. Key Contributions

Unified Architecture: Introduction of RECTOR-SA, which moves beyond fixed anatomical priors by evolving static definitions into adaptive functional regions via AFP, effectively disentangling region-common and channel-specific dynamics.
Holistic SSL Paradigm: The proposal of MTRL, which jointly optimizes predictive modeling, topological structure modeling, and cross-view consistency, addressing the limitations of standard masked modeling that often learns superficial shortcuts.
Efficiency and Robustness: The framework achieves state-of-the-art performance with block-sparse computation, offering superior robustness to missing channels and cross-montage generalization compared to dense attention models.

4. Experimental Results

The authors evaluated RECTOR on diverse benchmarks:

EEG Emotion Recognition: On SEED, SEED-IV, and DEAP datasets, RECTOR achieved new state-of-the-art results in both subject-dependent (SD) and subject-independent (SI) settings, consistently outperforming leading task-specific models (e.g., MMM, mdJPT) and foundation models (e.g., PopT, CBraMod) by an average margin of >3%.
sEEG Task Engagement: On MSIT and ECR datasets, RECTOR significantly outperformed supervised baselines (Seegnificant) and self-supervised methods (Brant, Du-IN, BaRISTA) in task engagement classification.
Robustness and Generalization:
- Missing Channels: RECTOR demonstrated superior stability under random channel dropout (up to 50%), maintaining performance where baselines degraded significantly.
- Cross-Montage: The model showed minimal performance decay when transferring representations between different sensor layouts (e.g., DEAP to SEED), whereas fixed-topology models suffered significant drops.
Scalability: Performance improved consistently with larger pre-training regimes (more data and model size), indicating strong scaling properties.

5. Significance and Claims

The paper claims that RECTOR represents a significant advancement in neural representation learning by:

Bridging the Gap: Successfully unifying region, channel, and temporal modeling without relying on rigid anatomical constraints, thereby capturing the adaptive nature of brain network dynamics.
Clinical Potential: Providing a foundation for large-scale pre-training on heterogeneous EEG/sEEG data, addressing critical barriers to clinical deployment such as data heterogeneity and hardware variability (sensor failure/montage differences).
Interpretability: Offering deep interpretability through multi-level visualizations. The authors demonstrate that AFP learns stable, spatially coherent refinements beyond anatomical priors (reassigning ~36-51% of channels) that align with known neurophysiological patterns (e.g., frontal asymmetry in emotion processing).

The authors position RECTOR not merely as a performance booster but as a framework that enables more interpretable neurocognitive diagnostics and adaptive, personalized neuro-interventions by learning neurophysiologically plausible representations.