Boundary-aware Prototype-driven Adversarial Alignment for Cross-Corpus EEG Emotion Recognition

This paper proposes a unified Prototype-driven Adversarial Alignment (PAA) framework that addresses cross-corpus EEG emotion recognition challenges by integrating prototype-guided subdomain alignment, contrastive semantic regularization, and boundary-aware adversarial optimization to achieve state-of-the-art performance and robustness across heterogeneous datasets.

The Gist

The Big Problem: The "Accent" Barrier

Imagine you are teaching a robot to recognize human emotions by looking at brainwaves (EEG). You train this robot using data from Group A (people in a quiet lab in China). The robot gets really good at it.

But then, you try to use that same robot on Group B (people in a different lab in a different country, using different machines). Suddenly, the robot fails miserably.

Why?
Think of it like an accent. Even though everyone is speaking the same language (emotions like "Happy" or "Sad"), the "accent" of the brainwaves changes depending on:

• The Machine: Different EEG headsets record signals slightly differently.
• The Person: Everyone's brain is wired uniquely.
• The Environment: A noisy lab vs. a quiet one.

In the world of AI, this is called the "Cross-Corpus" problem. The robot learned the "Chinese accent" of brainwaves but can't understand the "American accent," even though they are both speaking English.

The Old Way: Blending the Soup

Previous methods tried to fix this by forcing the robot to ignore the differences between the two groups. They tried to blend the data from Group A and Group B into one big, smooth "soup."

The Flaw: Imagine you are trying to teach a student to distinguish between Apples and Oranges. If you just mix all the apples and oranges together in a giant pile and say, "Make them look the same," you lose the ability to tell them apart! The robot gets confused, and the line (decision boundary) between "Happy" and "Sad" gets blurry.

The New Solution: PAA (The "Smart Translator")

This paper proposes a new framework called PAA (Prototype-driven Adversarial Alignment). Instead of just blending the soup, PAA acts like a smart translator that understands the structure of the emotions, not just the raw noise.

They built this translator in three steps, like upgrading a video game character:

Level 1: PAA-L (The "Group Captain" Strategy)

• The Idea: Instead of treating everyone as a faceless crowd, the robot picks a "Captain" (a Prototype) for each emotion.
• The Analogy: Imagine you have a "Happy Captain" and a "Sad Captain." The robot looks at the new people (Group B) and asks, "Who does this person look like? Are you closer to the Happy Captain or the Sad Captain?"
• The Result: It aligns the groups based on their roles (emotions) rather than just their raw data. It says, "Okay, your 'Happy' brainwaves might look different, but they still belong to the 'Happy' team."

Level 2: PAA-C (The "Social Distancing" Strategy)

• The Idea: Now that the teams are aligned, we need to make sure the teams don't mix up.
• The Analogy: Imagine a dance floor. We want all the "Happy" dancers to huddle close together (compactness), and we want the "Happy" dancers to stay far away from the "Sad" dancers (separability).
• The Result: This creates a clear, wide gap between the emotions. It prevents the robot from getting confused when a "Happy" brainwave looks a little bit like a "Sad" one.

Level 3: PAA-M (The "Border Patrol" Strategy)

• The Idea: This is the full, super-powered version. It focuses on the people standing right on the fence between two emotions.
• The Analogy: Imagine a border patrol with two guards (Dual Classifiers).
  • Guard 1 says, "This person is definitely Happy!"
  • Guard 2 says, "No, I think they are Sad!"
  • When the guards disagree, the robot knows, "Ah, this person is a Controversial Sample standing right on the border."
• The Result: The robot specifically targets these confused people. It doesn't ignore them; it trains extra hard to make sure these "borderline" cases get sorted correctly. This fixes the "blurry line" problem that old methods had.

Why This Matters (The Real-World Test)

The researchers tested this on three different datasets (SEED, SEED-IV, SEED-V) and even tried it on a real-world medical problem: detecting depression.

• The Result: The new method (PAA-M) was significantly better than all previous methods. It improved accuracy by about 6% to 7% on average, which is huge in the world of AI.
• The Medical Win: When they used it to detect depression (a condition linked to negative emotions), it worked very well, proving that this "Smart Translator" can handle messy, real-world data where machines and people vary wildly.

Summary

• The Problem: AI gets confused when switching between different brainwave datasets because of "accents" (different machines/people).
• The Old Fix: Just mix everything together (failed because it blurred the lines).
• The New Fix (PAA):
  1. PAA-L: Use "Captains" to group similar emotions.
  2. PAA-C: Push different emotions apart so they don't mix.
  3. PAA-M: Use two "guards" to find and fix the people standing on the border between emotions.

This approach makes the AI robust, meaning it can learn from one group of people and successfully recognize emotions in a completely different group, even with different equipment.

Technical Summary

1. Problem Statement

Electroencephalography (EEG)-based emotion recognition faces a critical bottleneck: severe performance degradation when models are transferred across heterogeneous datasets (cross-corpus).

• Root Causes: Physiological variability, differences in experimental paradigms, recording devices, and subject populations create significant distribution shifts between source and target domains.
• Limitations of Existing Methods: Current domain adversarial methods primarily enforce global marginal alignment. They often overlook:
  1. Class-conditional mismatch: The assumption that source and target domains share identical class-conditional structures is often violated.
  2. Decision boundary distortion: Global alignment can misalign samples near inter-class regions, leading to "negative transfer."
  3. Label Noise: Emotion labels induced by video stimuli may not perfectly reflect internal states, introducing noise that standard models struggle to handle.

2. Methodology: Prototype-driven Adversarial Alignment (PAA)

The authors propose a unified framework, PAA, which reformulates emotion recognition as a relation-driven representation learning problem. The framework is progressively instantiated in three configurations to systematically refine cross-domain consistency.

Core Components

1. Prototype Learning: Instead of global distribution matching, the method uses class centroids (prototypes) in the embedding space to guide local class-conditional alignment.
2. Relation-aware Learning (RaL): A module that transforms single-sample classification into a similarity problem between samples. It captures relational structures ($\phi$) between embeddings and prototypes to suppress label noise.
3. Dual-Classifier Adversarial Mechanism: Two structurally identical but optimization-divergent classifiers ($C_1, C_2$) are used to identify "controversial samples" near decision boundaries by maximizing their prediction discrepancy.

Three Progressive Configurations

• PAA-L (Local Class-Conditional Alignment):
  • Decomposes domain alignment into subdomain-level prototype matching.
  • Uses a Local-MMD (Maximum Mean Discrepancy) loss to align source and target distributions for each class separately, preserving semantic structure while mitigating coarse distribution shifts.
• PAA-C (Contrastive Semantic Enhancement):
  • Builds on PAA-L by adding contrastive semantic regularization.
  • Simultaneously minimizes intra-class discrepancy (compactness) and maximizes inter-class separation (separability) across domains. This prevents feature collapse and refines the geometry of class manifolds.
• PAA-M (Boundary-Aware Aggregation):
  • The full configuration integrating a three-stage adversarial optimization scheme:
    1. Representation Learning: Jointly optimize feature extractor and classifiers to learn basic representations.
    2. Discrepancy Maximization: Freeze the feature extractor; optimize classifiers to maximize disagreement on target samples, explicitly identifying ambiguous/boundary regions.
    3. Boundary Refinement: Freeze classifiers; optimize the feature extractor to minimize the discrepancy for those controversial samples, forcing them toward confident clusters.
  • This explicitly refines decision boundaries and reduces hypothesis divergence.

3. Key Contributions

1. Unified Framework: Proposed PAA, which shifts the paradigm from global marginal matching to class-conditional and boundary-aware feature refinement.
2. Progressive Design: The three-stage instantiation (PAA-L, PAA-C, PAA-M) allows for systematic analysis, moving from subdomain alignment to discriminative regularization and finally to decision boundary stabilization.
3. Novel Optimization Scheme: The PAA-M introduces a dual-classifier architecture with a three-stage training process (Representation $\to$ Discrepancy Maximization $\to$ Boundary Refinement) to explicitly handle samples near decision boundaries.
4. Robustness to Noise: The integration of the Relation-aware Learning (RaL) module significantly reduces sensitivity to noisy emotion labels by focusing on sample-to-sample relationships rather than hard labels.

4. Experimental Results

The framework was evaluated on three public datasets (SEED, SEED-IV, SEED-V) under four rigorous cross-corpus protocols:

1. Cross-corpus Cross-subjects Single-session.
2. Cross-corpus Cross-subjects Cross-session.
3. Leave-one-subject-out (LOSO) on unseen targets (Single-session).
4. LOSO on unseen targets (Cross-session).

Key Findings:

• State-of-the-Art Performance: PAA-M achieved the best results across all protocols.
  • Single-session: Average improvement of 6.72% over the best baseline.
  • Cross-session: Average improvement of 5.59%.
  • LOSO (Unseen-target): Average improvement of 6.69% and 4.83% respectively.
• Clinical Validation: The model was successfully transferred to a Major Depressive Disorder (MDD) identification task (healthy vs. MDD), achieving SOTA performance with improvements of up to 6.49%, validating its robustness in real-world heterogeneous settings.
• Ablation Studies: Confirmed that removing any component (RaL, Prototype, Dual-classifier stages) significantly degrades performance, proving the necessity of the full architecture.
• Noise Robustness: Under 40% label noise, PAA-M showed a performance drop of only ~3.13%, compared to ~5.61% for traditional methods.

5. Significance

• Theoretical Advancement: The paper addresses the theoretical gap in domain adaptation where global alignment fails due to class-conditional shifts. By tightening the upper bound of target risk through boundary-aware refinement, it offers a more robust theoretical foundation for cross-corpus learning.
• Practical Deployment: The ability to generalize across different datasets and even to clinical depression scenarios suggests that PAA can facilitate the real-world deployment of EEG-based affective computing systems, reducing the need for expensive, subject-specific recalibration.
• Handling Data Scarcity & Noise: The relation-driven approach provides a viable solution for scenarios where labeled data is scarce or noisy, a common issue in medical and psychological EEG studies.

In conclusion, the PAA framework represents a significant leap in cross-corpus EEG emotion recognition by moving beyond simple feature alignment to a sophisticated, boundary-aware, and relation-driven learning strategy.