Graph Neural Networks in EEG-based Emotion Recognition: A Survey

This survey provides a comprehensive review and unified framework for Graph Neural Networks in EEG-based emotion recognition, categorizing existing methods by graph construction stages to offer guidance on their unique physiological foundations while outlining open challenges and future directions.

The Gist

Imagine your brain is a bustling, high-tech city. Every time you feel an emotion—like joy, anger, or sadness—it's not just one part of the city reacting; it's a complex conversation happening between different neighborhoods (the brain regions).

EEG (Electroencephalogram) is like a drone flying over this city, recording the electrical chatter of the streets. Emotion Recognition is the art of listening to that chatter to figure out what the city is feeling.

For a long time, computers tried to listen to this chatter by treating each street (electrode) as an isolated voice. But emotions are a group effort. A new type of AI called Graph Neural Networks (GNNs) has emerged, which is much better at understanding how these neighborhoods talk to each other.

This paper is a survey (a big map) of how researchers are currently using these "City-Listening" GNNs to understand emotions. The authors realized that while everyone is building these maps, they are doing it in very different ways. So, they created a Unified Framework—a standard blueprint—to organize all these methods into three simple construction stages.

Here is the blueprint, explained with everyday analogies:

Stage 1: The Node-Level (Choosing the "Characters")

Before you can map a conversation, you need to decide who is speaking. In the brain, these "speakers" are the electrodes on your scalp.

• The Question: What data do we feed the computer for each speaker?
• The Options:
  • Univariate (The Specialist): We pick just one type of data for each speaker. Maybe we only listen to the speed of their speech (Time Domain) or the pitch of their voice (Frequency Domain). This is the most common approach.
  • Hybrid (The Generalist): We give the computer a mix of everything—speed, pitch, and volume all at once. It's like giving the detective a full dossier instead of just one clue. It's powerful but can be messy if there isn't enough data to sort it all out.

Stage 2: The Edge-Level (Drawing the "Connections")

Now that we have our speakers, we need to draw lines between them to show who is talking to whom. These lines are the Edges.

• The Question: How do we decide who is connected to whom?
• The Options:
  • Model-Independent (The Mapmaker): We draw lines based on fixed rules, like a physical map. "If two speakers are standing next to each other, they must be connected." Or, "If their voices sound similar, they are connected." These rules don't change; they are based on physics or biology.
  • Model-Dependent (The Detective): We let the computer learn the connections. As it studies the data, it says, "Hey, Speaker A and Speaker C seem to be conspiring together right now, even though they are far apart." The computer draws the lines itself based on what it learns, making the map smarter but more complex.

Stage 3: The Graph-Level (Building the "City Structure")

Finally, we assemble the speakers and connections into a full structure to understand the whole city's mood.

• The Question: What kind of city layout are we building?
• The Options:
  • Multi-Graph (The Multi-Layered City): Instead of one map, we build several at once. One map shows how neighborhoods talk horizontally (left to right), another shows vertical talk (front to back), and another shows how they talk over time. We combine all these views for a 3D understanding.
  • Hierarchical Graph (The Neighborhood Hierarchy): We group speakers into neighborhoods, then group neighborhoods into districts. This helps the computer understand both the local gossip (within a neighborhood) and the city-wide trends (between districts).
  • Time Series Graph (The Movie): Emotions change over time. This approach treats the city as a movie, not a photo. It looks at how the connections change from second to second.
  • Sparse Graph (The Filter): In a real city, not everyone talks to everyone. This approach tries to cut out the noise, keeping only the most important connections to avoid getting overwhelmed by irrelevant chatter.

What's Next? (The Future Directions)

The authors point out that while we are doing well, there are still some blind spots:

1. The "Time Travel" Problem: Current maps often miss how a conversation in one neighborhood now affects a different neighborhood later. We need a "fully connected time graph" to catch these delayed reactions.
2. Graph Condensation (The Compression): Our current maps are huge and full of redundant details. We need a way to compress the city into a tiny, efficient model that keeps only the essential emotional information, making it faster to run.
3. Heterogeneous Graphs (The Multi-Sensor City): Emotions aren't just in the brain; they affect your heart and sweat glands too. Future maps should combine brain data with heart data to get the full picture of human emotion.
4. Dynamic Graphs (The Living City): Most maps are static snapshots. We need maps that breathe and change shape in real-time as the emotion evolves, capturing the fluid nature of human feelings.

The Bottom Line

This paper is a guidebook for engineers and scientists. It says: "Stop reinventing the wheel. Here is a standard way to build these emotion-detecting AI systems, broken down into three clear steps. If you follow this blueprint, you can build better, faster, and more accurate tools to understand the human heart and mind."

Technical Summary

Here is a detailed technical summary of the survey paper "Graph Neural Networks in EEG-based Emotion Recognition: A Survey" by Liu et al.

1. Problem Statement

Emotion recognition is critical for mental health diagnosis and Brain-Computer Interfaces (BCIs). While various modalities exist (e.g., facial expressions, voice), Electroencephalogram (EEG) signals are considered the most unique and objective modality because they directly reflect neural activity and genuine emotional states.

The core challenge in EEG-based emotion recognition is modeling the complex dependencies and interactions between different brain regions. Traditional methods often fail to capture these non-linear, spatial-temporal relationships effectively. While Graph Neural Networks (GNNs) have emerged as a powerful tool for modeling such graph-structured data, their application in this specific domain faces unique challenges:

• Physiological Specificity: Brain region dependencies in emotional EEG have distinct physiological bases that differ from general time-series data.
• Lack of Standardization: There is no unified framework or comprehensive review guiding how to construct GNNs specifically for this task. Existing methods vary widely in how they define nodes, edges, and graph structures, making comparison and replication difficult.

2. Methodology: The Unified Framework

The authors propose a unified framework to categorize and analyze existing GNN approaches. Instead of grouping by specific model architectures, they decompose the GNN construction process into three distinct stages. This taxonomy provides a systematic view of how different methods handle the data:

Stage 1: Node-level Stage (Feature Selection)

This stage defines what constitutes a "node" in the graph (typically corresponding to EEG channels).

• Univariate Nodes: Utilize a single feature type.
  • Temporal Domain: Most common (e.g., Differential Entropy - DE).
  • Frequency Domain: Uses Power Spectral Density (PSD).
  • Raw Signal: Uses raw EEG data (maximizes information but increases dimensionality and noise).
• Hybrid Nodes: Fuse multiple features (e.g., combining Temporal and Frequency domains or aggregating various statistical features) to capture complementary dependencies, though this risks overfitting with small datasets.

Stage 2: Edge-level Stage (Connectivity Calculation)

This stage defines how relationships (edges) between nodes are computed.

• Model-independent Edges: Calculated based on fixed rules or prior knowledge without learnable parameters.
  • Prior: Based on physical distances or standard brain atlases (e.g., 10-20 system).
  • Distance: Euclidean or Cosine distance between signal vectors.
  • Similarity: Pearson correlation, Mutual Information, or Phase Locking Value.
• Model-dependent Edges: Learned dynamically during training via model parameters.
  • Parametric: Directly learned adjacency matrices.
  • Weighted/Bias: Node relationships modulated by learnable weights or bias terms.
  • Subspace: Uses projection matrices (similar to Attention mechanisms) to capture complex interactions.

Stage 3: Graph-level Stage (Structure Construction)

This stage defines the overall topology of the graph used for reasoning.

• Multi-graph: Uses multiple streams to capture different dependency types simultaneously (e.g., Horizontal vs. Vertical spatial dependencies, or Temporal vs. Frequency dependencies).
• Hierarchical Graph: Groups channels into brain regions to model both intra-region (local) and inter-region (global) dependencies. Can be Dynamic (learned grouping) or Predetermined (based on anatomical knowledge).
• Time Series Graph: Models temporal evolution by treating time slices as nodes or using temporal encoders (LSTM, Transformer) to link spatial graphs across time.
• Sparse Graph: Filters out weak connections to focus on essential brain regions, reducing noise and computational cost (via thresholding or sparse weights).

3. Key Contributions

1. First Comprehensive Survey: This is the first and only survey dedicated specifically to GNNs in EEG-based emotion recognition, filling a gap in the literature.
2. Novel Taxonomy: The authors introduce a three-stage framework (Node, Edge, Graph) that unifies disparate methods. This allows researchers to understand the specific design choices of any given method and provides a clear guideline for constructing new GNNs.
3. Identification of Open Challenges: The paper critically analyzes current limitations and proposes four specific future directions:
   • Temporal Fully-Connected Graphs: Addressing the lack of cross-channel temporal dependencies (e.g., delayed responses between different brain regions over time).
   • Graph Condensation: Techniques to compress large, noisy graphs into compact representations to improve efficiency and reduce redundancy.
   • Heterogeneous Graphs: Integrating multiple physiological signals (e.g., EEG + Heart Rate) to model the broader physiological regulation of emotion.
   • Dynamic Graphs: Moving beyond static or slice-based graphs to structures where edges and nodes evolve dynamically over time to capture true temporal dependencies.

4. Results and Analysis

• Performance Trends: The survey notes that while Temporal domain nodes (specifically Differential Entropy) are currently the most popular and generally perform slightly better than Frequency domain nodes, Hybrid nodes offer a more comprehensive view if data volume is sufficient.
• Edge Construction: Model-dependent edges generally offer higher accuracy by adapting to specific data distributions but come with higher computational costs and complexity compared to Model-independent edges.
• Graph Structures: Multi-graph and Hierarchical structures are shown to be effective in capturing the multi-scale nature of brain activity (local vs. global), while Sparse graphs are crucial for handling high-resolution EEG data by filtering noise.

5. Significance

This survey is significant for the Brain-Computer Interface (BCI) and neuroscience communities because:

• Standardization: It provides a standardized language and framework for discussing GNNs in emotion recognition, moving the field away from ad-hoc model designs.
• Guidance: It offers a practical "how-to" guide for researchers to design new models by selecting appropriate components from the three stages.
• Future Roadmap: By highlighting specific technical gaps (like the need for cross-channel temporal modeling and heterogeneous signal integration), it sets a clear research agenda for the next generation of emotion recognition systems, potentially leading to more robust and clinically viable BCI applications.