Involuntary facial muscle activity during imagined vocalisation contaminates EEG and enables emotion decoding

This study demonstrates that above-chance decoding of emotion from single-trial EEG during imagined vocalisation is largely driven by involuntary facial muscle activity, particularly for happiness, rather than purely neural speech signals.

The Gist

The Big Idea: The "Ghost in the Machine"

Imagine you are trying to listen to a whisper from inside a locked room (your brain) using a microphone placed outside the door (the EEG headset). The goal of this study was to see if we could understand what people were thinking about saying, even when they didn't actually speak out loud.

The researchers wanted to know: Can we tell if someone is imagining a happy, sad, or angry voice just by looking at their brainwaves?

They found that yes, we can decode these emotions with surprising accuracy. However, the paper reveals a twist: the "brain signals" we thought we were hearing were actually mostly the sound of tiny, invisible facial muscles twitching.

The Experiment: The "Silent Movie" Test

The researchers ran two tests with volunteers:

1. The Loud Test: People actually spoke emotional words (like shouting "Hooray!" or "No!").
2. The Silent Test: People imagined saying those same words without moving their mouths or making a sound.

They recorded the brain activity (EEG) of 21 people during the silent test. They also put tiny sensors on the faces of 5 of those people to catch any tiny muscle movements (sEMG).

The Surprise: The "Railroad Tracks"

When the researchers looked at the data, they found something strange. The brainwaves that helped them guess the emotion (especially happiness) looked exactly like electrical noise caused by muscles.

They called this pattern the "Railroad Cross-Tie Pattern."

• The Analogy: Imagine a quiet train track. Suddenly, you see a series of fast, sharp spikes in the signal, looking like the wooden ties on a railroad track.
• The Reality: These spikes weren't coming from the brain's "thinking" center. They were coming from the face. Even though the participants were told not to move, their brains were so excited about imagining "happiness" that their facial muscles (specifically the ones used for smiling) twitched involuntarily. It's like your foot tapping when you hear a great song, even if you are trying to sit still.

The Proof: The "Double-Check"

To prove this, the researchers did a clever comparison:

1. They tried to guess the emotion using only the brain sensors.
2. They tried to guess using only the face sensors.
3. They tried using both together.

The Result: Adding the brain sensors didn't make the guess any better than just using the face sensors.

• The Metaphor: It's like trying to guess the weather by looking at a thermometer (the brain) and a rain gauge (the face). If the thermometer just copies the rain gauge, you don't need the thermometer. The "brain signal" was just echoing the "face signal."

Why This Matters: The "Leaky Bucket"

This study is a huge wake-up call for the field of Brain-Computer Interfaces (BCIs).

• The Old Belief: Scientists thought that when you imagine speaking, your brain lights up in specific ways that tell us what you are feeling. They thought the high-frequency "buzz" in the brain (High-Gamma band) was pure thought.
• The New Reality: That "buzz" was likely just the electrical noise of your face muscles twitching. The brain wasn't necessarily "screaming" the emotion; the face was "whispering" it through tiny, involuntary twitches that the brain sensors picked up.

The Takeaway

1. We can decode emotions: We can tell if someone is imagining being happy or sad.
2. But it's not "pure" brain power: The success is largely because our faces betray us. Even when we try to be still, our muscles react to our feelings.
3. Future Tech: If we want to build better devices for people to control computers with their minds, we need to be careful. We might need to record the face and the brain separately to make sure we aren't just reading muscle twitches instead of thoughts.

In short: The researchers thought they were reading the mind, but they were actually reading the "micro-expressions" of the face that the mind couldn't quite hide.

Technical Summary

1. Problem Statement

Brain-Computer Interfaces (BCIs) aim to decode human intent, including speech and emotion, directly from brain activity. While significant progress has been made in decoding semantic content (words, syllables) from imagined speech using Electroencephalography (EEG), emotion decoding remains largely unexplored in the context of imagined speech.

A critical challenge in EEG-based BCI is physiological contamination, specifically Electromyography (EMG) artifacts from facial and neck muscles. EMG signals overlap spectrally with neural EEG signals (particularly in the high-frequency gamma bands) and spread widely across the scalp. Previous studies often ignored or inadequately filtered these artifacts, potentially leading to the misattribution of muscle activity as neural correlates of emotion. The authors hypothesize that "imagined" speech may not be free of EMG contamination due to involuntary facial muscle micro-movements, which could be the primary driver of successful emotion decoding rather than cortical neural activity.

2. Methodology

Experimental Design

The study involved two experiments with healthy participants:

• Experiment 1 (Overt Vocalisation, $n=14$): Participants listened to emotional vocalizations (Anger, Happiness, Neutral, Pleasure, Sadness) and then overtly produced the vowel /ah/ with the corresponding emotion.
• Experiment 2 (Imagined Vocalisation, $n=21$): Participants imagined producing the same emotional vocalizations without moving their facial or articulatory muscles.
  • Sub-group: Five participants in Experiment 2 were simultaneously recorded with facial surface EMG (sEMG) in addition to 64-channel EEG.

Data Acquisition & Preprocessing

• EEG: 64-channel recording (BioSemi ActiveTwo) at 2048 Hz, re-referenced to average, and filtered (1–100 Hz).
• sEMG: Four bipolar channels placed on the left medial frontalis, temporalis, orbicularis oculi, and zygomatic major regions.
• Preprocessing: Signals were band-pass filtered, notch-filtered (50/100 Hz), and epoched. Eye-blink artifacts were removed using FastICA.
• Feature Extraction: Time-frequency features were extracted using Hilbert transforms to compute instantaneous power in six bands: Delta, Theta, Alpha, Beta, Low-Gamma (30–50 Hz), and High-Gamma (50–100 Hz).

Classification & Analysis

• Classifier: Logistic Regression with 25-fold cross-validation (within-participant).
• Validation: Chance-level accuracy was established via label shuffling (100 repeats).
• EMG Contamination Analysis:
  1. Correlation: Computed Spearman's $\rho$ between instantaneous high-gamma power of EEG and sEMG channels.
  2. Feature Comparison: Compared classification accuracy using EEG-only, sEMG-only, and Combined (EEG+sEMG) features.
  3. Railroad Cross-Tie Pattern: Systematically detected a specific EMG artifact pattern (fast, high-amplitude "spikes" at 8–18 Hz in the 50–100 Hz band) known as the "railroad cross-tie pattern" and analyzed its prevalence across emotion categories.

3. Key Results

Decoding Accuracy

• Overt Vocalisation: Emotion was decoded with 78.1% accuracy (chance = 20%).
• Imagined Vocalisation: Emotion was decoded with 36.4% accuracy (significantly above chance).
• Frequency Importance: Accuracy increased with frequency. The High-Gamma band (50–100 Hz) yielded the highest performance for both overt and imagined tasks.
• Spatial Importance: Lateral EEG channels (specifically T7, T8, and proximal temporal regions) were the most critical for decoding, particularly for Happiness.

Evidence of EMG Contamination

1. EEG-sEMG Correlation: Strong instantaneous correlations were found between sEMG (zygomatic major and orbicularis oculi) and lateral EEG channels (T7/T8) during imagined vocalisation. This spatial pattern matched the channels identified as most important for emotion decoding.
2. Feature Redundancy: When combining EEG and sEMG features, classification accuracy did not improve over using sEMG features alone. In the high-gamma band, sEMG-only accuracy (42.8%) was significantly higher than EEG-only (34.5%), suggesting EEG added no unique neural information for emotion decoding in this context.
3. Railroad Cross-Tie Pattern:
   • This specific EMG artifact (sharp spikes in the 50–100 Hz range) was detected significantly more often during Happiness trials compared to other emotions.
   • The number of spikes per trial was highest for happiness (mean 15.56) vs. anger (8.33), neutral (8.01), etc.
   • This pattern was prominent on lateral channels (T7, FT8, FC6, C6, T8), directly linking the "neural" decoding signal to involuntary muscle activity.

4. Key Contributions

• Demonstration of Involuntary Activity: Proved that "imagined" speech is not free of facial muscle activity; participants involuntarily activated facial muscles (specifically zygomatic major and orbicularis oculi) associated with the intended emotion (especially happiness).
• Re-evaluation of High-Gamma Decoding: Provided strong evidence that high-gamma (50–100 Hz) and lateral channel features, often cited as neural signatures of emotion in literature (e.g., SEED, DEAP datasets), are likely dominated by EMG contamination rather than cortical neural correlates.
• Methodological Framework: Introduced a rigorous protocol for detecting the "railroad cross-tie" EMG pattern and correlating it with decoding performance to distinguish artifacts from neural signals.
• Practical Implication: Suggested that for BCI applications, facial sEMG should be recorded alongside EEG to either verify the absence of artifacts or utilized as a primary signal source, as it may offer a better signal-to-noise ratio for emotion decoding than residual EEG.

5. Significance and Implications

• Scientific Caution: The study challenges the interpretation of existing emotion decoding literature. Many high-accuracy studies using high-frequency EEG features may have inadvertently decoded muscle activity rather than brain states.
• BCI Design: For users with facial paralysis (a target demographic for BCIs), the assumption that imagined emotion elicits detectable EMG may not hold, potentially limiting the efficacy of EMG-based decoding strategies for this group.
• Future Directions: The authors advocate for simultaneous sEMG recording in all emotion decoding studies. They propose that future BCIs might benefit from treating facial sEMG as a complementary or primary feature rather than an artifact to be filtered out, especially for affect-aware systems.

Conclusion: The ability to decode emotion from imagined vocalisation using EEG is largely driven by involuntary facial muscle activity (specifically the "railroad cross-tie" pattern in the high-gamma band) rather than direct neural correlates of emotion. This finding necessitates a paradigm shift in how EEG-based emotion decoding is analyzed and interpreted.