Exploring sex-related Biases in Deep Learning Models for Motor Imagery Brain-Computer Interfaces

This study investigates sex-related biases in deep learning models for motor imagery brain-computer interfaces and finds that while initial performance disparities favor females, these differences are primarily driven by inherent EEG pattern discriminability rather than model bias, with deep learning ultimately improving performance across all groups, particularly for those with less distinguishable signals.

The Gist

Imagine you are building a mind-reading machine. This machine is designed to help people control computers or robotic arms just by thinking about moving their hands (a process called "Motor Imagery"). To do this, the machine listens to the electrical whispers of the brain using a cap of sensors (EEG).

Recently, scientists started using Deep Learning (DL)—a type of super-smart AI—to make these mind-reading machines better. But there's a big worry in the tech world: AI can be biased. Just like a human might accidentally favor one type of person over another, AI might learn to work better for men than for women, or for young people than for old ones.

This paper asks a very specific question: "Does our super-smart mind-reading AI treat men and women unfairly?"

Here is the story of what they found, explained simply:

1. The Setup: Two Groups of Testers

The researchers took data from two different groups of people (about 100 people total) who tried to imagine moving their left or right hand. They fed this data into two types of "decoders":

• The Old School Decoder (CSP+LDA): A traditional, simpler math method.
• The Super AI Decoder (Deep Learning/EEGNet): A complex, modern neural network that learns on its own.

They made sure to balance the training data perfectly, so the AI saw an equal number of men and women while learning.

2. The First Glance: "Wait, Women Are Winning!"

When they first looked at the results, they saw something interesting. The AI worked better for women than for men.

• The Fear: "Oh no! The AI is biased against men!"
• The Reality: It looked like the AI was unfair, but the researchers suspected the problem wasn't the AI itself. They thought the signal (the brain data) might be the culprit.

3. The Detective Work: The "Clear Voice" Analogy

To solve the mystery, the researchers invented a way to measure how "clear" a person's brain signal was. Let's use an analogy:

Imagine you are trying to hear a friend's voice in a noisy room.

• Person A (High Clarity): Their voice is loud, clear, and easy to hear.
• Person B (Low Clarity): Their voice is quiet, muffled, and hard to hear.

In this study, women happened to have "clearer voices" (more distinct brain patterns) in these specific datasets. Men, on average, had "muffled voices" (less distinct patterns).

When you have a clear voice, any listener (even a simple one) can understand you. When you have a muffled voice, you need a super-listener (like the Deep Learning AI) to make sense of the noise.

4. The Big Reveal: It's Not the AI, It's the Signal

The researchers ran a deep dive and found the truth:

• The AI didn't create the bias. The AI actually helped everyone get better at the task.
• The AI helped the "muffled voices" the most. The Deep Learning model was so good at filtering out noise that it helped the men (who had harder-to-read signals) catch up significantly.
• The "Female Advantage" was an illusion. The reason women scored higher wasn't because the AI liked them; it's because, in these specific groups of people, women naturally produced clearer brain signals for this specific task.

If you look at the data without separating men and women, the AI is a hero for everyone. If you look at it by gender, it looks like it favors women, but that's only because the women started with a "head start" in signal clarity.

5. The Takeaway: Why This Matters

This study is like a safety check for the future of brain-computer interfaces.

• Don't blame the tool: We shouldn't assume the AI is racist or sexist just because the results look different between groups. Sometimes, the difference comes from the people themselves, not the machine.
• AI is a great equalizer: Deep Learning is actually good at helping people who struggle to control their brain signals. It helps the "muffled voices" become understandable.
• We need better data: The researchers admit that in their specific test groups, women had clearer signals. But in the real world, we need to make sure we test on everyone (different ages, backgrounds, and skill levels) to make sure the AI works fairly for all of us.

In a nutshell: The "Mind-Reading AI" isn't biased against men. It's actually a powerful tool that helps everyone, especially those whose brain signals are harder to read. The apparent difference in scores was just a reflection of how clear the brain signals were to begin with, not a flaw in the machine's heart.

Technical Summary

1. Problem Statement

Motor Imagery (MI) Brain-Computer Interfaces (BCIs) are critical for neurorehabilitation, yet they face significant challenges regarding algorithmic fairness. Deep Learning (DL) models are increasingly used to decode MI signals from Electroencephalography (EEG) data. However, DL models are known to be susceptible to biases based on protected attributes (e.g., biological sex, age), potentially leading to disparate performance across subpopulations.

While previous studies have demonstrated that sex and age can be decoded from resting-state EEG signals with high accuracy, it remains unclear whether MI-BCI systems inherently amplify these biases. A critical gap exists in understanding whether performance disparities observed between male and female users are caused by the DL model itself or are intrinsic to the data (e.g., differences in the user's ability to generate discriminable brain patterns).

2. Methodology

Datasets and Preprocessing

The study utilized two public MI-BCI datasets:

• Lee 2019: 54 subjects (23 females), 62 channels, 1000 Hz sampling.
• Cho 2017: 52 participants (19 females), 64 channels, 512 Hz sampling.
• Preprocessing: Signals were band-pass filtered (0.5–40 Hz), notch-filtered (50 Hz), re-referenced to Fz, downsampled to 128 Hz, and cropped to the 0.5–2.5 s window post-cue. Only channels C3, C4, and Cz were used.

Experimental Design

• Evaluation Scheme: A Leave-One-Subject-Out (LOSO) cross-validation strategy was employed.
• Balancing: To isolate the effect of the model from data imbalance, the training and validation sets were explicitly balanced by sex (equal numbers of males and females) for every iteration.
• Replicates: For each test subject, 20 replicates were generated, each with 5 different random initialization seeds, resulting in 100 trained models per test subject to account for variance.
• Models Tested:
  • Deep Learning: EEGNet (primary), plus Deep4Net, ShallowFBCSPNet, HybridNet, FBCNet, CTNet, and ATCNet.
  • Baseline: Common Spatial Patterns (CSP) + Linear Discriminant Analysis (LDA).

Metrics and Statistical Analysis

• Performance Metrics: Accuracy and Area Under the ROC Curve (AUC).
• Class Distinctiveness: A novel metric calculated directly from raw EEG signals (independent of the model) using Riemannian geometry. It quantifies the distance between the spatial covariance matrices of left-hand vs. right-hand MI trials. This serves as a proxy for the intrinsic difficulty of the classification task for a specific user.
• Statistical Modeling: Due to the non-independence of data points in LOSO schemes, the authors employed Linear Mixed-Effects Models. These models treated "subject" as a random effect and analyzed fixed effects for Sex, Age, and Class Distinctiveness to determine their contribution to performance metrics. Partial correlations were also used to isolate the relationship between performance and distinctiveness while controlling for sex.

3. Key Results

Performance Disparities

• DL vs. Traditional: DL models (EEGNet) significantly outperformed the traditional CSP+LDA method globally and for both sexes.
• Sex-Based Gaps: A consistent trend showed higher accuracy for females compared to males in both datasets.
  • CSP+LDA: Showed a small performance gap (1–4%) but poor overall accuracy.
  • DL Models: Showed a larger performance gap (5–7%) but achieved significantly higher overall accuracy for both groups.
• Crucial Finding: The performance gap was not driven by the model amplifying bias against males. Instead, the gap correlated strongly with Class Distinctiveness.

The Role of Class Distinctiveness

• Correlation: There was a strong positive correlation between Class Distinctiveness and model accuracy.
• Mixed-Effects Analysis:
  • Class Distinctiveness was a statistically significant predictor of performance across all models.
  • Sex was generally not a significant predictor in DL models (EEGNet) once distinctiveness was controlled for. In CSP+LDA, sex showed some significance, but this was likely due to the model's inability to handle low-distinctiveness signals effectively.
  • Age remained a significant predictor in several cases.
• Interpretation: The observed "female advantage" in these specific datasets was attributed to the fact that female participants, on average, generated more discriminable (higher distinctiveness) EEG patterns. The DL model did not introduce bias; rather, it successfully leveraged the higher quality of signals from these specific users.

Robustness Checks

• Sex Classification: When the EEGNet was trained to classify the sex of the subject using MI data, it performed near chance level. This suggests that explicit sex features are not easily exploitable in active MI tasks, unlike in passive resting-state tasks.
• Frequency Ablation: Removing specific frequency bands did not destabilize performance, indicating the model relies on robust MI features rather than spurious correlations.

4. Key Contributions

1. Decoupling Data Bias from Model Bias: The study provides robust evidence that performance disparities in MI-BCI are primarily driven by intrinsic user capability (signal discriminability) rather than demographic attributes or the DL architecture itself.
2. Methodological Rigor: The introduction of Class Distinctiveness as a model-independent metric allows researchers to separate user skill from algorithmic performance.
3. Advanced Statistical Framework: The paper demonstrates the necessity of using Mixed-Effects Models and Partial Correlations in BCI research to handle the non-independence of LOSO data, avoiding erroneous conclusions from standard t-tests.
4. Validation Across Architectures: The findings were validated across 7 different DL architectures and 38,400 individual model instances, confirming that the results are generalizable to the DL approach in BCI, not just a specific network.

5. Significance and Implications

• Ethical Deployment: The findings are reassuring for the deployment of DL-based BCIs in clinical settings (e.g., stroke rehabilitation). They suggest that DL models do not inherently discriminate against specific sexes; rather, they improve performance for all users, with the most significant gains seen in users with lower signal discriminability.
• Fairness Assessment: The paper argues that standard performance stratification (e.g., "Accuracy by Sex") is insufficient and potentially misleading. Future fairness assessments must account for confounding variables like signal quality and user skill.
• Future Directions: The authors call for more diverse datasets with richer metadata (socioeconomic status, ethnicity, experimenter details) and the development of standardized protocols to better understand inter-subject variability. They emphasize that ensuring equitable access requires analyzing model behavior beyond standard metrics to ensure that "fairness" is not compromised by unmeasured confounders.

In conclusion, this work shifts the narrative from "DL models are biased" to "DL models are powerful tools that reveal intrinsic data heterogeneity," urging the community to adopt more sophisticated analytical frameworks to ensure the equitable development of BCI technologies.