🤖 machine learning

Class-Dependent Hybrid Data Augmentation for Multiclass Migraine Classification under Severe Class Imbalance

This paper presents a reproducibility-oriented re-evaluation of migraine classification that corrects methodological flaws and introduces a clinically motivated class aggregation combined with a class-dependent hybrid data augmentation strategy, achieving robust performance improvements across multiple classifiers under severe class imbalance.

Original authors: Elvin Somón, Miguel A. Gutiérrez-Naranjo

Published 2026-05-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Elvin Somón, Miguel A. Gutiérrez-Naranjo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Fixing a Broken Game

Imagine a group of scientists trying to build a computer program (an AI) that can look at a patient's medical chart and guess exactly which of seven different types of migraine they have.

The problem is, the data they have is messy. It's like trying to teach a child to recognize animals using a picture book where 200 pages show dogs, but only 14 pages show tigers. The computer gets really good at guessing "dog" but fails miserably at guessing "tiger."

Furthermore, previous studies claimed their computers were almost perfect (99% accurate). This paper argues that those studies were cheating, much like a student who peeked at the answer key before taking the test.

The Three Big Mistakes Found

The authors found three main reasons why previous results looked too good to be true:

The "Cheating" Test (Data Leakage): In past studies, the computer was allowed to "peek" at the test answers while it was studying. They mixed the training data and the test data together before splitting them up. When the authors fixed this and made sure the computer never saw the test data during training, the scores dropped significantly. The "perfect" scores were an illusion.
The Wrong Scorecard (Bad Metrics): Previous studies used "Accuracy" as their main score. If 90% of patients have Type A migraines, a computer that just guesses "Type A" for everyone gets 90% accuracy. But it's useless for the other types. The authors switched to a "Macro-F1" score, which treats every migraine type equally, like a teacher grading a test where every question is worth the same amount of points, regardless of how many students got it right.
The "One-Size-Fits-All" Fix (Uniform Augmentation): To fix the lack of data for rare migraines, scientists usually use "Data Augmentation"—a way of creating fake, but realistic, patient records to fill in the gaps. Previous studies used the same method to create fake data for every type of migraine. The authors found this is like trying to bake a cake and a soufflé using the exact same recipe; it works for one but ruins the other.

The New Solution: A Tailored Approach

The authors proposed a new, smarter way to handle this problem, which they call a "Class-Dependent Hybrid Framework." Here is how it works:

1. The "Smart Merging" (Label Aggregation)

The paper discovered that two specific types of migraines (Sporadic and Familial Hemiplegic) are so similar in their symptoms that the computer cannot tell them apart using the available data. It's like trying to distinguish between two twins who are wearing the exact same clothes and have the exact same voice.

The Fix: They merged these two confusing types into a single category called "Hemiplegic Migraine."
The Result: This was the biggest win. By removing the impossible task of telling the twins apart, the computer's overall score jumped up significantly. The paper notes that how you define the problem (the labels) matters more than the fancy math used to solve it.

2. The "Specialized Chef" (Class-Dependent Augmentation)

Instead of using one method to create fake data for everyone, they created a rule:

For the "Tiny" Classes (very few real patients): They use a simple, stable method (Gaussian Copula) that doesn't need much data to work. It's like using a simple sketch to fill in a blank space.
For the "Medium/Large" Classes: They use a complex, powerful method (CTGAN) that can learn intricate patterns. It's like using a high-definition 3D printer for the spaces that have enough reference material.
The Result: This tailored approach worked better than using just one method for everyone.

3. The "Fair Ratio" (Proportional Growth)

When you create fake data to balance the classes, you risk creating a situation where the "rare" classes are made almost entirely of fake data, while the "common" classes are 100% real. This creates a "Fidelity Asymmetry"—the computer is learning from a mix of real and fake for some, but only real for others.

The Fix: Instead of forcing all classes to have the exact same number of patients (Full Balance), they used "Proportional Growth." They multiplied the number of patients in every class by the same amount (e.g., doubled everyone).
The Result: This kept the ratio of "Real vs. Fake" data consistent across all groups, making the training environment fairer and more stable.

The Final Scorecard

After fixing the cheating, merging the confusing twins, and using a specialized chef for the data:

The "Honest" baseline (without any fancy tricks) was around 0.71 (on a scale where 1.0 is perfect).
The best previous studies claimed scores near 0.99, but the authors proved those were inflated.
With their new, honest, and tailored method, they achieved a score of 0.914.

The Main Takeaway

The paper concludes that in medical AI, how you set up the problem is more important than the complexity of the model.

Don't cheat with your test data.
Don't use a "one-size-fits-all" fix for data shortages.
Sometimes, the best way to improve an AI is to simplify the question it's trying to answer (like merging the two confusing migraine types) rather than making the AI smarter.

The authors emphasize that this framework is a template for better research, not a ready-to-use medical tool yet. It shows how to run these experiments correctly so that future tools can be trusted.

Technical Summary: Class-Dependent Hybrid Data Augmentation for Multiclass Migraine Classification under Severe Class Imbalance

1. Problem Definition

The paper addresses the challenge of multiclass migraine classification using clinical data, a task hindered by three primary factors:

Severe Class Imbalance: Clinical registries often contain rare but clinically relevant subtypes represented by very few samples, while majority classes dominate.
Methodological Flaws in Prior Work: Previous studies on this specific dataset (400 patients, 7 subtypes) reported inflated performance metrics (e.g., accuracies up to 99.7%) due to data leakage (applying augmentation before train/test splitting) and the use of majority-dominated metrics (accuracy) rather than class-balanced metrics.
Inappropriate Augmentation Strategies: Standard data augmentation (e.g., SMOTE) is typically applied uniformly across all classes. This ignores the heterogeneity of class sizes and feature distributions, often generating implausible samples in low-sample regimes or failing to capture complex dependencies in moderate-sized classes.

The study posits that without correcting these foundational issues, model reliability and clinical utility cannot be accurately assessed.

2. Methodology

2.1. Reproducibility-Oriented Re-evaluation

The authors first established a corrected baseline by re-evaluating prior studies. They enforced a strict protocol where:

Data Leakage was eliminated: All preprocessing, including augmentation, was applied exclusively within the training folds of a stratified 5-fold cross-validation.
Metrics were corrected: The primary evaluation metric was shifted from accuracy to macro-averaged F1-score, which assigns equal weight to all classes, ensuring minority subtypes are not obscured by majority predictions.
Baseline Result: These corrections reduced the previously inflated performance estimates, establishing a corrected macro-F1 baseline of 0.71 for the original seven-class configuration.

2.2. Clinically Motivated Label Aggregation

To address a structural performance ceiling observed in the seven-class setup, the authors proposed merging two specific subtypes: Sporadic Hemiplegic Migraine and Familial Hemiplegic Migraine.

Rationale: According to ICHD-3 §1.2.3, these are variants of the same syndrome with identical symptomatic criteria. The only distinguishing feature in the dataset was a binary family history variable (DPF), which did not provide sufficient discriminative structure for the remaining clinical features (18 of 22 features had mean differences < 0.3).
Outcome: This aggregation transformed the problem into a six-class classification task, removing a non-learnable decision boundary.

2.3. Class-Dependent Hybrid Augmentation Framework

The core contribution is a framework that adapts augmentation strategies based on per-class sample size ( $n_k$ ) rather than applying a uniform method.

Generator Selection: A threshold $\tau = 29$ $τ = 29$ partitions classes:
- Small Classes ( $n_k < 29$ ): Augmented using Gaussian Copula, which models parametric joint distributions and remains stable in low-sample regimes.
- Large Classes ( $n_k \ge 29$ ): Augmented using CTGAN, a deep generative model capable of capturing non-linear feature dependencies when sufficient data exists.
Growth Modes (Volume Control): The framework evaluates three modes to control the volume of synthetic data:
- Balance: Augments minority classes to match the majority class size.
- Proportional $\times 2$ and $\times 4$ : Multiplies all classes by a constant factor, preserving original class proportions.
Fidelity Asymmetry: The authors introduce the concept of "fidelity asymmetry," noting that full balancing creates a dataset where minority classes are dominated by synthetic data (low fidelity) while the majority class remains entirely real. Proportional augmentation mitigates this by maintaining a uniform fraction of real data across all classes.

2.4. Experimental Setup

Dataset: 400 patients, 22 clinical features, 7 subtypes (reduced to 6).
Models: Eight classifiers ranging from classical (SVM, KNN, RF, XGBoost) to deep tabular architectures (MLP, TabNet, FT-Transformer, GANDALF).
Protocol: Stratified 5-fold cross-validation with hyperparameter optimization performed separately for augmented and non-augmented regimes to ensure fair comparison.

3. Key Results

3.1. Impact of Methodological Corrections

Correcting for data leakage and metric bias revealed that prior high-performance claims were artifacts. The corrected baseline for the seven-class problem stood at 0.71 macro-F1.

3.2. Impact of Label Aggregation

Aggregating the hemiplegic subtypes into a single class yielded the most significant performance gain. The FT-Transformer without augmentation achieved a macro-F1 of 0.896 ± 0.038 in the six-class setting, compared to 0.845 ± 0.029 in the seven-class setting. This demonstrated that problem formulation (label design) was a more critical determinant of performance than augmentation volume.

3.3. Performance of the Hybrid Framework

The proposed class-dependent hybrid framework consistently outperformed individual augmentation strategies:

Average Performance: The hybrid framework achieved an average macro-F1 of 0.862 across eight classifiers, compared to 0.836 for Gaussian Copula alone, 0.815 for CTGAN alone, and 0.801 for the no-augmentation baseline.
Peak Performance: The best result was achieved by FT-Transformer with the hybrid framework using proportional $\times 2$ growth, reaching a macro-F1 of 0.914 ± 0.047.
SMOTE Failure: SMOTE-based methods consistently degraded performance in this severely imbalanced multiclass setting, likely due to introducing label noise in overlapping feature spaces.

4. Significance and Claims

The paper claims that its primary contribution is not merely a new model, but a reproducible and clinically grounded approach to handling low-data, imbalanced medical classification.

Problem Formulation is Paramount: The study emphasizes that "problem formulation is the most consequential modelling decision." The largest single gain (+0.051) came from clinically motivated label redesign, not from augmentation.
Augmentation as a Design Choice: The authors argue that data augmentation should not be a uniform preprocessing step but a context-sensitive design choice. They demonstrate that proportional augmentation (preserving class ratios) can be superior to full balancing in low-data regimes by mitigating "fidelity asymmetry."
Reproducibility: By correcting leakage and using appropriate metrics, the study provides a "honest starting point" (0.71) for future research, contrasting with the inflated estimates found in prior literature.
Limitations: The authors modestly note that the framework relies on a single dataset from one clinical center, lacks external validation, and that the specific threshold $\tau$ and growth modes may not transfer directly to other distributions without sensitivity analysis. They explicitly state the framework is an evaluation template rather than a deployable diagnostic tool without further validation.

In conclusion, the paper asserts that improving medical machine learning in low-data settings depends less on increasingly complex models and more on rigorous problem formulation, data fidelity, and evaluation design.