Do Synthetic Brain MRIs Reliably Improve Tumour Classification? A StyleGAN2-ADA Class-Plane Augmentation Study on BRISC 2025

This study demonstrates that while StyleGAN2-ADA-generated synthetic brain MRIs do not universally improve tumor classification across all model architectures, they provide a statistically significant accuracy boost for MobileViTV2 when using feature-space filtering at a 1:1 real-to-synthetic ratio, highlighting that augmentation utility depends on the specific classifier and data configuration rather than visual fidelity alone.

The Gist

The Big Question: Can Fake Medical Scans Help Doctors (or Computers) Learn?

Imagine you are trying to teach a student how to identify different types of tumors in brain scans (MRIs). The problem is, you only have a small library of real textbooks (real MRI scans). Because there are so few, the student might memorize the specific pictures in the book rather than learning the actual rules of what a tumor looks like.

To fix this, researchers asked: "What if we use an AI artist to draw fake brain scans that look real, and add them to the student's library? Will this help the student learn better?"

This study didn't just ask if the fake drawings looked good; they asked if they actually helped the student pass the final test.

The Setup: The "Class-Plane" Kitchen

The researchers didn't just make one big pile of fake scans. They realized that brain scans look very different depending on two things:

1. The Diagnosis: Is it a Glioma, a Meningioma, a Pituitary tumor, or no tumor at all?
2. The Angle: Is the scan taken from the top (Axial), the front (Coronal), or the side (Sagittal)?

So, instead of one big AI, they built 12 tiny, specialized AI artists. Each one was assigned a specific job, like "Draw only Meningioma tumors seen from the side." This is like having a chef who only knows how to make one specific type of dish perfectly, rather than a chef trying to cook everything at once.

They used a powerful tool called StyleGAN2-ADA to create these images. They generated thousands of fake scans, but they were careful. They didn't just dump them all in; they used a "quality filter" (a mathematical check) to make sure the fake scans looked like they belonged in the same family as the real ones.

The Test: Three Different "Students"

To see if the fake scans helped, they tested three different types of computer "students" (classifiers) with the same final exam (a set of real brain scans the AI had never seen before):

1. The "Old School" Student (Random Forest): This student looks at the pictures through a fixed pair of glasses (pre-trained features) and makes decisions based on simple rules. It's like a student who memorizes a checklist.
2. The "Hard-Worker" Student (Compact CNN): This student learns from scratch, looking at the pixels and figuring out the patterns on its own. It's like a student who studies the textbook cover-to-cover.
3. The "Smart" Student (MobileViTV2): This is a high-tech student that combines different learning styles (like a hybrid of a human and a super-computer). It's the most advanced learner in the group.

They tested these students under different conditions:

• Real Only: Studying only the real textbooks.
• Fake Only (Mixed In): Studying a mix of real and fake books (at different ratios, like 1 fake for every 1 real, or 2 fakes for every 1 real).
• Filtered: Using only the "best" fake books that passed the quality check.

The Results: It Depends on Who You Ask

The answer to "Do fake scans help?" wasn't a simple "Yes" or "No." It depended entirely on which student was learning.

1. The "Old School" Student (Random Forest): No Help

• Result: Adding fake scans didn't help this student at all. In fact, it sometimes made them slightly worse.
• Analogy: Imagine giving a student who relies on a strict checklist a bunch of fake examples that are almost right but have tiny, weird errors. The student gets confused by the errors and starts second-guessing their checklist. The fake data just added noise, not clarity.

2. The "Hard-Worker" Student (Compact CNN): A Little Help, But Not Proven

• Result: This student got slightly better scores when using fake scans, but the improvement was so small that it could have been a lucky fluke.
• Analogy: This student studied harder and learned a bit faster, but when it came time for the final test, the extra practice didn't guarantee a higher grade.

3. The "Smart" Student (MobileViTV2): Yes, It Helped!

• Result: This student showed a clear, statistically significant improvement. When they used a mix of real scans and filtered fake scans (1 fake for every 1 real), their accuracy went up by about 1%.
• Analogy: This student was smart enough to ignore the tiny errors in the fake drawings and use the extra variety to understand the "big picture" better. The fake scans acted like extra practice drills that filled in the gaps in their knowledge.

The Hidden Bonus: Learning Faster

Even when the final test scores didn't jump up dramatically, the fake scans helped the students learn faster.

• The Efficiency Gain: The students who used fake scans reached their "best possible performance" much sooner.
  • The "Hard-Worker" student needed 42–64% fewer passes through the real textbook to find their best learning spot.
  • The "Smart" student needed 50–67% fewer passes through the real data.
• Analogy: Imagine you are trying to find the best route through a city. With only a few real maps, you have to drive the same streets over and over to learn them. If you have a bunch of good fake maps to practice on, you can figure out the general layout much faster, so you spend less time driving the real streets before you are ready for the final race.

The "Blind Test": Can a Robot Tell the Difference?

The researchers also asked a very advanced AI (GPT-5.5) to look at the real and fake scans and guess which was which.

• Result: The AI guessed correctly only 57.7% of the time. Since a random guess would be 50%, this means the fake scans were very hard to distinguish from the real ones.
• Analogy: The fake drawings were so good that even a super-smart robot couldn't easily tell them apart from the real thing. This proves the AI artists did a good job of making the images look realistic.

The Bottom Line

The paper concludes that synthetic (fake) medical images are not a magic cure-all.

• They don't help every type of computer model.
• They don't work if you just throw them in without checking their quality.
• They work best when you have a smart model, a specific ratio of fake-to-real data, and a filter to keep the bad fake images out.

However, when the conditions are right, fake scans can be a powerful tool. They can help advanced models learn more accurately and, perhaps more importantly, help them learn much faster, saving valuable time and computing power when real medical data is scarce.

Technical Summary

Technical Summary: Do Synthetic Brain MRIs Reliably Improve Tumour Classification?

Problem Statement
Medical imaging datasets are often constrained by privacy, annotation costs, disease rarity, and class imbalance. While Generative Adversarial Networks (GANs) have advanced in visual fidelity, the critical scientific question remains whether synthetic images can improve downstream task performance (e.g., tumour classification) on real held-out data, rather than merely achieving high visual realism or low Fréchet Inception Distance (FID). Specifically, this study addresses the "limited-data problem" in brain MRI, where datasets shrink significantly when stratified by both diagnostic class and anatomical plane (axial, coronal, sagittal). The study investigates whether synthetic supplementation (adding GAN-generated images to the training pool) improves classification accuracy, reduces training volatility, or alters training efficiency compared to real-data-only baselines.

Methodology
The study utilized the BRISC 2025 dataset, comprising 6,000 contrast-enhanced T1-weighted MRI scans. After removing 104 exact image duplicates found between training and test sets, the final training set contained 4,896 real images, and the held-out test set contained 1,000 images.

1. Generative Model:

   • Twelve independent StyleGAN2-ADA generators were trained, one for each combination of four tumour classes (glioma, meningioma, pituitary, no tumour) and three anatomical planes.
   • Training was constrained to a single NVIDIA RTX 5070 GPU with a resolution of 128×128 pixels and a budget of 1,000 kimg (thousands of images) per generator.
   • Adaptive Discriminator Augmentation (ADA) was applied with plane-specific restrictions (e.g., excluding horizontal flips for sagittal views to prevent anatomical leakage).
   • Synthetic Filtering: Generated images underwent a feature-space filtering process using InceptionV3 pool3 features. A Mahalanobis distance gate (calibrated to the 97.5th percentile of real data) rejected outliers, followed by farthest-point sampling to ensure diversity.

2. Downstream Classifiers:
   Three distinct classifier families were evaluated to test architecture dependency:

   • Random Forest (RF): Operated on fixed 2,048-dimensional InceptionV3 features.
   • Compact Two-Headed CNN: Trained end-to-end on 128×128 images with separate heads for tumour classification and anatomical plane prediction.
   • MobileViTV2: A pretrained hybrid convolutional-transformer model, adapted for the two-task setup.

3. Experimental Design:

   • Classifiers were trained under five conditions: real-only baseline, unfiltered 1:1 and 1:2 (real:synthetic) ratios, and filtered 1:1 and 1:2 ratios.
   • Evaluation involved ten independent seeds with paired permutation tests and Holm correction for statistical significance.
   • Efficiency Metrics: Beyond accuracy, the study measured the number of epochs (CNN) or real-data epochs (MobileViTV2) required to reach the best validation-loss checkpoint.
   • Blind Test: An independent GPT-5.5 audit assessed the ability to distinguish real from synthetic images on a gated subset (images where the model correctly identified tumour type and plane).

Key Results

• Generative Quality: The StyleGAN2-ADA generators produced coherent MRI-like images. However, recall metrics were low (0.0097–0.0863), indicating that while the generators produced plausible samples, they covered only a small fraction of the real data manifold.
• Blind Discrimination: GPT-5.5 achieved 57.73% accuracy (95% CI: 54.48–60.92%) in distinguishing real from synthetic images on the model-legible subset, only modestly above chance. This suggests the synthetic images lack obvious source-identifying artifacts at the preprocessed resolution.
• Classifier Performance:
  • Random Forest: Synthetic augmentation provided no benefit. Mean accuracy decreased slightly or remained unchanged across all conditions. The filtered 1:1 condition significantly worsened the pituitary F1 score.
  • Compact CNN: Showed consistent mean accuracy gains (up to +0.72%) across all synthetic conditions, but none survived Holm correction for statistical significance. However, all augmented conditions reached their selected checkpoints 42–64% earlier in terms of epochs than the baseline.
  • MobileViTV2: Demonstrated the clearest benefit. The filtered 1:1 condition improved tumour classification accuracy by 1.02% absolute (95% CI: 0.54–1.54%; Holm-corrected p = 0.0104). Unfiltered 1:1 and 1:2 also showed significant gains. Furthermore, MobileViTV2 augmented runs reached their selected checkpoints after 50–67% fewer real-data epochs than the baseline, while simultaneously reducing seed-to-seed variance.
• Ratio and Filtering: The 1:2 ratio was not uniformly superior to 1:1. Filtering was harmful for the RF, neutral for the CNN, and most beneficial for MobileViTV2 at the 1:1 ratio.

Significance and Claims
The paper concludes that synthetic augmentation is not a guaranteed remedy for small medical datasets; its utility is strictly architecture- and ratio-dependent.

1. No Universal Benefit: Visual realism (FID) and low discrimination by general-purpose models (GPT-5.5) do not guarantee downstream utility. The Random Forest, operating on fixed features, failed to benefit, suggesting that synthetic data cannot compensate for a lack of domain-specific feature learning in frozen-feature models.
2. Architecture Sensitivity: Higher-capacity, trainable models (MobileViTV2) benefited most, particularly when synthetic data was filtered and balanced (1:1 ratio). This suggests that end-to-end models can leverage synthetic variation to reduce bias without overfitting, provided the synthetic distribution is not too divergent.
3. Training Efficiency: A significant finding is that synthetic augmentation can improve training efficiency even when final accuracy gains are modest. Augmented models reached optimal checkpoints with significantly fewer passes over the scarce real data, a practical advantage in resource-constrained medical settings.
4. Methodological Rigor: The study emphasizes that synthetic augmentation must be evaluated via held-out task performance rather than image-quality metrics alone. The negative results for the RF and the mixed results for the CNN serve as evidence that "more data" (synthetic or otherwise) does not automatically equate to better performance if the data distribution or model architecture is mismatched.

The authors assert that while StyleGAN2-ADA can assist under specific conditions (high-capacity models, filtered 1:1 ratios), it is not a substitute for real annotated data or clinical validation. The boundary between useful augmentation and harmful distribution shift remains the critical area of inquiry.