Ctrl-GenAug: Controllable Generative Augmentation for Medical Sequence Classification

This paper introduces Ctrl-GenAug, a novel generative augmentation framework that addresses the scarcity of medical datasets by enabling controllable, semantically and sequentially coherent synthesis of diagnosis-promotive samples while filtering out unreliable data to significantly improve medical sequence classification performance.

The Gist

Imagine you are trying to teach a student (an AI) how to be a doctor. To do this, you need to show them thousands of examples of healthy and sick patients. But here's the problem: in the real world, serious diseases are rare, and getting a doctor to label (annotate) every single medical scan is slow, expensive, and exhausting. It's like trying to teach someone to recognize a rare type of bird when you only have three photos of it, and the rest of your photo album is full of sparrows.

This is where Ctrl-GenAug comes in. Think of it as a super-smart, controllable "Imagination Machine" that creates fake but realistic medical videos to fill in the gaps.

Here is a simple breakdown of how it works, using some everyday analogies:

1. The Problem: The "Empty Classroom"

Medical AI models are like students who need to study hard to pass exams. But the "textbooks" (medical datasets) are often:

• Too small: Not enough examples of rare diseases.
• Unbalanced: Too many examples of mild cases, too few of severe ones.
• Fragile: If the student studies only in one specific hospital, they might fail when they see a patient from a different hospital with slightly different equipment.

2. The Solution: The "Imagination Machine" (The Generator)

Instead of just copying existing pictures, Ctrl-GenAug uses a Diffusion Model. Think of this like a master artist who starts with a blank canvas covered in static (noise) and slowly paints a clear picture.

But a normal artist might paint a weird monster when you ask for a "mild heart condition." Ctrl-GenAug is special because it has four specific instruction manuals (conditions) to ensure the painting is exactly what the doctor needs:

• The Label: "This is a severe case." (Like telling the artist the genre of the movie).
• The Description: "The nodule has a smooth edge." (Like giving a detailed script).
• The Reference Photo: A real scan of a similar patient to copy the style. (Like a mood board).
• The Motion Map: A guide showing how the heart should beat or how blood flows. (Like a storyboard for movement).

By combining these, the machine doesn't just make any fake video; it makes a customized, high-quality video that looks and moves exactly like a real patient with a specific disease.

3. The Safety Net: The "Quality Control Inspector" (The Filter)

Here is the tricky part: Even the best AI sometimes hallucinates. It might generate a video that looks cool but is medically nonsense (e.g., a heart beating backward or a tumor in the wrong place). If you feed these "bad fake videos" to the student doctor, they will get confused and fail the exam.

Ctrl-GenAug has a built-in Inspector (the Noisy Synthetic Data Filter).

• The Semantic Check: The Inspector asks, "Does this video actually look like the disease label we gave it?" If the AI said "Severe" but the video looks "Mild," the Inspector throws it in the trash.
• The Motion Check: The Inspector checks if the movement is smooth. If the video flickers or jumps weirdly, it gets rejected.
• The Variety Check: The Inspector makes sure we aren't just generating the exact same fake video 1,000 times. It ensures we have a diverse library of examples.

4. The Result: A Super-Student

Once the "Imagination Machine" creates thousands of custom videos and the "Inspector" filters out the bad ones, the AI student gets a massive, diverse, and high-quality library to study.

The paper tested this on 5 different medical datasets (heart, lungs, thyroid, knee, and carotid arteries) and compared it against 15 other methods.

• The Outcome: The AI students trained with Ctrl-GenAug became significantly better at diagnosing diseases.
• The Special Win: They were especially good at spotting rare, high-risk diseases (which usually get ignored because there aren't enough examples) and they didn't get confused when tested on data from different hospitals (out-of-domain).

The Big Picture

Think of Ctrl-GenAug as a bridge. It takes the limited, expensive real-world medical data and builds a massive, safe, and diverse training ground on the other side. This allows AI to learn faster, spot rare diseases better, and become a more reliable assistant for real doctors, all while saving time and money on manual labeling.

In short: It's a tool that teaches AI to imagine rare medical scenarios perfectly, checks its work to ensure it's not lying, and uses those imaginary scenarios to make real-world medical diagnosis more accurate.

Technical Summary

1. Problem Statement

Medical sequence classification (e.g., video sequences from ultrasound, MRI, or CT) faces three critical challenges:

• Data Scarcity & Annotation Cost: Dynamic clinical cases are rare, and annotating them is labor-intensive.
• Class Imbalance: High-risk positive cases (e.g., severe stenosis or tumors) are often underrepresented, skewing model training.
• Domain Generalization: Models often suffer performance degradation when tested on out-of-domain data (e.g., from different medical centers).

While existing generative models (like GANs and Diffusion models) offer solutions, current approaches for medical sequences suffer from:

• Lack of Control: Insufficient semantic steerability (e.g., inability to control specific lesion attributes) and sequential coherence (temporal/stereoscopic consistency).
• Quality Issues: Noisy or incorrectly synthesized samples (e.g., wrong disease grades or anatomical inconsistencies) that degrade downstream classifier performance if used blindly.
• Image-Centric Focus: Most existing works focus on static image generation rather than complex 3D/4D sequence synthesis required for modalities like Cine-MRI or Ultrasound.

2. Methodology: Ctrl-GenAug

The authors propose Ctrl-GenAug, a general framework comprising two main components: a Multimodal Conditions-Guided Sequence Generator and a Noisy Synthetic Data Filter.

A. Sequence Generator Architecture

The generator is built upon a Latent Diffusion Model (LDM) with a two-stage training scheme (Pretraining on 2D images, then Finetuning for sequences) to reduce training costs.

1. Multimodal Conditions Guidance: To ensure high-fidelity and controllable synthesis, the model accepts four types of conditions:
   • Descriptive Text: Provides coarse-grained semantic concepts (e.g., "smooth margin," "hyperechoic"). Text and visual features are aligned using a medical-specific CLIP model.
   • Class Label: Treated as a separate tag (not merged with text) to provide direct disease-specific guidance, crucial for balancing classes.
   • Image Prior: The first frame/slice of a real sequence is used as a prior to bridge the domain gap and ensure the synthetic data matches the real-world style.
   • Motion Field: A computation-efficient motion field (derived from real data) is used as a sequential condition to guide pixel-wise motion between frames, ensuring dynamic consistency.
2. Sequential Augmentation Module (SAM): To address temporal inconsistency, the authors introduce a cascaded attention mechanism:
   • Key-frame/Slice Attention (KA): Selects the first frame and the previous frame as references to propagate spatial-temporal information, aligning latent features across the sequence.
   • Motion Field Attention (MFA): Reuses the motion field condition to ensure patches communicate along specific motion pathways, eliminating flickering and ensuring smooth dynamics.

B. Noisy Synthetic Data Filter

Recognizing that not all generated samples are useful, a three-stage filtering process is applied before training downstream classifiers:

1. Class Semantics Misalignment Filtering: Uses a pre-trained classifier to calculate cross-entropy loss for each synthetic clip. Clips with loss exceeding a class-specific threshold (indicating a mismatch between visual content and the intended label) are discarded.
2. Inner-sequence Filtering: Filters out clips with "gentle dynamics" (too static) or extreme inconsistencies. It uses a novel metric, VAE-Seq (based on VAE latent embeddings rather than CLIP), to measure cross-frame consistency and retains clips within a pre-computed optimal range.
3. Inter-sequence Filtering: Ensures diversity within the synthetic set by calculating cosine distances between clips. Redundant or highly similar clips are removed to prevent overfitting.

3. Key Contributions

• First Comprehensive Study: This is the first work to systematically analyze controllable generative augmentation specifically for medical sequence classification across multiple modalities.
• Novel Framework: Introduction of a framework that combines multimodal conditioning (text, label, image prior, motion) with a quality control filter, addressing both generation quality and downstream reliability.
• Sequential Augmentation Module: A specialized attention mechanism (KA + MFA) that significantly improves temporal and stereoscopic coherence in medical video/3D synthesis.
• Robust Filtering Strategy: A data-centric approach to prune harmful synthetic samples, effectively linking the generation task to downstream classification performance.

4. Experimental Results

The method was evaluated on 5 medical datasets (Carotid US, Thyroid US, Cardiac Cine-MRI, Lung CT, Knee MRI) covering 4 modalities.

• Benchmarking: Ctrl-GenAug outperformed 15 existing augmentation methods (including traditional, mixing-based, GAN-based, and other diffusion-based approaches).
  • It achieved the highest accuracy and AUROC across all datasets. For example, on the Carotid dataset, it reached 85.03% accuracy (vs. ~82% for the best diffusion competitor).
• Classifier Agnosticism: Evaluated with 11 different classifiers (CNNs, Transformers, and Hybrids) under three training paradigms (Baseline, Real-finetune, Joint-train). Ctrl-GenAug consistently improved performance regardless of the backbone architecture.
• High-Risk & Out-of-Domain Performance:
  • Underrepresented Classes: Significant improvements in sensitivity for rare/high-risk classes (e.g., moderate/severe carotid stenosis), where baseline models often failed.
  • Out-of-Domain (OOD): When tested on data from unseen medical centers, the framework significantly enhanced robustness, reducing the domain gap.
• Ablation Studies:
  • Confirmed that all four conditioning types (Text, Label, Image Prior, Motion) contribute to performance.
  • Showed that the Sequential Augmentation Module and Noisy Data Filter are critical; removing them leads to performance drops or even degradation compared to baselines.
  • Demonstrated that the framework can reduce annotation burden, achieving competitive results even when trained with only 25% of labeled real data.

5. Significance and Conclusion

Ctrl-GenAug represents a significant step forward in medical AI by moving beyond simple image synthesis to diagnosis-reliable sequence generation.

• Clinical Utility: By generating high-fidelity, controllable, and diverse synthetic sequences, it directly addresses the data scarcity and imbalance issues that hinder the deployment of deep learning in real-world clinical settings.
• Reliability: The inclusion of a rigorous filtering mechanism ensures that synthetic data acts as a true asset rather than a source of noise, making it a practical tool for augmenting training datasets.
• Generalizability: The framework's success across diverse organs (heart, thyroid, carotid, lung, knee) and modalities (US, MRI, CT) suggests it is a versatile solution for the broader medical imaging community.

The authors conclude that while synthetic data should complement rather than replace real data, Ctrl-GenAug provides a robust, automated pipeline to enhance medical sequence classification, particularly for underrepresented populations and challenging out-of-domain scenarios.