🤖 AI

Generating Reports or Repeating Templates? Measuring and Mitigating Template Collapse in 3D CT Report Generation

This paper identifies and addresses "Template Collapse" in 3D CT report generation, where models produce fluent but clinically inaccurate generic text, by proposing CLarGen, a decoupled framework that significantly improves diagnostic accuracy and output diversity through separate pathology detection and language synthesis components.

Original authors: Tom Maye-Lasserre, Yitong Li, Bailiang Jian, Morteza Ghahremani, Benedikt Wiestler, Christian Wachinger

Published 2026-06-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Tom Maye-Lasserre, Yitong Li, Bailiang Jian, Morteza Ghahremani, Benedikt Wiestler, Christian Wachinger

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Copy-Paste" Doctor

Imagine a highly intelligent robot doctor that looks at 3D CT scans of lungs and hearts. Its job is to write a detailed report for a human doctor, explaining exactly what it sees.

The researchers found that these robot doctors have a serious flaw they call "Template Collapse."

Think of it like a student taking a test. Instead of actually looking at the specific patient's scan and describing what is unique about that person, the robot gets lazy. It memorizes a few "safe" answers (templates) that sound very professional and fluent.

The Result: The robot writes reports that sound perfect grammatically (like a native speaker), but they are often wrong. It might say, "Everything looks normal," for a patient who actually has a serious lung infection, because "Everything looks normal" is the most common template in its memory.
The Danger: In medicine, missing a rare but critical finding (like a small tumor) because the robot defaulted to a generic "all clear" template can be dangerous. The robot is prioritizing sounding smooth over being accurate.

Why Does This Happen?

The paper explains that 3D CT scans are incredibly complex (millions of tiny 3D pixels called voxels), but the data available to train these robots is limited and unbalanced.

The Analogy: Imagine trying to teach a student to recognize rare birds, but 99% of the photos you show them are of pigeons. The student learns to say "Pigeon" for everything because it's the safest bet to get a good grade.
The Trap: The robots are trained to predict the next word in a sentence. Since "normal" reports are common in the training data, the robots learn that the easiest way to get a high score is to repeat common phrases rather than hunt for rare, difficult-to-spot abnormalities.

The Solution: CLarGen (The "Specialist Team")

To fix this, the authors created a new system called CLarGen. Instead of asking one giant robot to do everything (look at the scan and write the report), they broke the job down into three specialized steps, like a team of experts:

The Detective (Clinical Perception):
- First, a specialized tool looks at the 3D scan and acts like a detective. It doesn't try to write sentences yet. It just asks: "Is there fluid? Is there a nodule? Is the heart enlarged?"
- It creates a strict checklist of findings based on the visual evidence. This ensures the "what" is identified before worrying about the "how."
The Librarian (Pathology-Guided Retrieval):
- Next, a librarian searches a database of past reports. But this librarian doesn't just look for reports that sound similar. They look for reports that have the same medical checklist as the current patient.
- If the Detective found a rare heart issue, the Librarian finds a past report that also had that specific heart issue, ensuring the context is medically relevant.
The Scribe (Frozen Language Model):
- Finally, a highly trained medical writer (a large language model) takes the Detective's checklist and the Librarian's relevant examples.
- Crucial Step: This writer is "frozen," meaning it isn't allowed to change its brain or learn new shortcuts. It simply uses its existing medical knowledge to turn the checklist and examples into a fluent, professional report. Because it can't "guess" the answer, it must stick to the facts provided by the Detective.

The Results: Accuracy Over Style

The researchers tested this new team against the old "single robot" models.

The Old Robots: They wrote beautiful, flowing reports that sounded like a doctor wrote them, but they missed a huge number of actual diseases. They were like a smooth-talking salesman who ignores the product defects.
CLarGen: It wrote reports that were slightly less "perfect" in terms of fancy word overlap, but they were much more accurate.
- It caught rare diseases that the others missed.
- It didn't just repeat the same "normal" template over and over.
- It successfully balanced sounding professional with being medically truthful.

The Big Takeaway

The paper concludes that for medical AI, you cannot just rely on making the text sound good. If you want a robot to write a medical report that is actually safe and useful, you have to force it to explicitly identify the medical facts before it starts writing sentences. You have to separate the "detective work" from the "storytelling" to prevent the robot from collapsing into lazy, generic templates.

Technical Summary: Measuring and Mitigating Template Collapse in 3D CT Report Generation

1. Problem Statement: Template Collapse

The paper identifies a critical failure mode in modern 3D medical vision-language models (VLMs) used for radiology report generation, termed Template Collapse. While these models produce linguistically fluent reports that score well on standard NLP metrics (e.g., BLEU, ROUGE), they exhibit critically low clinical accuracy and output diversity.

Instead of generating patient-specific descriptions, autoregressive VLMs tend to converge on a small set of generic, "normal" report templates. This behavior is driven by the unique constraints of 3D medical imaging:

Data Scarcity: 3D CT report datasets are significantly smaller than 2D chest X-ray datasets.
Label Imbalance: Pathological findings are extremely sparse; most findings are absent in any given scan.
Weak Signals: Volumetric encoders struggle to align large 3D volumes with fine-grained clinical semantics.

Under these conditions, standard next-token prediction objectives incentivize "shortcut learning." The models learn to reproduce frequent, generic phrasing (often describing normal anatomy) to maximize fluency metrics, while suppressing rare but critical pathological findings. This results in reports that are linguistically plausible but diagnostically uninformative or even harmful due to missed abnormalities.

2. Methodology: CLarGen Framework

To address Template Collapse, the authors propose CLarGen, a decoupled framework that explicitly separates clinical detection (what to say) from language synthesis (how to say it). Unlike end-to-end fine-tuning approaches, CLarGen operates in three distinct stages:

Stage 1: Latent Query Transformer (LQT) for Clinical Perception

Architecture: A frozen 3D image encoder (e.g., HLIP) outputs features $Z$ . A Latent Query Transformer with $C=18$ learnable pathology queries attends to these features via cross-attention.
Mechanism: Each query corresponds to a specific pathology (e.g., pleural effusion, atelectasis). The model extracts pathology-specific evidence and models correlations between findings via self-attention.
Training: The LQT is trained with a focal loss to handle extreme class imbalance, optimizing for multi-label pathology detection independently of language generation.

Stage 2: Pathology-Guided Retrieval

Goal: Retrieve exemplar reports that are visually similar to the query CT but, crucially, clinically consistent with the predicted pathologies from Stage 1.
Scoring: A clinical alignment score ( $s_{find}$ ) is computed to reward reports containing high-probability predicted findings and penalize those containing unlikely ones.
Selection: A coverage-aware greedy selection algorithm iteratively chooses $k$ exemplars to ensure the retrieved set collectively covers all high-confidence predicted pathologies, preventing the retrieval of only "normal" templates.

Stage 3: Frozen Medical LLM Synthesis

Architecture: A large, pre-trained medical LLM (MedGemma-27B) is kept frozen during training.
Input: The LLM receives:
1. Predicted pathology labels from the LQT.
2. The $k$ retrieved exemplar reports.
3. Patient metadata (age, sex).
Reasoning: The LLM synthesizes the final report by reasoning over the detected findings and retrieved context. Keeping the LLM frozen prevents it from overfitting to dataset-specific shortcuts or learning the "normal template" bias, preserving its pre-trained clinical reasoning capabilities.

3. Key Contributions

Identification of Template Collapse: The paper formally characterizes a failure mode where VLMs sacrifice diagnostic accuracy for fluency, converging on generic templates and suppressing rare findings.
Systematic Measurement: The authors propose a multi-dimensional diagnostic framework to measure this failure, including:
- Clinical Fidelity: Macro-F1, Precision, Recall, and CRG scores.
- Output Diversity: Distinct-n scores and effective n-gram vocabulary ( $effV_n$ ) to detect semantic degeneration.
- Bias Analysis: Normal Bias Similarity (NBS) and prevalence-stratified survival metrics ($Survrare$ vs. $Survcommon$) to quantify the suppression of rare pathologies.
CLarGen Framework: A pathology-guided, retrieval-augmented generation pipeline that decouples perception from synthesis, demonstrating that explicit clinical grounding is necessary to mitigate template collapse.

4. Experimental Results

The framework was evaluated on the CT-RATE dataset (25,678 chest CT studies) against state-of-the-art baselines (CT-CHAT, SAMF, Reg2Rg, HULU-Med).

Clinical Accuracy: CLarGen achieved a Macro-F1 of 0.486 and a CRG score of 0.472, significantly outperforming the best baseline (HULU-Med, F1=0.194). This represents a ~150% improvement in clinical detection accuracy.
Rare Finding Survival: Baselines showed a massive gap between common and rare pathology recall (e.g., SAMF had 0.000 recall for rare findings). CLarGen achieved balanced performance with $Survrare$ = 0.628 and $Survcommon$ = 0.622, effectively recovering rare findings at the same rate as common ones.
Diversity: Baselines exhibited low effective n-gram vocabularies (e.g., SAMF $effV_2$ = 271 vs. Ground Truth 2402), indicating heavy reliance on templates. CLarGen reached 76% of the ground-truth effective bigram repertoire.
LLM-as-a-Judge: While baselines scored high on "Radiology Style," they dropped sharply on "Factual Accuracy" and "Clinical Safety." CLarGen maintained high scores across all dimensions, including style.
Visual Encoder Scaling: Unlike end-to-end VLMs, whose performance plateaued regardless of encoder quality, CLarGen's performance scaled faithfully with stronger visual encoders (recovering 89-95% of raw detection potential).

5. Significance and Claims

The paper argues that explicit, measurable clinical grounding is essential for robust 3D CT report generation. The authors claim that:

Standard text-optimization objectives are insufficient for medical tasks where accuracy is paramount; they inherently bias models toward frequent "normal" templates in low-signal, high-imbalance regimes.
Decoupling clinical perception (detection) from language synthesis allows the system to attend to weak visual signals that joint pipelines overlook.
Keeping the LLM frozen prevents the model from learning dataset-specific shortcuts, ensuring it acts as a genuine reasoning module rather than a pattern-matching predictor.

The authors conclude that reliable 3D CT report generation requires moving beyond stronger text generation alone and instead prioritizing explicit clinical evidence integration. They acknowledge limitations regarding the current scope (CT-RATE dataset only) and dependency on the quality of upstream 3D medical encoders.