TRACE: Temporal Radiology with Anatomical Change Explanation for Grounded X-ray Report Generation

Technical Summary: TRACE – Temporal Radiology with Anatomical Change Explanation

Problem Statement
Clinical radiology relies heavily on the temporal comparison of chest X-rays to detect disease progression, treatment response, and new findings. While existing automated methods have advanced single-image report generation and visual grounding, they fail to address the critical need for temporal change detection. Current approaches typically fall into three disjoint categories: (1) single-image report generation lacks temporal context; (2) temporal classification methods detect progression but output only labels without spatial localization; and (3) grounded report generation localizes findings but operates on single images without temporal reasoning. Consequently, no existing method simultaneously performs temporal comparison, change classification (worsened/improved/stable), and spatial localization to generate grounded reports describing what changed, where, and how.

Methodology
The authors introduce TRACE (Temporal Radiology with Anatomical Change Explanation), a unified framework designed to generate natural language reports with bounding box coordinates for interval changes between a prior and a current chest X-ray.

• Architecture: TRACE builds upon the LLaVA framework, utilizing a frozen BioViL-T vision encoder (pre-trained on chest X-rays with temporal awareness) and a large language model (LLM) decoder (Vicuna-7B or Mistral-7B).
• Feature Fusion: Instead of using explicit difference images or temporal attention modules, the model employs feature concatenation. Features from the prior and current images are extracted independently using shared weights and concatenated to form a sequence of 392 visual tokens. The LLM's self-attention mechanism implicitly handles the cross-image comparison.
• Training Strategy: The model is trained in two stages:
  1. Alignment: The MLP projector is trained to align visual features with the LLM embedding space.
  2. Instruction Tuning: End-to-end fine-tuning using Low-Rank Adaptation (LoRA) on the LLM to generate grounded temporal reports. The training objective minimizes negative log-likelihood for target tokens, which include natural language descriptions and normalized bounding box coordinates (e.g., <box>x1,y1,x2,y2</box>).
• Dataset: The model is trained on a constructed dataset of 79,202 training samples and 22,553 test samples, derived from MIMIC-CXR-JPG and Chest ImaGenome. This dataset pairs consecutive studies from the same patient with grounded temporal annotations indicating whether findings worsened, improved, or remained stable.

Key Results

• Performance: TRACE achieves 48.0% accuracy on the challenging three-way change classification task (worsened, improved, stable). It demonstrates high spatial precision with 90.2% grounding accuracy (IoU > 0.5) and strong clinical efficacy scores (RadGraph F1: 0.406; CheXbert F1: 0.432).
• Ablation Study (Emergent Capability): A critical finding is that change detection is an emergent capability requiring the joint learning of temporal comparison and spatial grounding.
  • Models with access to both images but without grounding supervision failed completely, predicting "stable" for all samples (43.8% accuracy, matching the majority baseline).
  • Single-image models (lacking the prior study) also failed to generate temporal comparisons.
  • The authors hypothesize that grounding provides a necessary spatial attention mechanism, forcing the model to focus on specific anatomical regions to enable meaningful temporal reasoning.
• Anatomical Disparities: Performance varies by anatomy. The model detects cardiac silhouette (68.8%) and mediastinal (75.5%) changes more accurately than lung parenchymal changes (~45%), likely due to the diffuse and subtle nature of pulmonary pathologies.
• Model Sensitivity: A comparison between Vicuna-7B and Mistral-7B revealed a sensitivity-specificity trade-off. Vicuna-7B favored conservative "stable" predictions (high specificity), while Mistral-7B exhibited aggressive change detection (high sensitivity), suggesting different backbones suit different clinical deployment scenarios.

Significance and Contributions
The paper claims three primary contributions:

1. New Task Definition: It introduces "grounded temporal change detection," unifying temporal comparison, classification, and localization into a single framework.
2. Discovery of Emergent Behavior: It uncovers that temporal reasoning in this context is not inherent to multi-image inputs alone but emerges only when combined with spatial grounding supervision. This suggests that grounding acts as a critical inductive bias for temporal analysis.
3. Dataset Construction: It provides a large-scale dataset with grounded temporal annotations to enable systematic evaluation of this new task.

The authors position TRACE as a foundational step toward clinically actionable AI that can describe interval changes with precise spatial context, a capability previously absent in automated radiology systems.