💻 computer science

Transition-Aware best-of-N sampling for Longitudinal Chest X-ray Reports

This paper introduces a training-free, transition-aware best-of-N sampling framework that leverages longitudinal prior exams to improve chest X-ray report generation by scoring candidate reports based on their alignment with ground-truth transition vectors derived from changes between sequential visits.

Original authors: Halil Ibrahim Gulluk, Max Van Puyvelde, Wim Van Criekinge, Olivier Gevaert

Published 2026-06-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Halil Ibrahim Gulluk, Max Van Puyvelde, Wim Van Criekinge, Olivier Gevaert

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a radiologist looking at a patient's chest X-ray. You don't just see a single picture; you see a story. To understand what's happening now, you have to compare it to what happened last time. Did a shadow get bigger? Did a fluid pocket disappear? Did things stay the same?

This paper introduces a new way to help computers write these medical reports by teaching them to pay attention to that "story" of change.

Here is the breakdown of their idea, using simple analogies:

1. The Problem: The "Isolated" AI

Currently, most AI models that write X-ray reports look at a single image in a vacuum. They generate a few different versions of a report (let's say 5 different drafts) and then pick the one that looks most like a "good" report based on general rules.

The Flaw: It's like asking a writer to describe a movie scene without telling them what happened in the previous scene. The AI might write a perfectly grammatical sentence like "The heart is enlarged," but if the patient's heart was already enlarged last week and hasn't changed, that sentence is misleading. The AI might also accidentally "hallucinate" a new problem that isn't there, just because it sounds like a typical medical report.

2. The Solution: The "Change Detector"

The authors propose a method called Transition-Aware Best-of-N Sampling.

Think of the AI as a chef who is asked to cook 5 different versions of a soup based on a photo of ingredients.

Old Way: The chef tastes all 5 soups and picks the one that tastes the most like a "standard soup."
New Way: The chef knows the customer had a different soup last week. The chef compares the 5 new soups to the difference between last week's soup and this week's ingredients. They pick the soup that best represents the change (e.g., "We added more salt this time," or "We removed the carrots").

3. How It Works (The "Magic" Steps)

Step A: Breaking it into Lego blocks
The computer takes the medical report and chops it up into individual sentences. It turns each sentence into a mathematical "fingerprint" (a vector). Instead of caring about the order of the sentences (since medical findings can be listed in any order), it treats them as a messy pile of Lego blocks.

Step B: Measuring the "Shift"
The system looks at the "pile of blocks" from the Previous Visit and the Current Visit. It calculates a specific "Change Vector."

Analogy: Imagine you have a pile of red blocks (yesterday) and a pile of blue blocks (today). The system doesn't just compare the piles; it calculates the exact "arrow" or "wind" that blew the red pile to become the blue pile. Did the pile grow? Did it shrink? Did it shift to the left?

Step C: The "Memory Bank"
The system has a library (a cache) of thousands of these "Change Vectors" from real, human-written reports in the past. It knows what a "healthy, stable change" looks like, and what a "sudden worsening" looks like.

Step D: The Selection
When the AI generates 5 new candidate reports, it turns each one into a "Change Vector" and compares it to the Memory Bank.

If a candidate report suggests a dramatic change that doesn't match any real-world patterns in the Memory Bank (e.g., "The patient suddenly developed a massive tumor" when the X-ray looks fine), the system rejects it.
It picks the report whose "change vector" is closest to the real, human-observed changes in the bank.

4. The Results

The authors tested this on a large dataset of patients who had multiple X-rays over time. They used three different AI models to generate reports.

The Winner: The "Change Detector" method consistently picked better reports than just guessing randomly.
The Big Win: The improvement was most noticeable in the "Impression" section of the report. This is the summary where the doctor says, "Things are stable," or "There is a new problem." This makes sense because the "Impression" is entirely about the change from the last visit.
No Extra Training: The best part is that they didn't have to retrain the AI models. They just added this "filter" at the end to pick the best answer.

Summary

In short, this paper teaches AI to stop looking at X-rays as isolated snapshots and start seeing them as a sequence. By measuring the "direction" of the change between visits and comparing it to a library of real human changes, the AI can pick the report that tells the most accurate story of the patient's health journey.

Technical Summary: Transition-Aware Best-of-N Sampling for Longitudinal Chest X-ray Reports

Problem Statement
In longitudinal clinical practice, chest X-rays are rarely interpreted in isolation; they are read in the context of a patient's prior examinations. Radiologists focus heavily on the change between visits (e.g., "stable interval," "new opacity," "improved aeration"). While modern Vision-Language Models (VLMs) can generate credible chest X-ray reports, standard Best-of-N sampling pipelines treat each test image independently. These pipelines typically score candidate reports against a distribution of single-image reference reports, ignoring the temporal transition from the prior study. Consequently, a candidate report might appear plausible in isolation but describe a clinically invalid transition (e.g., hallucinating acute worsening or silently dropping chronic findings) relative to the patient's history.

Methodology
The authors propose a training-free, Transition-Aware Best-of-N Sampling framework. This method operates downstream of the generator, requiring no modifications to the pre-trained VLMs. The pipeline consists of three main stages:

Sentence-Set Report Representation:
Each report (divided into Findings and Impression sections) is split into individual sentences. A frozen sentence transformer (specifically all-mpnet-base-v2) embeds each sentence into a fixed-dimensional vector ( $d=768$ ). The report is represented as an unordered set of these embeddings, ensuring permutation invariance, which aligns with the clinical reality that the order of findings carries no diagnostic meaning.
Directional Transition Representation:
For a patient with a prior visit $r^{(k-1)}$ and a current visit $r^{(k)}$ , the method computes a directional set-to-set distance between the sentence embedding sets of the two visits. This reduces the pair to a single fixed-dimension vector ( $g \in \mathbb{R}^{d'}$ ) representing the "change." The authors instantiate four specific directional distance designs:
- Mean-Shift: The centroid shift of the current set relative to the prior set.
- Novelty Residual: The mean residual of each current sentence relative to its nearest neighbor in the prior set (capturing what is new).
- Directed-Hausdorff Anchor: The displacement at the worst-covered point in the current set relative to the prior set.
- Cost-Weighted Optimal Transport (Cost-OT): A displacement vector weighted by both transport mass and cosine cost, avoiding trivial collapse.
Transition-Aware Selection:
A bank of ground-truth transition vectors is pre-computed from the training corpus. At test time, for a given patient and $K$ candidate reports generated by a VLM, each candidate is converted into a transition vector (comparing the patient's actual prior report to the generated current report).
- Candidates are scored based on the cosine distance between their transition vector and the cached bank of training transition vectors.
- Aggregation strategies include Min (closest to any single training transition) and k-Nearest Neighbors (kNN) (average distance to the $k$ closest training transitions).
- The candidate with the minimum total distance (summed across Findings and Impression sections) is selected.

Key Contributions

First Training-Free Longitudinal Selector: To the authors' knowledge, this is the first Best-of-N sampling scheme for pre-trained chest X-ray generators that is explicitly aware of the longitudinal prior-to-current transition.
Transition Space Scoring: Unlike previous methods that score candidates in "report space" (similarity to a static reference), this method scores in "transition space." This penalizes candidates that describe clinically implausible changes, even if the text itself looks like a valid report.
Modular Framework: The approach is generator-agnostic and prompt-agnostic, demonstrated across three different VLMs (Gemini-2.5-Flash-Lite, Gemini-3.1-Flash-Lite-preview, Mistral-Small-2603) and three prompting regimes (zero-shot, few-shot, and zero-shot conditioned on prior reports).

Experimental Results
The framework was evaluated on a multi-visit AP/PA cohort derived from the ReXGradient-160K dataset (22,745 training transitions, 1,727 test transitions). Performance was measured against a random selection baseline using NLP overlap metrics (BLEU, ROUGE, METEOR, BERTScore) and clinical content metrics (RadGraph, CheX-14).

Impressions: The method yielded substantial relative gains. The strongest configurations achieved up to +8.5% relative improvement in ROUGE-L, +13.6% in RadGraph averaged-F1, and +4.4% in METEOR. The authors note that the Impression section, being a short, change-oriented summary, is the regime where the transition signal provides the most ranking information.
Findings: Every transition method (across all distance types and aggregation strategies) outperformed the random baseline on every headline metric. The Cost-OT/min configuration showed the strongest results for BERTScore-F1 (+3.1%) and RadGraph (+5.1%).
Consistency: The approach consistently outperformed random selection across all three VLMs and all three prompting strategies.

Significance and Claims
The paper claims that modeling the longitudinal context at the selection stage is a critical, previously unexploited avenue for improving report generation. By leveraging a bank of ground-truth transitions, the system provides a prior on how reports change between visits, rather than just what individual reports look like. This effectively filters out candidates that, while linguistically fluent, describe clinically impossible progressions (e.g., sudden disappearance of chronic conditions without explanation). The authors position this as a practical, training-free scaling tool that enhances the reliability of automated radiology reporting in longitudinal settings.