Technical Summary: Clinical Consensus Selection (CCS) for Radiology Report Generation

1. Problem Statement

Radiology Report Generation (RRG) is typically formulated as a single-path generation task where a Multimodal Large Language Model (MLLM) produces a single decoded report as the final output. While recent advancements have focused on scaling model capacity, training data, and retrieval mechanisms, the paper argues that improving report quality at inference time remains underexplored.

The core problem identified is that fixed radiology MLLMs often generate clinically superior reports within their candidate pool that are discarded by default decoding strategies (e.g., greedy or standard sampling). A single decoding step can lead to omitted findings or unsupported assertions with no mechanism for recovery. Existing inference-time optimization methods (e.g., Best-of-N, self-consistency) rely on generic selection criteria such as perplexity, token confidence, or surface-level textual similarity. These criteria are ill-suited for radiology, where clinical correctness depends on specific anatomical findings, laterality, and temporal interpretations that may not correlate with fluency or lexical overlap. Furthermore, at test time, ground-truth reference reports are unavailable, making reference-free selection critical.

2. Methodology: Clinical Consensus Selection (CCS)

The authors propose Clinical Consensus Selection (CCS), a decoder-agnostic, reference-free inference-time framework. CCS reframes RRG as a candidate selection problem over a rollout pool rather than a single generation trajectory.

2.1 Framework Overview

CCS operates in four stages (illustrated in Figure 2):

1. Rollout Pool Generation: Given an input image $x$ and question $q$, the MLLM samples $N$ candidate reports ($Y = \{y_1, ..., y_N\}$) using stochastic decoding with temperature $\tau$. This stage requires no model retraining or parameter updates.
2. Pairwise Utility Scoring: The framework computes a utility score $U(y_i, y_j)$ for every pair of candidates in the pool. CCS utilizes two types of utilities:
   • Textual Utility: Repurposes standard evaluation metrics (e.g., ROUGE, BERTScore, RadGraph-F1) as pairwise agreement scores.
   • Image-Grounded Utility: This is the novel contribution. The authors adapt Qwen3-VL-Embed, a multimodal embedder trained on image-report pairs, to encode candidate reports. The utility is defined as the cosine similarity between the embeddings of two reports in this radiology-adapted representation space. This measures agreement beyond surface text, capturing semantic alignment with the underlying clinical findings.
3. Consensus Aggregation: For each candidate $y_i$, a consensus score $s_i$ is calculated as the mean pairwise utility against all other $N-1$ candidates in the pool.
4. Final Selection: The report with the highest consensus score is selected as the final output.

2.2 Key Design Choices

• Decoder-Agnostic: CCS can be applied to any pre-trained radiology MLLM without modifying its architecture or weights.
• Reference-Free: The selection process does not require access to ground-truth reports during inference.
• Multimodal Grounding: By using an image-trained embedder, the system attempts to select reports that are consistent with the visual evidence, addressing the limitation of text-only consensus which may favor conservative or "no finding" reports.

3. Key Contributions

The paper outlines three primary contributions:

1. Inference-Time Perspective: The authors demonstrate that candidate pools generated by fixed MLLMs routinely contain reports with higher clinical reliability and consistency than the single-path output, identifying inference-time decision-making as a critical bottleneck.
2. CCS Framework: They introduce a Best-of-N framework that aggregates pairwise clinical consensus using both textual and image-grounded multimodal utilities.
3. Empirical Validation: Extensive experiments across three datasets (MIMIC-CXR, IU-Xray, CheXpert Plus) and multiple MLLM backbones (LLaVA-Med, LLaVA-Rad, Libra) show that CCS consistently improves performance, particularly on clinical metrics, over single-path decoding and generic Best-of-N baselines.

4. Experimental Results

The evaluation was conducted on standard radiology benchmarks using both lexical metrics (ROUGE-L, BLEU, BERTScore) and radiology-specific metrics (RadGraph-F1, RaTEScore, RadEval-BERT, CheXbert-F1).

• Performance Gains: CCS, instantiated with the Qwen3-VL-Embed utility, consistently outperformed the sampling baseline and generic Best-of-N methods (Perplexity, Self-Certainty, ModeX). Notably, gains were most significant on radiology-specific metrics (e.g., RadGraph-F1, RaTEScore), indicating improved clinical correctness rather than just textual fluency.
• Cross-Backbone Generalization: The improvements were consistent across different MLLM backbones and datasets, suggesting the method is not tied to a specific generator.
• Ablation Studies:
  • Utility Comparison: Textual utilities tended to perform best on the metrics they were derived from but failed to capture symptom-level consensus effectively. The image-grounded utility provided distinct selection signals, bridging the gap between textual agreement and clinical findings.
  • Pool Size: Increasing the rollout size ($N$) improved selection performance, though with diminishing returns. $N=8$ was identified as a practical trade-off.
  • Temperature: Moderate sampling temperatures ($\tau \in [0.25, 0.50]$) provided the best balance between candidate diversity and report quality.
• Qualitative Analysis: Case studies revealed that single-path decoding often produced hallucinations (e.g., unsupported findings) or omitted key observations. CCS successfully recovered reports that preserved critical findings (e.g., pulmonary edema, catheter positions) while avoiding factual errors.

5. Significance and Claims

The paper claims that radiology MLLMs often generate better reports than those they initially commit to, and that these superior candidates can be recovered without retraining the model.

• Inference-Time Opportunity: The work highlights a "headroom" for improving RRG at inference time, suggesting that the bottleneck lies not in the model's generative capacity but in the selection mechanism.
• Image-Grounding as a Selection Axis: The study establishes that image-grounded utility forms a selection axis distinct from textual consensus. It helps mitigate the "silence bias" of text-based consensus, where models might agree on the absence of findings (conservative reporting) rather than correctly identifying abnormalities.
• Practicality: As a parameter-free, decoder-agnostic approach, CCS offers a low-cost deployment solution that can be integrated with existing pre-trained models to enhance clinical reliability.

6. Limitations

The authors acknowledge several limitations:

• Dataset Scope: Experiments were limited to chest X-rays and curated datasets, which may not fully capture the diversity and noise of real-world clinical workflows.
• Evaluation: The study relied on automatic clinical metrics rather than assessment by licensed radiologists.
• Scope of Validation: The work did not include LLM-as-a-judge evaluations or explore larger multimodal embedding backbones for consensus estimation.

The authors emphasize that their method is intended for assistive radiology AI research and that practical deployment requires licensed clinician oversight and rigorous validation.