From Attribution to Abstention: Training-Free Attention-Based Auditing for Clinical Summarization

1. Problem Statement

The deployment of Multimodal Large Language Models (MLLMs) in clinical settings (e.g., summarizing doctor-patient dialogues or radiology reports) faces two critical challenges:

• Hallucinations: MLLMs often generate fluent but factually unsupported claims (e.g., misattributing symptoms or inventing diagnoses), which poses severe risks in high-stakes medical domains.
• Lack of Transparency: Existing solutions for transparency (post-hoc attribution) or hallucination mitigation (RLHF, contrastive decoding) are often computationally expensive, require retraining, or fail to provide per-sentence provenance.
• The Gap: There is a need for a training-free, zero-cost mechanism that can simultaneously trace the source of every generated sentence and flag statements that lack evidential support, allowing clinicians to verify outputs efficiently.

2. Methodology: ClinTrace

The authors propose ClinTrace, a training-free framework that extracts two clinically useful signals directly from the decoder attention weights of standard transformer-based MLLMs during generation. It requires no retraining, no auxiliary models, and no additional inference cost beyond enabling attention output.

Core Mechanism

The framework operates in three stages:

1. Source Chunking: The input document (text or multimodal text+image) is segmented into sentence-level evidence units. Image patches are treated as a single contiguous evidence unit.
2. Attention Pooling: For every generated token, the model extracts attention weights across all layers and heads. These are averaged to create a pooled attention vector ($\bar{a}_t$), serving as a denoised estimate of source relevance.
3. Signal Extraction (Dual Output):
   • Source Attribution: Tokens are mapped to the source sentence with the highest summed attention. A generated sentence is attributed to a source span if a majority of its tokens agree on that source (controlled by a threshold $\tau$).
   • Groundedness Scoring ($\phi$): A scalar score representing the "concentration" of attention. High scores indicate the model is attending strongly to specific source evidence; low scores indicate diffuse attention, suggesting a potential hallucination.

Abstention Strategy

The framework utilizes the groundedness score to implement an abstention mechanism. Sentences with scores below a threshold ($\phi_{min}$) are flagged for clinician review rather than presented as confident output. This shifts the burden of verification to the human for uncertain claims, improving overall system reliability.

3. Key Contributions

1. Training-Free Auditing: Demonstrates that decoder attention weights, routinely discarded during inference, contain sufficient signal for both fine-grained source attribution and hallucination detection.
2. Dual-Function Signal: Shows that the same attention tensor provides both provenance (where the info came from) and reliability (how trustworthy it is).
3. Impact of Domain Adaptation: A critical finding that medical finetuning significantly enhances the interpretability of attention patterns. Medical-finetuned models produce sharper, more semantically structured attention, making them far more amenable to self-auditing than general-purpose models.
4. Abstention for Safety: Proposes a practical deployment strategy where the system "knows when it doesn't know," improving faithfulness by withholding low-confidence outputs.

4. Experimental Results

The framework was evaluated on two datasets: CliConSummation (doctor-patient dialogues, text + image) and MIMIC-CXR (radiology reports). Two models were tested: Qwen3-8B (general-purpose) and HuatuoGPT-Vision-7B (medical-finetuned).

A. Source Attribution

• Performance: ClinTrace substantially outperformed baselines (self-attribution, embedding-based similarity, and random).
  • Radiology (Text F1): 92.15% (Qwen3) and 97.22% (HuatuoGPT).
  • Dialogue (Text F1): 88.48% (Qwen3) and 94.09% (HuatuoGPT).
• Multimodal: Achieved ~80% Image F1, correctly linking generated text to patient images.
• Insight: Medical finetuning yields significantly more structured attention patterns, leading to higher attribution accuracy.

B. Hallucination Detection

• Metric: AUROC for distinguishing faithful vs. unfaithful sentences.
• Results:
  • HuatuoGPT (Medical): Achieved 0.77 AUROC, outperforming embedding-based confidence baselines (0.76) at zero extra cost.
  • Qwen3 (General): Achieved 0.59 AUROC, underperforming embedding baselines.
• Conclusion: The attention signal is a strong hallucination detector only when the model has been domain-adapted to medicine.

C. Clinical Utility (Abstention)

• Strategy: Withholding the bottom 20% of sentences based on groundedness scores.
• Outcome for Medical Models: Faithfulness of the retained output increased from 61.7% to 72.6% (an 11-point gain) while maintaining 80% coverage.
• Outcome for General Models: The improvement was marginal and non-monotonic, reinforcing the need for domain-specific models in clinical auditing.

5. Significance and Implications

• Cost-Efficiency: ClinTrace offers a "free" safety layer for clinical AI pipelines, requiring no additional compute or training.
• Model Selection Criteria: The paper argues that for clinical deployment, medical finetuning is not just about task performance (accuracy) but is essential for safety and interpretability. General-purpose models may generate fluent text, but their attention mechanisms are too diffuse to reliably flag their own errors.
• Design Principle: The "Abstention over Silent Failure" approach is highlighted as a critical design pattern for healthcare AI. Instead of silently correcting errors or hallucinating, the system should surface uncertainty to the clinician.
• Correlation of Signals: The study validates that attribution accuracy, groundedness scores, and faithfulness are three views of the same underlying signal: when attention is concentrated, the output is both well-sourced and factually correct.

In summary, ClinTrace provides a robust, zero-cost method to audit clinical LLMs, proving that with the right model (medical-finetuned), the internal mechanics of the transformer can be leveraged to ensure transparency and safety in high-stakes medical documentation.