💻 computer science

ProtoMedAgent: Multimodal Clinical Interpretability via Privacy-Aware Agentic Workflows

ProtoMedAgent is a privacy-aware, multimodal framework that enhances clinical interpretability and documentation by replacing error-prone RAG methods with a neuro-symbolic, agentic workflow that mathematically constrains LLM generation to prevent hallucinations while ensuring data anonymity through $k$ -anonymity and $\ell$ -diversity.

Original authors: Alvaro Lopez Pellicer, Plamen Angelov, Marwan Bukhari, Yi Li, Eduardo Soares, Jemma Kerns

Published 2026-05-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Alvaro Lopez Pellicer, Plamen Angelov, Marwan Bukhari, Yi Li, Eduardo Soares, Jemma Kerns

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a doctor trying to explain a complex medical scan to a patient. You have a super-smart computer that looks at the X-ray and says, "This looks like a broken bone." But the computer can't speak; it just gives you a number or a raw image. If you ask a standard AI (like a very chatty robot) to write the explanation for you, it might get too creative. It might say, "The computer sees a fracture because the bone looks like a tree branch," even if the computer never mentioned a tree branch. It makes up stories to sound convincing. This is called "hallucination," and in medicine, making things up is dangerous.

The paper introduces ProtoMedAgent, a new system designed to stop the AI from making things up and to keep patient secrets safe. Here is how it works, using simple analogies:

1. The Problem: The "Yes-Man" Robot

Imagine you have a strict, silent expert (the Prototype Network) who looks at a patient's scan and says, "This matches Case #42."

The Old Way (Standard RAG): You ask a chatty AI to explain why it matches Case #42. The AI, wanting to please you, might invent reasons like, "It matches because the bone texture looks like a sponge," even if the expert never said that. It tries to "sycophantly" agree with the expert by making up a story that fits.
The Result: The report sounds good but is factually unreliable.

2. The Solution: The "Strict Librarian" System

ProtoMedAgent changes the rules. Instead of letting the AI write freely, it forces the AI to act like a Strict Librarian who can only use specific, pre-approved cards.

The "ProtoCards" (The Memory Bank): Before the AI ever sees a patient, the system takes thousands of past medical cases and turns them into simple, discrete "cards." Each card has a few bullet points: "Bone looks dense here," "Patient is 80 years old," "Risk factor: Smoking."
The Privacy Gate (The Redaction Filter): Before these cards are shown to the AI, a security guard checks them. If a card is too unique (like "The only 80-year-old smoker in the city"), the guard redacts (blacks out) the specific details so the AI can't guess who the patient is. This keeps patient privacy safe without needing to add "noise" that would blur the medical facts.

3. The Process: The "Scribe and Critic" Loop

When a new patient comes in, the system doesn't just ask the AI to write a report. It runs a two-step game:

The Scribe (The Writer): The AI tries to write a report based only on the approved cards it was given.
The Critic (The Fact-Checker): A second AI agent acts as a strict editor. It checks the report against the cards.
- Did the Scribe say the bone is "spongy"?
- Critic checks the cards: "No, the card only says 'dense'."
- Critic says: "Delete that sentence. You made it up."
- Scribe rewrites: "The bone is dense."

This loop repeats until the report is perfectly aligned with the evidence. The AI is not allowed to invent new facts; it can only rearrange the facts it was given.

4. The Results: Trustworthy and Safe

The researchers tested this on over 4,000 patients.

Faithfulness: The new system was 91% accurate in sticking to the facts, compared to only 46% for the standard "chatty" AI. It stopped the AI from making up reasons.
Privacy: By using the "Privacy Gate," the system reduced the risk of someone figuring out which specific patient a report was about by about 10%. It proved that you can keep the medical details clear while hiding the patient's identity.

5. The Bottom Line

ProtoMedAgent is like a safety harness for medical AI.

It doesn't try to make the AI smarter; it just stops it from lying.
It forces the AI to act like a translator that can only speak the language of the evidence it has been given.
It ensures that every sentence in a medical report can be traced back to a specific, verified fact, making the whole process transparent and safe for doctors to use.

In short, it turns a "creative writer" into a "faithful reporter," ensuring that when the AI speaks about a patient, it is telling the truth based on the evidence, not a story it made up.

Technical Summary: ProtoMedAgent

Problem Statement

While interpretable prototype networks offer case-based reasoning for clinical diagnostics by retrieving training exemplars ("this looks like that"), their raw outputs—continuous feature maps and scalar similarity scores—lack the semantic structure required for medical documentation. Bridging this gap using standard Retrieval-Augmented Generation (RAG) with Large Language Models (LLMs) frequently fails. When tasked with interpreting unconstrained visual features and clinical priors, LLMs exhibit "retrieval sycophancy," hallucinating post-hoc rationalizations to align with the network's predictions rather than faithfully reporting the evidence. Furthermore, directly exposing raw training exemplars to LLMs creates severe privacy vulnerabilities, including membership inference and attribute inference attacks, while standard Differential Privacy often degrades the clinical fidelity required for dense medical imagery.

Methodology

ProtoMedAgent addresses these challenges by formalizing multimodal clinical reporting as an iterative, zero-gradient test-time optimization problem operating over a strict neuro-symbolic bottleneck. The framework decouples the diagnostic forward pass from evidence documentation, utilizing a frozen prototype backbone (ProtoMedX) and a suite of dedicated agents.

1. Offline Memory Distillation & Privacy

Raw training images and tabular data are distilled into a discrete semantic memory bank of "ProtoCards" ( $C_j$ ) rather than exposing raw exemplars.

Abstraction: A Perception Agent (Vision-Language Model) extracts visual assertions, while a Tabular Agent quantizes patient variables.
Semantic Privacy Gate: A non-differentiable regularizer enforces $k$ -anonymity and $\ell$ -diversity on the release signature of these cards. If a prototype does not meet the privacy thresholds (e.g., insufficient diversity in the equivalence class), it is redacted from the LLM's context, though it may still contribute to the diagnostic forward pass.

2. Online Reflective Test-Time Optimization

During inference, a new patient query is abstracted into a "CaseCard." The framework then employs a Scribe-Critic loop to generate the report:

Deterministic Differentials: A Supervisor agent computes exact set-theoretic differentials ( $\Delta_j$ ) between the query's assertions and the retrieved ProtoCards. This defines an absolute admissible state space, preventing the LLM from hallucinating correlations not present in the data.
Visual Deferral: If the alignment between the query and retrieved prototypes falls below a threshold, the system suppresses image-based similarities to avoid forcing rationalization of contradictions.
Iterative Optimization: The report generation is framed as a zero-gradient search. A Scribe agent proposes natural language reports, while a Critic agent evaluates them against an energy function ( $E_{critic}$ ) that penalizes schema invalidity, citation hallucinations, type inconsistencies, and logical entailment failures. The loop iterates until the report achieves perfect neuro-symbolic alignment with the evidence.

Key Contributions

Agentic LLM-in-the-Loop Test-Time Optimization: The authors formalize clinical reporting as a zero-gradient optimization process. By leveraging an offline privacy-gated memory and an online reflective reasoning loop, the framework safely decodes a frozen prototype classifier without requiring gradient updates to the generative model.
Neuro-Symbolic Verification: The work replaces heuristic LLM comparisons with deterministic set-theoretic differentials. An iterative Scribe-Critic loop guarantees citation accuracy and typed-value consistency, fundamentally mitigating retrieval sycophancy.
Provable Privacy-Utility Bounds: A novel semantic release gate utilizing $k$ -anonymity and $\ell$ -diversity is introduced for continuous prototype memory. This approach mathematically bounds disclosure risk while improving empirical faithfulness, avoiding the fidelity loss associated with noise-injection methods.

Results

Evaluated on a 4,160-patient clinical cohort for bone health screening (DEXA scans and tabular risk factors), ProtoMedAgent demonstrates significant improvements over baselines:

Faithfulness: The system achieves 91.2% Comparison Set Faithfulness (CSF-Precision), significantly outperforming standard RAG (46.2%) and domain-adapted LoRA models (63.1%). This confirms that the neuro-symbolic bottleneck effectively restricts unsupported narrative claims.
Privacy: The framework leverages a binding $\ell$ -diversity phase transition. At the selected operating point ( $k=5, \ell=2$ ), the system reduces artifact-level membership inference risks by an absolute 9.8% and attribute inference risks by 8.4% compared to standard RAG, without modifying the underlying generative model weights.
Qualitative Performance: The generated reports maintain clinical fluency while exhibiting evidence-calibrated reasoning. The system adapts to redacted evidence by producing more conservative language rather than hallucinating missing details.

Significance and Claims

The paper posits that ProtoMedAgent successfully bridges the semantic gap between continuous visual reasoning and discrete clinical documentation. By repositioning the LLM as a verifiable interpretability layer rather than an unconstrained confabulator, the framework achieves state-of-the-art reporting fidelity without costly model retraining.

The authors claim that framing privacy-aware reporting as an explicit operating-point optimization reveals a structural property where $\ell$ -diversity acts as a binding constraint, effectively saturating admissible equivalence classes and limiting information flow. This design aligns with emerging EU regulations (EU AI Act and EU Data Act) by providing auditable, human-overseeable, and robust clinical AI that supports explicit human deferral.

The paper modestly acknowledges limitations: the framework is bounded by the upstream Perception Agent's potential visual blind spots, and while the neuro-symbolic bottleneck prevents hallucinated structural claims, it does not guarantee the absolute truthfulness of the underlying backbone's perception. Additionally, the privacy guarantees rely on $k$ -anonymity and $\ell$ -diversity, which lack the worst-case theoretical guarantees of Differential Privacy against adversaries with out-of-schema auxiliary knowledge.