CARE: Privacy-Compliant Agentic Reasoning with Evidence Discordance

1. Problem Statement

The paper addresses a critical gap in using Large Language Models (LLMs) for high-stakes clinical decision-making: evidence discordance.

• The Core Challenge: In real-world Intensive Care Unit (ICU) settings, subjective evidence (e.g., patient-reported pain, bedside observations) often contradicts objective physiological signs (e.g., low blood pressure, high lactate). Standard LLMs, particularly those operating in a "single-pass" manner, struggle to reconcile these conflicting signals. They often default to "automation bias," trusting reassuring subjective narratives over subtle but critical objective abnormalities, or they collapse into degenerate predictions (always predicting one class).
• The Privacy Constraint: High-performance reasoning often requires proprietary, closed-source LLMs (e.g., GPT-5). However, strict privacy regulations prevent these remote models from accessing raw patient data. Conversely, local open-source models can access the data but often lack the reasoning capability of frontier models.
• The Goal: Develop a framework that leverages the reasoning strength of proprietary models while keeping sensitive patient data local, specifically designed to handle cases where signs and symptoms disagree.

2. Methodology: The CARE Framework

The authors propose CARE (Privacy-Compliant Agentic REasoning), a multi-stage agentic workflow that decouples global reasoning guidance from local patient-specific execution.

A. The MIMIC-DOS Dataset

To evaluate this problem, the authors constructed MIMIC-DOS (Discordance-Oriented Subset) from the MIMIC-IV dataset.

• Selection Criteria: It isolates ICU stay-hours where subjective indicators (Pain score = 0, RASS score indicating alert/calm) suggest stability, while objective hemodynamic indicators (Mean Arterial Pressure < 65 mmHg for >5 mins) suggest instability.
• Task: Binary classification to predict if a patient will experience worsening organ dysfunction (SOFA score increase ≥ 2) within the next 12 hours.
• Scale: A balanced benchmark of 1,000 samples (500 positive, 500 negative) derived from 912 ICU stays.

B. The Four-Stage CARE Workflow

CARE operates through a privacy-preserving pipeline involving a Local LLM (processing raw data) and a Proprietary LLM (providing abstract guidance).

1. Stage 1: Rubric Generation & Initial State Assignment

   • The Proprietary LLM generates a Rubric Schema (structured categories like "Likely Stable," "Potential Occult Shock," "Likely Worsening") based on task semantics, without seeing patient data.
   • The Local LLM maps the patient's raw data to this schema to assign an initial state.

2. Stage 2: Category-Aware Data Acquisition

   • Based on the initial state, the Local LLM determines if current evidence is sufficient.
   • It performs targeted retrieval of specific objective features (e.g., lactate, urine output) only if the current state suggests a need for cross-domain verification. This avoids exposing all data at once.

3. Stage 3: Transition Reasoning (The Core Innovation)

   • The Local LLM sends an abstract view to the Proprietary LLM: the current category, the types of features available, and the rubric structure. No raw values are sent.
   • The Proprietary LLM provides Transition Guidance: It suggests plausible state transitions (e.g., "If perfusion data is poor, escalate to 'Occult Shock'") and candidate transitions.
   • The Local LLM performs Constrained Merge: It combines the Proprietary LLM's guidance with the actual retrieved patient values to update the patient's state. This allows the system to resolve discordance by weighing new evidence against the initial state.

4. Stage 4: Final Decision-Making

   • The Local LLM synthesizes the accumulated reasoning trace (initial state, retrieved evidence, transition guidance, updated state) to make the final binary prediction (Trigger Alert vs. Do Not Trigger).

3. Key Contributions

1. Identification of Evidence Discordance: The paper highlights "sign-symptom discordance" as a specific, underexplored failure mode for LLMs in clinical AI, distinct from general reasoning errors.
2. MIMIC-DOS Benchmark: Introduction of a specialized benchmark isolating cases where subjective and objective signals conflict, providing a rigorous testbed for robust reasoning.
3. CARE Framework: A novel architecture that achieves privacy-compliant agentic reasoning. It successfully combines the abstract reasoning power of closed-source models with the data privacy of local inference, explicitly handling state transitions rather than flat prediction.

4. Experimental Results

The authors evaluated CARE against several baselines on MIMIC-DOS, including:

• Single-pass LLMs (Local and Proprietary).
• Majority Voting (Multiple local agents).
• Multi-Agent Debate (Round-synchronous and Confidence-aware).

Key Findings:

• Baseline Failure: Single-pass models and debate baselines frequently suffered from class collapse. They tended to predict the majority class (usually "No Deterioration") or showed extreme bias, failing to detect the subtle "Occult Shock" cases defined in the benchmark.
• CARE Performance: CARE achieved the best overall performance across all metrics (Balanced Accuracy, G-mean, MCC).
  • Robustness: It was the only workflow where both True Positive Rate (TPR) and True Negative Rate (TNR) exceeded 0.5 (TPR=0.522, TNR=0.570), indicating it successfully balanced sensitivity and specificity.
  • Privacy: It maintained strong performance without ever exposing raw patient values to the proprietary model.
• Ablation Study: Removing Stage 3 (Transition Reasoning) caused a significant drop in TPR, proving that the "state-transition" mechanism is critical for resolving discordance, not just a refinement step.

5. Significance and Conclusion

• Clinical Relevance: The paper demonstrates that for high-stakes medical decisions involving conflicting data, a "flat" prompt is insufficient. A structured, multi-stage reasoning process that explicitly models state transitions is necessary to avoid dangerous automation bias.
• Privacy-Utility Trade-off: CARE provides a practical solution to the "privacy vs. performance" dilemma in healthcare AI. It proves that proprietary models can guide local inference effectively without violating data sovereignty.
• Future Direction: The work suggests that future clinical AI systems must move beyond single-pass inference toward agentic workflows that can iteratively acquire evidence and reason about state changes, particularly in complex, noisy environments like the ICU.

In summary, CARE represents a significant step forward in making LLMs viable for real-world, privacy-sensitive, and high-stakes clinical scenarios where data is inherently contradictory.