CARE-Bench: Benchmarking Patient-Facing LLM Triage

Technical Summary of CARE-BENCH: Benchmarking Patient-Facing LLM Triage

Problem Definition
Patient-facing medical Large Language Models (LLMs) and agents are increasingly deployed to answer symptom queries before patients reach clinicians. In this context, the primary safety challenge is not merely diagnostic accuracy or response quality, but determining the correct next action a user should take. Current evaluation benchmarks often rely on static clinical vignettes or broad health QA tasks, which fail to capture the dynamic nature of patient-facing triage. In real-world interactions, the appropriate action (e.g., asking for clarification, recommending self-care, or escalating to emergency care) depends on the information disclosed at each turn. Premature advice can lead to under-triage (delaying needed care) or over-triage (creating unnecessary demand), while ambiguous responses can confuse users with low health literacy. Existing studies indicate that current models struggle with calibration, often under-triaging emergencies and over-triaging non-urgent cases, or providing mixed-action signals that obscure the immediate next step.

Methodology
The authors introduce CARE-BENCH, a source-grounded benchmark designed to evaluate sequential patient-facing triage as a per-turn current-action task.

• Task Formulation: The benchmark treats triage as a classification problem where the system receives a "patient-disclosure prefix" (accumulated patient text up to the current turn) and must output a response. This response is mapped to one of four action labels:
  1. INFONEEDED: Targeted clarification is required before a care level can be chosen.
  2. SELFCARE/MONITOR: Current information supports self-care, monitoring, or future-contingent safety-netting.
  3. NONURGENTCARE: Clinical evaluation is warranted without urgent language.
  4. URGENTCARE: Same-day, emergency, or delay-unsafe care is warranted.
• Dataset Construction: The dataset comprises 500 cases and 1,059 evaluated rounds reconstructed from medical dialogue, consultation, and follow-up question sources (MedDialog/OpenMed, ChatDoctor, PriMock57, and Followup-Q). An iterative workflow involving GPT-5.5 and human reviewers (including a public health PhD candidate and a medical trainee) abstracts source facts into sequential patient disclosures. Each round is labeled with the warranted current action based on the information state at that specific turn.
• Evaluation Protocol: The authors evaluated 11 models (spanning closed, open-weight general, and open healthcare models) under two protocols:
  1. Unprompted: The model receives only the patient text, simulating a general user query without specific instructions.
  2. Prompted: A minimal instruction is added to be "brief, safe, and action-oriented," without providing examples or label names.
• Scoring: Responses are mapped to the four-label space using a fixed gpt-5.5 mapper. The primary metric is macro-F1, supplemented by accuracy, ordered over/under-triage, and threshold-specific error rates (missed care, unnecessary care, urgent false negatives, and false urgent escalation).

Key Contributions

1. Operationalization of Triage: The paper operationalizes patient-facing triage as a per-turn current-action evaluation task over controlled patient-disclosure prefixes, moving beyond static vignette classification.
2. Source-Grounded Benchmark: It releases a dataset of 500 cases and 1,059 rounds with auditable source grounding, covering development, validation, and held-out test splits.
3. Empirical Evaluation: It provides a comprehensive evaluation of 11 LLMs, revealing persistent failures in action timing and calibration that are not resolved by simple prompting.

Results

• Performance: Unprompted macro-F1 scores were low, ranging from 31.2% to 50.4%. Prompting improved performance for 10 of 11 models, with macro-F1 ranging from 46.9% to 63.4%, but substantial threshold errors remained.
• Clarification Failure: A dominant error mode is the failure to recognize insufficient information. When the correct action was INFONEEDED, only 19.0% of unprompted and 33.5% of prompted outputs correctly preserved the step of asking for clarification; the majority (81.0% of unprompted and 66.5% of prompted) prematurely recommended care.
• Threshold Trade-offs: Prompting often shifts model behavior toward recommending professional care. While this reduces missed professional care (false negatives), it frequently increases unnecessary care (false positives) and over-triage. For instance, GPT-5.5 achieved a low urgent false-negative rate (9.8%) but a high unnecessary-care rate (56.9%).
• Sensitivity Analysis: Performance improved when all patient disclosures were available at once (terminal prefix), suggesting models can make better final recommendations but struggle with the timing of actions during the information-gathering phase. Trajectory exact match (correct action at every turn) remained low (22.0% unprompted, 34.9% prompted).
• Model Variability: No single model emerged as a clear leader across all safety dimensions. Open healthcare models did not uniformly outperform general models; for example, HuatuoGPT-o1 showed high urgent false-negative rates despite being a medical-tuned model.

Significance and Claims
The paper concludes that patient-facing triage is not a simple prompting problem. The persistence of errors—specifically the failure to delay care advice until sufficient information is gathered—suggests that current models lack the ability to appropriately time their actions based on evolving information states.

The authors claim that CARE-BENCH provides a controlled resource for studying whether patient-facing medical LLMs communicate care-seeking advice with appropriate timing and calibration. They explicitly state that the benchmark evaluates the current action communicated to a patient, not clinical decision-making within a care system. The results support the claim that current models make substantial threshold errors, particularly when clarification is needed before care advice. The paper emphasizes that these findings support the need for explicit evaluation of action timing before deployment, rather than relying solely on aggregate accuracy or broad health QA benchmarks. The authors caution that the benchmark does not certify clinical safety, diagnostic accuracy, or readiness for autonomous routing.