PSEBench: A Controllable and Verifiable Benchmark for Evaluating LLMs in Patient Safety Event Triage

Technical Summary: PSEBench

Problem Statement
Patient safety event (PSE) triage involves determining whether a clinical incident is reportable under specific jurisdictional policies (e.g., Minnesota's 29 Reportable Adverse Health Events, MN29). This is a high-stakes task requiring experts to ground decisions in legal policy, identify missing critical facts, and abstain from making a binary decision when policy is ambiguous. While Large Language Models (LLMs) offer potential for automation, existing benchmarks fail to evaluate them adequately. Prior work focuses on isolated descriptive classification (e.g., event type, severity) rather than the complex, policy-grounded reasoning required for triage. Furthermore, generating reliable benchmarks for this domain is challenging due to the need for strict policy faithfulness, the requirement for multi-turn information seeking, and the necessity of handling irreducibly ambiguous cases without expensive post-hoc human annotation.

Methodology
The authors propose PSEBench, a controllable and verifiable benchmark constructed through a policy-grounded methodology centered on clause cards.

1. Clause Card Construction:

   • The core innovation is the "clause card," a structured decision specification that factorizes regulatory text into auditable variables: event type ($s$), boundary conditions ($B$), event elements ($E$), a mapping $\rho$ from conditions to elements, the intended verdict ($y$), legal basis ($G$), and instantiation constraints ($\Phi$).
   • These cards are created via an LLM-assisted, human-in-the-loop protocol. LLMs perform "loyal extraction" of policy text and "structured drafting" of candidate cards, which are then rigorously audited by human experts to ensure boundary conditions are necessary and jointly sufficient for the verdict.

2. Controllable and Verifiable Generation Pipeline:

   • Anchor-Driven Instantiation: Real-world de-identified incident reports (from the Japan Council for Quality Health Care) serve as "anchor materials" to inject clinical realism and diversity.
   • Phase I (Factual Instantiation): An LLM instantiates the event elements ($E$) from the anchor material into a structured factual record. An automated verifier checks if these facts satisfy the boundary conditions ($B$) and constraints ($\Phi$) of the clause card.
   • Phase II (Narrative Verbalization): A second LLM converts the verified facts into a free-text clinical narrative.
   • Closed-Loop Verification: An Extractor parses the generated narrative to recover the underlying facts, which are cross-checked against the original structured record by a Narrative Verifier. This ensures the narrative faithfully encodes the intended facts without "silent label drift" (hallucinations that change the verdict).

3. Benchmark Variants:

   • Complete Cases: All decision-critical facts are present.
   • Missing-Information Cases: A subset of decision-critical facts ($E'$) is withheld from the narrative, requiring the agent to proactively ask for them. The ground truth explicitly tracks which fields are missing.
   • Uncertain (Gray-Zone) Cases: All facts are known, but the policy is silent or contradictory, requiring the agent to escalate rather than force a binary verdict.

4. Agentic Evaluation Environment:

   • The evaluation simulates a multi-turn interaction between an evaluated LLM ($A_{eval}$) and an automated Information Provider ($A_{info}$).
   • $A_{info}$ acts as a stateless oracle holding the canonical facts, answering only queries aligned with known facts and declaring "unknown" otherwise.
   • $A_{eval}$ must decide when to ask questions (ASK) and when to commit to a verdict (ANSWER).

Key Contributions

• PSEBench Dataset: A dataset of 5,074 cases (3,455 complete, 1,362 missing-information, 257 uncertain) grounded in the MN29 policy. Each case includes multi-dimensional ground truth (verdict, clause, legal basis, boundary conditions, and withheld fields) derived by construction, eliminating the need for manual annotation.
• Methodological Framework: A scalable, policy-grounded construction pipeline using clause cards and closed-loop verification to generate high-fidelity, auditable benchmarks for regulatory triage.
• Agentic Evaluation: A two-role environment specifically designed to assess proactive information seeking and principled abstention under uncertainty, moving beyond static classification metrics.

Results
The authors evaluated 15 representative LLMs (closed-source frontier, open-source frontier, small models, and medical-specialty models).

• Verdict Accuracy vs. Agentic Readiness: While top-tier closed-source models (e.g., GPT-5.5, Gemini 3.1 Pro) achieved high verdict accuracy (>90%) on complete cases, significant performance gaps emerged in interactive behaviors.
• Information Seeking: Models exhibited divergent behaviors. Some (e.g., GPT-5.5) actively identified missing facts and asked targeted questions. Others (e.g., Llama-3.1-8B) asked indiscriminately, and medical-specialty models (e.g., HuatuoGPT-o1) almost never asked, defaulting to "commit-without-asking."
• Uncertainty Handling: The ability to correctly escalate ambiguous cases was the most polarized capability. Only the strongest models (GPT-5.5, Gemini 3.1 Pro) demonstrated meaningful uncertainty awareness. Most models, including medical-specialty models, exhibited an "over-reporting" bias, forcing a "Reportable" verdict on uncertain cases (e.g., MedGemma routed 98.4% of uncertain cases to Reportable).
• Evidence Grounding: High verdict accuracy often masked failures in evidence grounding; models frequently guessed the correct verdict for the wrong or hallucinated reasons, as evidenced by lower scores in evidence citation and boundary condition hit rates.

Significance and Claims
The paper claims that PSEBench provides the first comprehensive benchmark for policy-grounded PSE triage that jointly evaluates verdicts, legal evidence grounding, targeted information seeking, and uncertainty recognition. The results suggest that current LLMs, even frontier models, are not yet ready for autonomous deployment in real-world patient safety triage. The gap between static classification performance and the rigorous demands of interactive, policy-aligned reasoning—particularly in handling missing information and policy ambiguity—remains substantial. The authors position PSEBench as a scalable, verifiable testbed to guide the development of safer, more policy-aligned LLM applications for clinical safety workflows.

Limitations Acknowledged
The authors note that while anchor materials improve realism, the narratives are synthetically generated and may not fully capture the structural disorder of raw EHR notes. Additionally, the evaluation abstracts information seeking into a conversational protocol, which does not test an agent's ability to navigate live, unstructured EHR systems. Finally, the benchmark is currently grounded in the MN29 policy, though the construction methodology is claimed to be generalizable to other regulatory frameworks.