End-to-End Evaluation and Governance of an EHR-Embedded AI Agent for Clinicians

1. Problem Statement

The rapid integration of Large Language Models (LLMs) into Electronic Health Records (EHRs) has shifted AI from passive query responders to active decision-makers producing structured medical records. However, current evaluation methods are insufficient for this new paradigm:

• Static vs. Dynamic: Traditional "point-in-time" evaluations cannot detect performance degradation after model updates, novel clinical scenarios, or workflow changes.
• Lack of Governance: There is no systematic framework for the continuous monitoring, iteration, and re-evaluation of deployed clinical AI.
• Attribution Difficulty: Black-box models producing free-form text make it difficult to trace errors to specific pipeline stages, hindering targeted engineering fixes.
• Quality vs. Efficiency: Prior studies focused on efficiency (time saved) rather than clinical output quality, leaving a gap in systematic quality assurance for ambient AI scribes.

2. Methodology: The Governance Framework

The authors propose and operationalize a continuous governance framework centered on the Hyperscribe agent, an EHR-embedded system that converts ambient audio into structured chart updates. The framework integrates four interconnected dimensions, gated by controlled experimentation before deployment:

A. System Architecture (Designed for Governability)

Hyperscribe is distinct from monolithic scribes; it uses a cyclical, four-stage processing pipeline with structured outputs and explicit intermediate reasoning:

1. Audio to Transcript: Speaker-attributed transcription.
2. Transcript to Instructions: Conversion of speech into discrete clinical intents (e.g., "add diagnosis").
3. Instructions to Parameters: Mapping intents to standardized codes (ICD-10, RxNorm) with minimal chart context to prevent hallucination.
4. Parameters to Commands: Executable actions updating the EHR (e.g., Diagnose, Prescribe).

• Key Design Features: Structured schemas (JSON), explicit intermediate layers for failure attribution, an EHR-bounded action space (preventing novel errors), and graduated context injection.

B. Four Governance Dimensions

1. Rubric Validation:
   • Dataset: 823 cases (736 real-world, 87 synthetic) involving 168 unique patients.
   • Process: 20 clinicians authored 1,646 validated rubrics (weighted criteria for "correct" documentation).
   • Scoring: An LLM-based scoring agent evaluates system outputs against these rubrics to generate quantitative scores (0–100%).
2. Live Clinician Feedback:
   • In-workflow feedback collection (free-text comments) during real patient encounters.
   • Thematic analysis categorizes feedback into failure modes (e.g., command generation, speaker misattribution) and positive observations.
3. Technical Performance Monitoring:
   • Real-time logging of latency, token usage, and error rates per pipeline stage.
   • Implementation of a two-layer retry mechanism (HTTP and JSON schema) to handle transient model errors.
4. Cost Tracking:
   • Granular tracking of compute costs (inference, rubric generation) and clinician time (validation hours) to ensure economic sustainability.

C. Controlled Experimentation

System changes are not deployed immediately. Instead, candidate versions are tested against the full 823-case benchmark. A version is only released if it demonstrates statistical improvement or maintains quality while reducing cost/latency.

3. Key Contributions

1. First End-to-End Governance Framework: A unified system connecting expert judgment (rubrics), live user feedback, technical monitoring, and cost tracking in a single continuous loop.
2. Operationalized Governability: Demonstrates how architectural choices (structured outputs, intermediate reasoning) enable targeted debugging and rapid iteration.
3. Empirical Evidence of Iterative Improvement: Provides data showing that feedback-driven engineering interventions directly improve clinical quality over time.
4. Cost-Benefit Analysis of Evaluation: Quantifies the trade-off between clinician-authored rubrics (high cost, high nuance) and LLM-generated rubrics (low cost, high scale), showing a 1000x cost difference.

4. Key Results

Performance Improvements

• Quality Gains: Median rubric scores improved from 84% (baseline) to 95% across seven experimental versions.
• Feedback Shift: Over three months, the composition of clinician feedback shifted from 79% error reports to 45% positive observations, indicating that engineering interventions successfully resolved major failure modes.
• Failure Resolution:
  • Command Generation: Addressed via JSON schema refactoring and decision-tree instruction detection.
  • Speaker Misattribution: Resolved by modifying prompts to distinguish patient/family/clinician and switching speech-to-text providers.
  • Granularity: Solved by enabling custom prompt versions for user-specific documentation styles.

Technical & Operational Metrics

• Latency: Median processing time of 8.1 seconds per audio segment.
• Reliability: 99.6% effective completion rate after retry mechanisms absorbed transient errors (6.6% initial error rate reduced to negligible levels).
• Cost Efficiency:
  • Clinician validation cost: ~$29.50 per rubric (919 hours total).
  • LLM-generated rubric cost: ~$0.02 per rubric.
  • Model switching and prompt minimization reduced inference costs by 20–30% without measurable quality degradation.

Case Provenance

• Clinicians could distinguish synthetic from real cases with 67% accuracy (74.9% balanced accuracy).
• Synthetic cases were identified by "unnaturally polished" language and perfect turn-taking, while real cases featured natural cognitive shifts and interruptions. This confirms synthetic data is useful for coverage but not a perfect substitute for real encounters.

5. Significance and Conclusion

This paper establishes that continuous, multi-channel governance is both achievable and effective for clinical AI.

• Shift in Paradigm: It moves the industry focus from "Can the system perform well?" to "Can the system be governed to perform well over time?"
• Scalability: The framework proves that combining automated technical monitoring with human-in-the-loop feedback creates a scalable loop for quality assurance.
• Architectural Necessity: The results suggest that clinical AI systems must be designed for governability (structured outputs, traceable reasoning) from conception to ensure safety and reliability in high-stakes medical environments.
• Economic Viability: The study demonstrates that high-quality evaluation is sustainable through a hybrid approach of expert validation for critical benchmarks and LLM automation for scale.

The authors conclude that without such a governance infrastructure, the deployment of clinical AI risks unmonitored degradation and safety failures, whereas a governed approach enables iterative, safe, and cost-effective improvement.