Development and validation of an algorithm to identify front-line clinicians using EHR audit log data

This study developed and validated a highly accurate, scalable algorithm using electronic health record audit logs to identify the primary frontline clinician for each patient day, demonstrating 91% agreement with manual chart review and enabling the analysis of care continuity patterns across a large inpatient cohort.

The Gist

Imagine a hospital as a massive, bustling orchestra. In this orchestra, hundreds of musicians (doctors, nurses, and specialists) play together to keep a patient healthy. But here's the problem: if you walk into the concert hall, it's incredibly hard to tell who is the conductor for any specific song. Is it the violinist? The drummer? The person who just walked in?

In the medical world, knowing who the "conductor" (the primary doctor in charge) is on any given day is crucial for understanding how well the team is working together. Traditionally, figuring this out was like trying to find a needle in a haystack by reading every single page of a patient's file by hand. It took forever, was boring, and couldn't be done for thousands of patients at once.

The New Solution: The "Digital Detective"

This paper is about building a smart, automated detective that uses the hospital's computer system (the Electronic Health Record, or EHR) to solve this mystery instantly.

Think of the EHR audit logs as a super-detailed security camera feed that records every single click, note, and order a doctor makes.

• The Old Way: A human investigator manually watching 24 hours of security footage to guess who was in charge.
• The New Way: A computer algorithm that scans the footage in seconds, looking for patterns to decide who was the "team captain" for that specific day.

How They Tested It

The researchers built four different versions of this "Digital Detective" and tested them against a group of real human experts who manually reviewed charts.

• The Naive Approach: They first tried a simple method that just counted who clicked the mouse the most. It was like assuming the person who talked the most in a meeting was the boss. This only got it right 78% of the time.
• The New Algorithm: Their sophisticated new detective looked at what the doctors did, not just how often they clicked. It got it right 91% of the time!

What They Found

When they ran this new detective on the entire hospital system (over 34,000 days of patient care), they discovered some interesting patterns:

1. The Hierarchy: Most days (79%), the "conductor" was the senior attending physician. Residents (doctors in training) and Advanced Practice Providers (like nurse practitioners) took the lead on about 21% of the days.
2. The Handoffs: On average, a patient only had their "captain" changed once or twice during their entire stay. This suggests the teams are actually quite stable, which is good news for patient safety.

Why This Matters

The main takeaway is that we finally have a scalable, fast, and accurate way to know who is in charge of a patient's care every single day.

Instead of spending months manually reading charts, hospitals can now use this "Digital Detective" to instantly see how care is flowing. This helps them spot gaps in communication, understand how teams work together, and ultimately, make sure the right person is always steering the ship for every patient. It turns a chaotic, invisible process into a clear, manageable picture.

Technical Summary

1. Problem Statement

In modern healthcare, interprofessional teams are essential for high-quality patient care. However, determining which specific clinician is primarily responsible for a patient on a given day (the "front-line clinician") is a significant operational challenge.

• Current Limitation: Traditional methods rely on labor-intensive manual chart reviews, which are not scalable for large datasets.
• The Gap: While Electronic Health Record (EHR) audit logs contain vast amounts of data regarding user activity, their potential to automatically identify the primary clinician per patient-day remains underutilized and unvalidated.

2. Methodology

The study employed a cross-sectional cohort design to develop and validate a new algorithmic approach.

• Data Source & Scope:

  • Setting: 12 hospitals within a single health system sharing a common EHR.
  • Population: Adult inpatient medicine encounters admitted between February 1, 2023, and April 30, 2023.
  • Inclusion Criteria: Length of stay $\ge$ 3 days; exclusion of Intensive Care Unit (ICU) admissions.
  • Total Cohort: 5,801 encounters comprising 34,001 patient-days.

• Algorithm Development:

  • Four distinct algorithm iterations were designed to parse EHR audit logs.
  • Goal: Identify the primary responsible clinician (Attending Physician, Resident Physician, or Advanced Practice Provider [APP]) for each specific patient-day.
  • Logic: The algorithms analyzed patterns of user actions within the EHR to infer responsibility, moving beyond simple frequency counts.

• Validation Strategy:

  • Gold Standard: A manual chart review of 1,401 patient-days derived from 246 randomly sampled encounters.
  • Comparison: The performance of each algorithm iteration was benchmarked against the manual review.
  • Statistical Analysis: Accuracy differences were evaluated using McNemar's test with Bonferroni-adjusted p-values to ensure statistical rigor.

3. Key Contributions

• Algorithmic Innovation: The study successfully developed a scalable, audit-log-based algorithm that automates the identification of front-line clinicians, eliminating the need for manual review in large-scale studies.
• Validation Framework: It provides a rigorous validation framework comparing algorithmic outputs against human expert review, establishing a new standard for EHR data mining in care continuity research.
• Care Continuity Metrics: The study generated empirical data on care continuity patterns, specifically quantifying handoff frequencies and the distribution of responsibility among different clinician types (Attendings, Residents, APPs).

4. Results

• Performance Accuracy:

  • The best-performing algorithm achieved 91% accuracy (1,268/1,401 patient-days) in identifying the correct primary clinician.
  • This significantly outperformed the "naive approach" (which simply counted the frequency of actions), which only achieved 78% accuracy (1,098/1,401).
  • The difference was statistically significant ($p < 0.001$).

• Error Analysis:

  • Algorithm failures were primarily attributed to:
    1. Misidentified clinician specialties.
    2. Ambiguity during days involving transitions of care.
    3. Scenarios with shared responsibilities between multiple clinicians.

• Cohort Application (Full Scale):
  When applied to the full cohort (34,001 patient-days), the algorithm identified the distribution of front-line responsibility as:

  • Attending Physicians: 79% (26,750 patient-days).
  • Advanced Practice Providers (APPs): 12% (4,145 patient-days).
  • Resident Physicians: 9% (3,106 patient-days).
  • Handoff Frequency: The median number of handoffs between front-line clinicians per encounter was 1 (Interquartile Range: 0–2).

5. Significance

This research demonstrates that EHR audit logs are a viable, high-fidelity data source for reconstructing care delivery models. By achieving excellent accuracy compared to manual chart review, the proposed algorithm enables:

• Scalability: Researchers and administrators can now analyze care patterns across thousands of patients without prohibitive labor costs.
• Quality Improvement: Accurate identification of front-line clinicians allows for better tracking of care continuity, handoff quality, and team dynamics.
• Policy & Research: The methodology supports large-scale studies on interprofessional collaboration and patient outcomes, facilitating data-driven improvements in hospital workflows.