📄 medicine

An Electronic Phenotype Series for Measuring Adherence to the Wake Up and Breathe Protocol in Mechanically Ventilated ICU Patients

This paper defines and validates a series of seven electronic phenotypes using real-world EHR data to enable scalable, reliable, and retrospective measurement of adherence to the Wake Up and Breathe protocol for mechanically ventilated ICU patients, demonstrating moderate to near-perfect agreement with manual chart review.

Original authors: Andrew J King, Billie S Davis, Kathryn A Connell, Leigh A Bukowski, Kanupriya Soni, Susan Graf, Jeremy M Kahn

Published 2026-06-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Andrew J King, Billie S Davis, Kathryn A Connell, Leigh A Bukowski, Kanupriya Soni, Susan Graf, Jeremy M Kahn

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the Intensive Care Unit (ICU) as a high-stakes, 24-hour relay race. The runners are patients on mechanical ventilators (machines that breathe for them), and the goal is to get them to run on their own as quickly and safely as possible.

There is a specific rulebook for this race called the "Wake Up and Breathe" protocol. It has two main steps:

Wake Up: Stop the heavy sleeping medication (sedation) to see if the patient is alert.
Breathe: Turn down the machine to see if the patient can breathe on their own.

If a hospital follows this rulebook, patients usually recover faster and spend less time in the ICU. But here's the problem: How do we know if the hospital is actually following the rules?

The Problem: The "Paperwork Maze"

In the past, checking if a hospital followed the rules was like trying to find a specific needle in a haystack made of other needles. Doctors and nurses wrote notes in different places, sometimes in handwriting, sometimes on different screens. If a researcher wanted to check 1,000 patients, they would have to read every single paper chart manually. It was slow, expensive, and prone to human error.

Also, there was a risk of "gaming the system." If a hospital knew they were being graded on how many times they tried to wake a patient up, they might just write down "We tried" without actually doing it, just to get a good score.

The Solution: The "Digital Detective"

The authors of this paper built a Digital Detective (an "Electronic Phenotype Series"). Think of this as a super-smart computer program that can read the hospital's electronic records and automatically figure out:

Is the patient eligible for the race?
Did the team actually try to wake them up?
Did they try to let them breathe on their own?
Did the patient successfully leave the machine?

Instead of a human reading 1,000 charts, this computer program scans the data in seconds. It looks for specific clues, like "Was the patient on a ventilator at 7:00 AM?" or "Was the sedation drug turned off between 7:00 AM and 11:00 AM?"

How They Tested It

To make sure their Digital Detective was accurate, the team played a game of "Spot the Difference."

They picked 200 patient days (a mix of different patients and situations).
They had two human experts read the actual paper charts and decide what happened.
They let the Computer Detective look at the same data and make its own decision.
They compared the two.

The Results:

The Good News: For the big questions (like "Is this patient eligible to start the process?"), the Computer Detective agreed with the humans almost perfectly. It was very reliable.
The Tricky Part: For the specific moments when the team tried to wake the patient up, the computer was a bit less accurate. Why? Because sometimes the doctors stopped the medicine and started it again within the same hour. The computer, which checks data in hourly chunks, missed these quick changes. It's like a security camera that only takes a photo every hour; if someone sneaks in and out in 10 minutes, the camera might miss it.

What This Means (According to the Paper)

The paper claims this Digital Detective is a reusable toolkit.

It allows hospitals to look back at their history (retrospectively) to see how well they followed the rules.
It helps them measure quality without needing a team of people to read every single chart.
It is not designed to be a real-time alarm system that tells a doctor right now to wake a patient up. It's a tool for looking at the past to improve the future.

The Limitations (What the Detective Can't Do)

The authors are honest about what their tool misses:

Missing Details: The computer can't always tell if a patient was in pain or anxious, because those feelings aren't always written down in a way the computer can read.
The "Why": It can tell you that a trial happened, but it can't always tell you why it failed or what the doctor did immediately after.
Local Rules: Every hospital writes things down differently. A hospital in Pittsburgh might use different codes than a hospital in New York, so this specific "Detective" needs to be tuned for each local team before it can be trusted.

In short, the paper presents a smart, automated way to check if hospitals are following a life-saving breathing protocol, acknowledging that while it's not perfect, it's a huge step forward from the old, slow method of reading paper charts by hand.

Technical Summary: An Electronic Phenotype Series for Measuring Adherence to the Wake Up and Breathe Protocol

Problem Statement

The "Wake Up and Breathe" protocol is an evidence-based, interprofessional strategy for Intensive Care Unit (ICU) patients receiving invasive mechanical ventilation. It integrates daily Spontaneous Awakening Trials (SATs) and Spontaneous Breathing Trials (SBTs) to facilitate earlier weaning, a practice associated with reduced mortality and shorter ventilation duration. However, measuring adherence to this complex, multistep protocol faces two significant barriers:

Informatics Complexity: The protocol relies on sequenced actions distributed across bedside monitors, clinical orders, and physiologic assessments. Progress is often response-dependent and non-deterministic. Documentation is frequently fragmented, and upstream exclusions (e.g., scheduled surgeries) are not consistently incorporated into standard data capture.
Risk of Metric Gaming: Performance measurement based on documentation rather than objective data can inadvertently alter behavior. For instance, financial incentives for SBT completion have been shown to increase the documentation of eligibility exclusions rather than improving actual patient outcomes, illustrating the risk of "gaming" when metrics rely on subjective charting.

Current methods struggle to provide scalable, retrospective, and reliable measurement of protocol adherence using real-world Electronic Health Record (EHR) data.

Methodology

The authors developed and validated a series of seven electronic phenotypes designed to measure adherence to the Wake Up and Breathe protocol. The study utilized a research data warehouse aggregating ICU data from 12 hospitals within a large integrated health system in the mid-Atlantic United States.

Phenotype Definition Strategy

Unit of Analysis: The "patient-day" was selected as the unit of analysis, reflecting the protocol's typical execution once per 24-hour period.
Sequential Logic: Definitions are structured and sequential, where criteria for one phenotype depend on satisfying a preceding one. This captures the multistep nature of the protocol.
Data Sources: Criteria were derived from clinical resources (notably Agency for Healthcare Research and Quality guidance) and mapped to available data elements in the research warehouse.
Exclusions: Patients ventilated via tracheostomy tubes were excluded to meet local quality goals. Components requiring data unavailable in the warehouse (e.g., specific documentation of agitation, respiratory distress, or cardiac arrhythmia) were excluded.
Temporal Logic: Definitions use local time stamps relative to each calendar day. They permit data that may lag the actual clinical decision time (e.g., assessing if surgery occurred later in the day) to support retrospective quality measurement rather than real-time decision support.

The Seven Phenotypes

The series comprises three categories:

Upstream Eligibility (2 phenotypes):
- @ReceivingIMV: Confirms the patient is on invasive mechanical ventilation.
- @WeanEligible: Defines the population to whom the protocol should apply (e.g., excluding new intubations, tracheostomies, comfort-care-only status, and scheduled surgeries).
Process Measures (4 phenotypes):
- @SATEligible: Criteria for initiating a Spontaneous Awakening Trial (e.g., receiving continuous sedation, no paralytics, stable vitals).
- @SATReceipt: Confirms the administration of the SAT (cessation of IV sedation).
- @SBTEligible: Criteria for initiating a Spontaneous Breathing Trial (e.g., oxygenation thresholds, FiO2/PEEP limits, hemodynamic stability).
- @SBTReceipt: Confirms the administration of the SBT (ventilator mode switch).
Downstream Outcome (1 phenotype):
- @SustainedExtubation: A more informative outcome than the protocol's final "consider extubation" step, requiring successful extubation without re-intubation or death within 48 hours.

Validation

Validation was conducted via a manual chart review of a stratified sample of 200 patient-days.

Sampling: Cases were drawn from various ICUs to ensure variation in settings and patient characteristics.
Review Process: Each case was independently reviewed by two trained abstractors, with disagreements resolved by a third reviewer.
Metric: Agreement between the electronic phenotype determination and the manual chart review was assessed using Cohen's Kappa ( $\kappa$ ).

Key Results

The validation results demonstrated varying levels of agreement between the electronic phenotypes and manual chart review:

Eligibility Phenotypes: Showed high agreement.
- @SATEligible: $\kappa = 0.82$ (95% CI: 0.73–0.92)
- @SBTEligible: $\kappa = 0.83$ (95% CI: 0.74–0.93)
- @WeanEligible: $\kappa = 0.68$ (95% CI: 0.57–0.78)
Process Phenotypes: Showed moderate agreement.
- @SATReceipt: $\kappa = 0.53$ (95% CI: 0.38–0.67)
- @SBTReceipt: $\kappa = 0.68$ (95% CI: 0.56–0.79)
Outcome Phenotype:
- @SustainedExtubation: $\kappa = 0.61$ (95% CI: 0.43–0.78)

Analysis of Discrepancies:
Lower agreement in process measures (@SATReceipt, @SBTReceipt) was primarily attributed to:

Rapid Clinical Events: If sedation was stopped and restarted within the same hour, the hourly data capture failed to register the SAT.
Documentation Granularity: When patients failed SBTs quickly, respiratory therapists often did not document the switch to an SBT ventilator mode, leading to false negatives.
Data Limitations: The inability to capture specific exclusion criteria (e.g., active seizures, alcohol withdrawal, high-frequency oscillatory ventilation) meant some patients were incorrectly classified as eligible.

Significance and Claims

The paper claims that this work demonstrates the feasibility of translating a complex, interprofessional ICU protocol into structured digital definitions for retrospective measurement. The significance of the phenotype series lies in:

Scalability: It offers a reusable framework for measuring protocol adoption across health systems using real-world EHR data.
Error Mitigation: By decoupling phenotypes (e.g., not making @SBTEligible contingent on @SATReceipt success), the series prevents error propagation, ensuring that inaccuracies in one step do not invalidate downstream analyses.
Quality Improvement: Measuring partial progression through the weaning pathway allows for the identification of systematic failures, such as missed trials due to overuse of neuromuscular blockade or coordination breakdowns.
Applicability: The definitions are specifically designed for retrospective process and outcome measurement, not real-time decision support.

The authors emphasize that while the series does not capture the full breadth of the protocol (omitting trial success/failure outcomes due to data gaps), it provides a robust method for measuring key processes and outcomes. They caution that local adaptation of value sets and validation are essential before adoption in different institutions due to variations in documentation patterns and coding standards.