Longitudinal effects ambient AI scribe use on documentation burden and financial productivity: A quasi-experimental study

This quasi-experimental study demonstrates that the adoption of ambient AI scribes by primary care clinicians leads to gradual, persistent improvements over 150 days, including a 15% reduction in documentation time, decreased work outside scheduled hours, and a 2% increase in financial productivity, underscoring the importance of longitudinal assessment to capture the full impact of such tools.

The Gist

Imagine a doctor's day is like a busy chef in a restaurant kitchen. The chef (the doctor) spends time talking to customers (patients), taking their orders, and cooking the meal (providing care). But for every hour the chef spends cooking, they have to spend two hours at the desk writing down the order, calculating the bill, and filing paperwork. This paperwork is the "documentation burden" that is causing many chefs to burn out, stay late, and feel like they aren't actually cooking enough meals.

To help, the restaurant introduced a new tool: a smart sous-chef (an Ambient AI Scribe). This AI listens to the conversation between the chef and the customer and automatically writes the order ticket for the chef to just sign off on.

This study is like a 5-month reality show where the researchers watched 220 doctors (the chefs) to see how this smart sous-chef actually changed their lives over time. They didn't just check in once; they watched them every day for 150 days to see how the doctors learned to use the new tool.

Here is what happened, day by day:

1. The First Day: A Quick Win, But Not a Miracle

When the doctors first got the AI scribe (Day 0), it was like handing a new tool to a chef who has never used it before.

• The Result: They immediately saved about 7% of their paperwork time.
• The Catch: It wasn't a huge transformation yet. The doctors were still figuring out how to talk to the AI, how to edit its notes, and how to fit it into their rhythm. It was like learning to drive a car with automatic transmission; you save gas, but you're still getting used to the pedals.

2. The Long Haul: The "Learning Curve" Effect

The most important finding of this study is that the benefits grew over time.

• By Day 150 (5 months later): The time saved on paperwork had doubled to 15%.
• The Analogy: Think of it like learning to play a new video game. On day one, you are clumsy and lose a lot of lives. But by month five, you know the controls, you know the shortcuts, and you are playing at a much higher level. The doctors didn't just use the tool; they adapted to it. They learned how to let the AI do the heavy lifting so they could focus on the patient.

3. The "Pajama Time" Factor

Doctors often feel guilty about working late at home, finishing notes after the clinic closes. The researchers called this "Pajama Time" (working in pajamas).

• The Result: At the start, the AI didn't really help with late-night work. But by Day 150, doctors were spending 18% less time in their pajamas writing notes.
• The Metaphor: It's like the difference between carrying a heavy backpack up a hill every night versus having a friend carry it for you. At first, you still carry it because you're used to it. But eventually, you trust your friend, and you finally get to go home and relax in your pajamas.

4. The "Note Length" Twist

Interestingly, when the doctors first started using the AI, their notes got longer (by about 5%).

• Why? Because the AI was so good at capturing details, the doctors felt they could write more. It was like having a super-accurate recorder; you don't have to worry about forgetting a detail, so you write everything down.
• The Shift: Over time, this effect faded. By Day 150, the notes were back to their normal length. The doctors had learned to edit the AI's "stream of consciousness" down to just what was necessary, finding the perfect balance between detail and brevity.

5. The Money Question (Productivity)

Did saving time mean the doctors could see more patients and make more money?

• The Result: Yes, but only a little bit. By Day 150, there was a 2% increase in the work they billed.
• The Reality Check: It wasn't a gold rush. The main win wasn't getting rich; it was getting their time back. The study suggests that while the AI helps doctors work slightly more efficiently, its biggest gift is giving them their evenings and weekends back.

The Big Takeaway

This study teaches us that new technology isn't a magic wand you wave once. It's more like training a new puppy.

• Day 1: The puppy is cute but clumsy. You save a little time, but you're also tired from training it.
• Day 150: The puppy is well-trained, knows the rules, and is actually helping you run the house.

The doctors didn't just "use" the AI; they evolved with it. The study proves that if you give doctors a smart tool and wait long enough, they will learn to use it in a way that genuinely reduces their stress and gets them home to their families sooner.

In short: The AI scribe didn't fix everything overnight, but over 5 months, it turned a heavy burden into a manageable task, giving doctors back about an hour of their week and, more importantly, their peace of mind.

Technical Summary

1. Problem Statement

The increasing burden of Electronic Health Record (EHR) documentation is a primary driver of clinician burnout, with research suggesting that for every hour of patient-facing time, clinicians spend approximately two hours on documentation. While Ambient Artificial Intelligence (AI) scribes have been proposed as a solution to automate note generation, existing literature suffers from significant methodological limitations:

• Measurement Granularity: Most studies rely on aggregated, clinician-level metrics provided by vendors (e.g., Epic Signal) rather than patient-encounter-level data, failing to control for covariates like visit type or productivity.
• Temporal Design: Prior research predominantly uses static "pre-post" designs (two time points), which fail to capture the evolution of user adaptation. Clinicians' workflows and tool usage patterns change over time, and single-point measurements may obscure the true trajectory of impact.
• Contradictory Findings: Previous studies report conflicting results regarding time savings and productivity gains, ranging from significant reductions in documentation time to marginal or non-existent benefits.

2. Methodology

The study employs a longitudinal, quasi-experimental interrupted time series (ITS) design to evaluate the impact of an ambient AI scribe (Abridge Inc.) on primary care clinicians.

• Study Setting & Population:

  • Setting: 36 primary care clinics within a large Midwestern healthcare system (WashU Medicine/BJC Health System).
  • Participants: 220 primary care clinicians (56.4% physicians, 43.6% advanced practice providers) with a mean age of 43.7 years.
  • Data Scope: 314,845 patient encounters occurring between January 5, 2024, and October 31, 2025.
  • Timeline: 48 weeks per clinician (24 weeks pre-implementation, 24 weeks post-implementation), centered on the clinician's first day of AI access ("Day Zero").

• Data Source:

  • Raw data extracted from the Epic EHR Clarity database (User Action Log Lite tables).
  • Granularity: Patient-encounter level, capturing specific activities (keystrokes, mouse clicks, scrolling) rather than aggregated summaries.

• Outcomes Measured:

  1. Documentation Time: Note writing time, Time Outside Scheduled Hours (TOSH), and "Pajama Time" (work performed between 5:30 PM and 7:00 AM).
  2. Note Patterns: Note length (character count) and note closure rate (signed within 24 hours).
  3. Financial Productivity: Work Relative Value Units (wRVUs).

• Statistical Analysis:

  • Model: A Bayesian non-linear interrupted time series model with a latent autoregressive term.
  • Mechanism: The model uses the pre-period to tune parameters for typical serial variability. In the post-period, it generates a counterfactual (what would have happened without the AI) and models non-linear deviations using a Generalized Additive Model (GAM) specification.
  • Metrics: Effects are reported as Incidence Rate Ratios (IRR) and predicted changes at specific intervals (Day 0, 30, 60, 90, 120, 150).
  • Uncertainty: Reported via 95% credible intervals (CIs) derived from Markov Chain Monte Carlo (MCMC) simulations.

3. Key Contributions

• Longitudinal Perspective: This is one of the first studies to move beyond static pre-post comparisons, revealing that the impact of AI scribes is gradual and persistent, evolving significantly over a 150-day period.
• Encounter-Level Granularity: By utilizing raw EHR activity logs at the encounter level, the study controls for confounding variables (e.g., visit type, clinician role) that aggregated vendor metrics miss.
• Adaptation Dynamics: The study quantifies how clinician behavior shifts over time, distinguishing between immediate effects (Day 0) and stabilized workflow integration (Day 150).
• Nuanced Outcome Analysis: It uncovers counter-intuitive findings, such as the initial increase in note length and closure speed, which later attenuate as clinicians adapt their workflows.

4. Key Results

The study analyzed changes from Day 0 (immediate implementation) to Day 150 (long-term adaptation):

• Documentation Time Savings:

  • Immediate (Day 0): 7% reduction in note writing time (IRR 0.93).
  • Long-term (Day 150): 15% reduction (IRR 0.85).
  • Magnitude: This translates to approximately 68 minutes of weekly note-writing time saved per clinician by Day 150.

• Work-Life Balance (Pajama Time & TOSH):

  • Day 0: No significant change in off-hours work.
  • Day 150: Significant reductions observed.
    • Pajama Time: 18% decrease (saving ~34.4 mins/week).
    • TOSH: 13% decrease (saving ~15.6 mins/week).

• Note Patterns (Length & Closure):

  • Note Length: Initially increased by 5% at Day 0, but attenuated to pre-period levels by Day 150, suggesting clinicians learned to edit AI drafts more efficiently over time.
  • Note Closure (<24h): Initially 31% higher at Day 0, but returned to pre-period levels by Day 150. The authors suggest this indicates a behavioral shift: clinicians initially rushed to close notes but later utilized the AI transcript to recall details, allowing them to delay note completion without cognitive penalty.

• Financial Productivity (wRVUs):

  • Day 0: No significant change.
  • Day 150: A 2% increase in total wRVUs (IRR 1.02).
  • Stratification: The gain was driven primarily by physicians (+3% wRVU), while Advanced Practice Providers (APPs) showed no significant gain. This suggests potential increases in billing intensity or risk adjustment.

5. Significance and Implications

• Adaptation is Critical: The study demonstrates that the benefits of ambient AI are not immediate. The "learning curve" involves a period of behavioral adjustment where initial gains in speed or note length may fluctuate before stabilizing into a more efficient workflow.
• Beyond Time Savings: While time savings are the most direct benefit, the reduction in "pajama time" is a crucial indicator of improved clinician well-being and burnout mitigation.
• Methodological Rigor: The findings challenge previous conclusions that AI scribes are not "productivity tools." The study argues that previous negative findings may stem from measurement limitations (aggregated data) and insufficient time horizons to capture adaptation.
• Clinical Workflow Integration: The attenuation of note length and closure rates suggests that clinicians are successfully integrating AI into their cognitive workflow, using the transcript as a memory aid rather than just a dictation tool, thereby reducing the cognitive load of recall.

Conclusion: Ambient AI scribes offer persistent, longitudinal benefits in reducing documentation burden and off-hours work, but these benefits evolve over time. Stakeholders should anticipate a gradual adaptation period rather than expecting immediate, static productivity gains.