Understanding Stigmatizing Language in Clinical Documentation: A Paired Comparison of Ambient AI Drafts and Clinician Finalized Notes

Technical Summary: Understanding Stigmatizing Language in Clinical Documentation

Problem Statement
Despite the widespread adoption of Electronic Health Records (EHRs) and the recent emergence of ambient artificial intelligence (AI) documentation tools designed to reduce clinician burden, the implications of these technologies for biased language remain unclear. Stigmatizing language (SL) in clinical notes can perpetuate health disparities, reinforce provider bias, and erode patient trust. While existing literature has documented the prevalence of SL in traditional documentation and the operational efficiency gains of ambient AI, there is a critical gap in understanding whether ambient AI faithfully and neutrally transcribes patient-clinician conversations or if the subsequent clinician editing process acts as a net source of stigmatizing language entering the EHR.

Methodology
The study conducted a large-scale, paired comparison analysis of 66,297 clinical note sections from outpatient clinics at the University of California, Irvine Health. The dataset comprised pairs of text generated by two commercial ambient AI platforms (Vendor A and Vendor B) and the corresponding clinician-finalized versions.

• Data Processing: The researchers excluded sections where the finalized text was empty, resulting in a final analytic sample of 66,297 paired sections.
• Stigmatizing Language Identification: A domain-specific lexicon of 251 stigmatizing terms, compiled from prior literature, was utilized. The study employed a rule-based clinical Natural Language Processing (NLP) pipeline using SciSpaCy and NLTK. This pipeline performed case-insensitive phrase matching, handled morphological variants, and utilized text assertion and contextual classification to identify SL mentions.
• Statistical Analysis: The study measured:
  1. The prevalence of SL in AI drafts versus finalized notes.
  2. The density of SL (mentions per 200 words).
  3. The frequency of SL removal (terms present in draft but absent in final) and introduction (terms absent in draft but present in final).
  4. Paired comparisons using McNemar's test for presence/absence asymmetry and the Wilcoxon signed-rank test for density changes.

Key Results

• Prevalence Increase: The proportion of note sections containing at least one stigmatizing term increased from 21.4% in AI drafts to 24.0% in clinician-finalized notes.
• Net Introduction of Bias: Statistical analysis revealed a significant asymmetry (P < 0.001) where the introduction of SL by clinicians occurred more frequently than the removal of SL. Specifically, 5,453 sections (8.2%) had new SL terms introduced, while only 3,307 sections (23.2% of those with initial SL) had terms removed.
• Term Composition:
  • Removals: Clinicians most frequently removed terms such as "control," "believes," "unsure," and "challenging."
  • Introductions: Clinicians most frequently introduced terms such as "states," "apparently," "control," and "declined."
  • Asymmetry in Editing: While 22 of the top 30 most frequently removed and introduced terms overlapped, distinct patterns emerged. Terms exclusively removed included personality descriptors like "drunk," "unpleasant," and "skeptical." Conversely, terms exclusively introduced included behavioral judgments such as "poor historian," "unreliable," "nonadherence," and "adamant."
• Sectional Distribution: SL was most prevalent in History of Present Illness (HPI) and Assessment & Plan (A&P) sections. The increase in SL prevalence from draft to final was most pronounced in these sections, suggesting that clinical interpretation and judgment areas are the primary loci for added stigma.
• Density: While the prevalence of sections with SL increased, the density of SL within affected sections remained relatively stable (median 0.98 in drafts vs. 0.99 in final notes per 200 words), indicating that edits primarily affected whether SL appeared in a section rather than drastically increasing the concentration of terms within a single section.

Key Contributions

• Quantitative Baseline: The study provides the first large-scale, paired quantitative analysis of stigmatizing language in ambient AI drafts versus clinician-finalized notes.
• Reframing the Editing Process: The findings challenge the assumption that clinician editing serves primarily as a corrective filter for AI errors. Instead, the data suggests that the editing process itself can be a net source of stigmatizing language entering the EHR.
• Granular Term Analysis: By distinguishing between removed and introduced terms, the study highlights a shift from overt personality descriptors to more subtle, behavior-based judgments (e.g., "nonadherence") during the finalization process.

Significance and Claims
The authors claim that this study offers a "quantitative blueprint" for measuring stigmatizing language throughout the ambient AI documentation pipeline. The primary significance lies in the finding that clinician editing does not merely remove SL from AI drafts but actively introduces new stigmatizing terms, resulting in a finalized record with a higher prevalence of bias than the initial AI draft.

The paper modestly concludes that while ambient AI tools offer efficiency gains, their deployment requires careful monitoring of the human-in-the-loop editing process. The authors note that their findings are limited to a single health system and outpatient settings, and they do not stratify by patient demographics or specific AI vendors. They posit that future work must trace the provenance of SL from audio to final note and investigate whether these patterns reflect workflow norms, model behaviors, or specific clinician practices to effectively mitigate stigma in digital health records.