Artificial intelligence for detecting bipolar disorder in electronic health records of patients with affective diagnoses: a diagnostic accuracy study

This diagnostic accuracy study evaluated a natural language processing-based AI model using 500 electronic health records from a psychiatric hospital in Colombia, demonstrating its high agreement in extracting clinical domains and its potential to improve the early detection of bipolar disorder among patients with affective diagnoses.

The Gist

The Big Picture: Finding a Needle in a Haystack

Imagine a psychiatrist as a detective trying to solve a mystery. The patient is a suspect who says, "I'm depressed." But the detective suspects the real story is more complex: the patient might actually have Bipolar Disorder, a condition where moods swing between deep sadness and wild highs (mania).

The problem is that Bipolar Disorder is often hidden. When patients first walk in, they usually only talk about the "sad" part, so doctors often treat them for regular depression. This is like trying to find a specific needle in a giant haystack, but the needle is camouflaged to look exactly like the hay.

This study asked: Can a computer program (Artificial Intelligence) read the patient's medical notes faster and better than a human to spot these hidden "needles"?

The Experiment: The AI vs. The Human Detective

The researchers took 500 real medical files from a psychiatric hospital in Bogotá, Colombia. These files were written in Spanish and contained messy, unstructured notes written by doctors over time (like a long, rambling diary of a patient's life).

They used an AI tool called Arkangel AI. Think of this AI as a super-fast, hyper-focused reading machine.

• The Human Task: A team of expert psychiatrists read these files manually to determine the "ground truth" (the actual diagnosis).
• The AI Task: The AI scanned the same files to look for 18 specific "clues" (symptoms) related to mania and depression, such as "sleeping very little," "talking too fast," or "feeling invincible."

How the AI "Read" the Notes

The AI didn't just guess; it was trained to look for specific patterns.

• The Clues: It looked for 9 signs of "highs" (like irritability or racing thoughts) and 9 signs of "lows" (like sadness or loss of energy).
• The Translation: It turned the messy, human-written notes into a clean, organized checklist. For example, if a doctor wrote, "Patient says they haven't slept in 3 days but feels full of energy," the AI checked the box for "Decreased need for sleep."

The Results: Speed and Accuracy

The study compared the AI's checklist against the human experts' final verdicts.

1. The Speed Record
This is where the AI truly shined.

• Humans: It took the human reviewers about 25 minutes to read and analyze one patient's file.
• AI: The AI did the same job in 0.2 minutes (about 12 seconds).
• The Analogy: If a human is a snail, the AI is a Ferrari. The AI was 120 times faster than the humans. It processed the entire set of 100 files in the time it took a human to drink a cup of coffee.

2. The Accuracy

• Spotting the "Needle": The AI was incredibly good at finding patients who did have Bipolar Disorder. It caught 96% of the true cases (Sensitivity). It rarely missed a patient who actually had the condition.
• Avoiding False Alarms: It was also quite good at saying "No" when a patient didn't have Bipolar Disorder, getting it right 84% of the time (Specificity).
• Overall Score: If you gave the AI a report card, it would get an A+ (0.93 out of 1.0).

Where the AI Stumbled

The AI wasn't perfect at every single detail.

• The "Maybe" Zone: The AI was great at saying "Definitely Bipolar" or "Definitely Not Bipolar." However, it got a little confused in the middle ground (the "High Probability" or "Some Probability" categories).
• The Analogy: Imagine the AI is a weather forecaster. It is excellent at predicting "Sunny" or "Stormy." But when it comes to "Partly Cloudy," it sometimes struggles to decide exactly which label fits best. The human experts were better at making those subtle, nuanced calls.

The Conclusion: A Helpful Assistant, Not a Replacement

The paper concludes that this AI tool is a powerful screening assistant.

• It can read thousands of files in the time it takes a human to read a few.
• It can flag patients who might have been missed, acting like a "safety net" to catch cases of Bipolar Disorder that might otherwise be hidden in the noise of a depression diagnosis.

Important Note: The paper emphasizes that this AI is not a replacement for a doctor. It is a tool to help doctors work faster and catch more cases early. The final diagnosis still needs a human expert to interpret the complex story of a patient's life.

In short: The AI is a super-fast librarian who can instantly find the "Bipolar" books in a massive library, but a human expert still needs to read the books to understand the full story.

Technical Summary

Technical Summary: AI for Detecting Bipolar Disorder in Electronic Health Records

Problem Statement
Bipolar disorder (BD) is a severe, chronic psychiatric condition frequently underdiagnosed, particularly in patients initially presenting with depressive disorders. This diagnostic delay often leads to inappropriate treatment, increased suicide risk, and substantial socioeconomic burdens. While artificial intelligence (AI) and natural language processing (NLP) offer potential for early detection by analyzing unstructured clinical data, there is a scarcity of validated models for Spanish-language electronic health records (EHRs), especially in Latin American contexts where mental health resources are limited. Existing models often lack external validity in these populations or fail to address the specific nuances of clinical narratives in Spanish.

Methodology
This study conducted a retrospective diagnostic accuracy analysis using 500 EHRs from a psychiatric referral hospital in Bogotá, Colombia (2020–2024). The population consisted of patients aged ≥18 with baseline diagnoses in the depressive or anxiety spectrum.

• Model Architecture: The "Arkangel AI" model operates as a two-stage system.
  1. Information Extraction: Using 18 independent prompts with guided chain-of-thought (CoT) reasoning, the model extracts binary (present/absent) data for 18 predefined clinical domains (nine manic/hypomanic and nine depressive) based on DSM-5 and MINI criteria. These prompts were iteratively refined through a pilot phase involving psychiatrist review to address errors in implicit narrative interpretation.
  2. Diagnostic Inference: A rule-based system aggregates the extracted symptoms to classify patients into four risk categories: (1) Confirmed BD, (2) High probability of BD, (3) Some probability of BD, and (4) No risk.
• Validation: A subsample of 100 records served as the validation set. The index test was the AI model's classification. The reference standard was an independent clinical assessment by psychiatrists using DSM-5 and MINI criteria, with a consensus process for resolving discrepancies. Reviewers were blinded to model outputs.
• Statistical Analysis: Performance was evaluated using sensitivity, specificity, positive/negative predictive values (PPV/NPV), F1-score, and the Area Under the Receiver Operating Characteristic Curve (AUC-ROC). Agreement in symptom extraction was measured using Cohen's kappa and Gwet's AC1 to account for high prevalence effects.

Key Results

• Symptom Extraction: The model achieved a mean agreement of 91.1% with clinical reviewers across the 18 domains. High agreement was observed for specific manic symptoms like pressured speech (97.0%, κ=0.891) and irritability (96.0%, κ=0.891). Lower agreement was noted for highly prevalent or semantically broad domains such as depressed mood and insomnia, largely attributed to the prevalence paradox.
• Diagnostic Performance: In the binary screening task (any BD risk vs. no risk), the model demonstrated:
  • Sensitivity: 96.4% (95% CI: 87.7%–99.0%)
  • Specificity: 84.4% (95% CI: 71.2%–92.3%)
  • F1-score: 0.92
  • AUC-ROC: 0.932 (95% CI: 0.881–0.975)
• Risk Classification: The model showed strong performance in distinguishing extreme categories (Confirmed BD and No risk) with F1-scores of 0.902 and 0.894, respectively. Performance was weaker in intermediate categories (High and Some probability), where the model and clinicians showed greater overlap and confusion.
• Efficiency: The AI system processed the 100 validation records in approximately 20 minutes, compared to 41.3 cumulative hours for human reviewers. This represents a 99.2% reduction in analysis time, or a 120-fold increase in speed.

Significance and Claims
The authors claim that NLP-based analysis of Spanish-language EHRs can achieve clinically meaningful performance in identifying BD-related symptom patterns while drastically reducing the time required for review. The study positions the model not as an autonomous diagnostic tool, but as a clinical decision support system designed to:

1. Prioritize Patients: Identify individuals with records containing symptom constellations compatible with bipolarity who may benefit from structured interviews or specialist reassessment.
2. Reduce Diagnostic Delay: Address the significant lag in BD diagnosis, particularly in resource-constrained settings where specialist time is scarce.
3. Optimize Resources: Improve health system efficiency by triaging large volumes of unstructured data.

The paper emphasizes that the model's greatest strength lies in symptom extraction rather than categorical diagnostic integration. It acknowledges that while the model effectively identifies explicit symptom mentions, the interpretation of their temporal organization and clinical significance remains a complex task requiring human oversight. The authors conclude that such tools are most effective within a "human-in-the-loop" framework, where computational efficiency complements, rather than replaces, psychiatric judgment.

Limitations Noted
The study acknowledges limitations including its retrospective design, single-center sample (limiting generalizability), and reliance on routinely documented narratives which may contain information bias. Furthermore, the model has not been validated in other languages, regions, or health systems, and the lower performance in intermediate risk categories suggests that some error stems from the inherent subjectivity of psychiatric interpretation rather than solely from detection failures.