📄 medicine

AI Benefits and Machine Learning Errors - Neurology, Psychiatry and Neurosurgery

This systematic review of 79 studies (2018–2026) highlights that while AI significantly enhances neurology, psychiatry, and neurosurgery through improved diagnostics and surgical precision, it also introduces substantial risks of misdiagnosis and bias—particularly in underrepresented populations and edge cases—necessitating strict error tracking, fairness audits, and human oversight to ensure safe clinical implementation.

Original authors: Saif M. Hassan

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Saif M. Hassan

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine Artificial Intelligence (AI) in medicine as a super-powered, incredibly fast apprentice doctor. This apprentice has read every medical textbook in existence and can spot patterns in brain scans, EEGs, and patient records faster than any human. However, like any apprentice, it has a specific set of "blind spots" where it gets confused, and if we aren't careful, those mistakes can hurt patients.

This paper, written by Saif M. Hassan, is a systematic review (a giant report card) that looked at 79 different studies between 2018 and 2026. It checked how this "apprentice" performed in three specific neighborhoods of medicine: Neurology (brain and nerve disorders), Psychiatry (mental health), and Neurosurgery (brain and spine operations).

Here is the breakdown of what the paper found, using simple analogies.

1. The Superpowers (The Benefits)

The paper admits that this AI apprentice is genuinely helpful in many ways. It acts like a high-speed turbocharger for doctors.

In Neurology (The Brain & Nerves):
- The Speedster: When a patient has a stroke, time is brain. The AI can spot the blocked blood vessel on a scan in 2 minutes, whereas a human radiologist takes 15 minutes. This saves about 33% of the time before surgery, getting help to the patient faster.
- The Pattern Hunter: For epilepsy, the AI can listen to brain waves (EEG) and spot seizures with 91% accuracy, matching human experts but doing it much faster.
- The Time-Saver: For Multiple Sclerosis (MS), the AI can map out damaged areas in the brain in 2 minutes, a job that takes a human 20 minutes.
In Psychiatry (Mental Health):
- The Predictor: The AI is better at guessing if someone with depression will respond to a specific drug than a doctor's gut feeling. It can also predict the risk of suicide better than a standard clinical interview.
- The Differentiator: It helps tell the difference between schizophrenia and bipolar disorder more accurately than a standard interview alone.
In Neurosurgery (Operations):
- The Precision Tool: When surgeons are removing brain tumors, AI helps them see the edges better, allowing them to remove more of the tumor (79% success rate vs. 62% without AI).
- The Guide: When placing screws in the spine, AI acts like a GPS, reducing the number of screws placed in the wrong spot from 9% down to 4%.

2. The Glitches (The Errors)

Here is the catch: 44% of the studies reviewed found that the AI made mistakes. These weren't just tiny math errors; they were real-world failures that hurt people. The paper compares these errors to a car that drives perfectly on a sunny highway but crashes when it rains or hits a pothole.

The AI fails most often when it encounters something it hasn't seen before in its "training school."

The "Blind Spot" for Children: In pediatric epilepsy, the AI missed 22% of the brain lesions in children (compared to only 8% missed by humans). Because of this, three children had to wait a year longer for necessary surgery because the AI told the doctors their scans looked "normal."
The "Bias" Problem:
- Race: An AI used to diagnose Parkinson's disease worked great for white patients but failed completely for Black patients (specificity dropped to zero in some cases).
- Ethnicity: A depression model trained on European patients failed when tested on South Asian patients. It wrongly flagged 14% of low-risk South Asian patients as high-risk, leading to 7 people being prescribed unnecessary antidepressants that made them sick.
The "Hardware" Confusion: If an AI is trained on old MRI machines (1.5T) and then used on a new, stronger machine (3T), its accuracy can drop to near zero. It's like teaching someone to drive a sedan and then expecting them to drive a truck without a lesson.
The "Bone" Failure: In spine surgery, the AI struggled with patients who had weak, brittle bones (osteoporosis). It misidentified the path for screws in 9% of these cases, leading to two patients needing a second, revision surgery to fix nerve damage.
The "Age" Gap: A suicide prediction tool worked well for adults but failed miserably with teenagers, flagging many girls for crisis intervention that wasn't needed.

3. Why Does This Happen?

The paper explains that these errors happen because the AI is trained on narrow, "clean" data.

The "Classroom" Analogy: Imagine a student who only studies for a test using questions from one specific textbook. They will ace that test. But if you give them a question from a different book, or a question about a topic the textbook skipped (like rare diseases or specific ethnic groups), they will fail.
The AI models were mostly trained on data from specific hospitals, specific scanners, and specific populations. They haven't learned how to handle the messy, diverse reality of the real world.

4. The Verdict and Recommendations

The paper concludes that AI is a powerful tool, but we cannot trust it blindly. The benefits are real, but the risks are also real and documented.

The "Human-in-the-Loop" Rule:
The paper argues that AI should be an assistant, not the boss.

For Doctors: You need to know exactly where your AI is weak. If you are treating a child with epilepsy, a South Asian patient with depression, or an elderly patient with weak bones, you must double-check the AI's work.
For Researchers: Stop only reporting the "wins." You must publish the "losses" and the errors, especially for different groups of people.
For Regulators: Before approving these tools, they need to be tested on diverse groups, not just the "average" patient.

In short: AI is a brilliant, fast apprentice that can save lives by speeding up diagnoses and guiding surgeries. But if we don't teach it to handle the "edge cases" (children, different races, rare conditions) and if we don't keep a human doctor watching its back, it can make dangerous mistakes that delay care or cause unnecessary harm.

Technical Summary: AI Benefits and Machine Learning Errors in Neurology, Psychiatry, and Neurosurgery

Problem Statement

While Artificial Intelligence (AI) and Machine Learning (ML) offer transformative potential for analyzing complex neurological, psychiatric, and neurosurgical data, their clinical deployment is hindered by a lack of comprehensive understanding regarding system failures. Current literature often highlights performance metrics (e.g., accuracy, AUC) while underreporting specific error types, biases, and the resulting clinical harms. This gap creates a risk of misdiagnosis, delayed treatment, unnecessary interventions, and patient injury, particularly when models encounter underrepresented populations, edge cases, or out-of-distribution data. The core problem addressed is the need to systematically catalog both the proven benefits and the documented failures of AI across these three specialties to ensure safer clinical integration.

Methodology

This study is a Systematic Review conducted by Saif M. Hassan (University of Hillah).

Data Sources: PubMed, Scopus, Web of Science, and IEEE Xplore.
Search Period: January 2018 to May 2026 (search conducted June 1, 2026).
Search Strategy: Combined terms for specialties (neurology, psychiatry, neurosurgery, specific conditions like epilepsy, schizophrenia, stroke) with AI/ML terms and error-related terms (bias, false negative, out-of-distribution, calibration).
Inclusion Criteria: Original studies, systematic reviews, or meta-analyses in English that:
- Utilized AI/ML for neurological, psychiatric, or neurosurgical diseases.
- Compared AI performance against medical experts, traditional rules, or gold standards.
- Reported quantitative benefits (accuracy, time saved) or disadvantages (subgroup variations, error rates, clinical harm).
Exclusion Criteria: Editorials, case reports, animal/phantom studies, simulated-only images, and non-English articles.
Data Synthesis: Two reviewers independently extracted data on design, sample size, AI architecture, task, comparator, and error metrics. Studies were categorized into Neurology, Psychiatry, and Neurosurgery. Pooled metrics were calculated only where heterogeneity was low ( $I^2 < 50\%$ ).

Key Contributions and Results

The review included 79 studies (32 Neurology, 28 Psychiatry, 19 Neurosurgery). A critical finding is that 44% of the included studies reported specific errors or limitations, challenging the narrative of uniform AI success.

1. Neurology

Benefits:
- Stroke: AI reduced door-to-thrombectomy time by 33% and achieved 0.94 sensitivity for large vessel occlusion (vs. 0.85 for radiologists).
- Epilepsy: AI detected interictal epileptiform discharges with 91% sensitivity, matching epileptologists but reducing analysis time by 70%.
- Multiple Sclerosis (MS): Lesion segmentation achieved a Dice coefficient of 0.86 with a 90% reduction in processing time.
- Parkinson's: DaTscan interpretation reached 94% accuracy.
Errors & Harms:
- Pediatric Epilepsy: AI missed 22% of focal cortical dysplasia lesions (vs. 8% for experts), leading to delayed surgery in three documented cases.
- Scanner Generalization: Models trained on 1.5T MRI failed on 3T scanners, with Dice scores dropping from 0.85 to 0.67.
- Racial Bias: Parkinson's DaTscan models showed specificity of 0.92 for White donors but significantly lower performance for Black patients.
- Artifact Interference: Vagus nerve stimulators caused a 19% false-positive rate in seizure detection.

2. Psychiatry

Benefits:
- Schizophrenia: AI classification of first-episode psychosis using fMRI achieved an AUC of 0.89, outperforming clinical interviews (0.76).
- Depression: AI predicted antidepressant non-response (AUC 0.85) better than clinical prediction (0.68).
- Suicide Risk: Prediction from healthcare data achieved an AUC of 0.89, surpassing medical assessment (0.72).
Errors & Harms:
- Population Shift: A depression model trained on European ancestry saw AUC drop from 0.88 to 0.69 when applied to a South Asian cohort, leading to 14% of low-risk patients being flagged as high-risk. This resulted in seven patients receiving unnecessary antidepressants with side effects (nausea, somnolence, hyponatremia).
- Vendor Generalization: Schizophrenia models trained on one MRI vendor dropped in accuracy from 0.89 to 0.71 on another; 12% of healthy controls were misclassified when motion artifacts were present.
- Age Bias: Suicide prediction models performed significantly worse in adolescents (AUC 0.77) compared to adults (0.91), with a 23% false alarm rate in adolescent girls.

3. Neurosurgery

Benefits:
- Tumor Resection: Intraoperative AI guidance increased gross total resection rates from 62% to 79%.
- Spine Surgery: AI-guided pedicle screw placement reduced misplacement rates from 9% to 4%.
- Deep Brain Stimulation (DBS): Targeting errors were reduced from 2.1mm to 1.2mm, correlating with improved motor outcomes.
Errors & Harms:
- Osteoporosis: In patients with osteoporotic vertebrae (T-score < -2.5), AI incorrectly identified screw paths in 9% of cases (vs. 2% in normal bone), leading to revision surgery in two cases.
- Tumor Margins: Intraoperative AI had an 11% false-negative rate for low-grade glioma margins, resulting in residual tumor and early recurrence requiring a second surgery.
- Prior Surgery Interference: DBS targeting models failed in patients with prior stereotactic surgeries, with mean errors increasing to 2.8mm.
- Obesity: Complication prediction models underperformed for patients with BMI > 35 (AUC 0.68 vs. 0.84 for normal BMI).

Significance and Claims

The paper asserts that while AI offers "real benefits" in speed, accuracy, and surgical precision, these advantages are currently counterbalanced by significant, non-random errors that disproportionately affect specific subgroups (children, racial minorities, patients with comorbidities like osteoporosis or obesity).

Key Claims:

Error Prevalence: Errors are not rare anomalies but occur in nearly half of the reviewed studies, often manifesting as "edge case" failures or demographic biases.
Clinical Consequence: These are not theoretical errors; the review documents specific instances of delayed surgeries, unnecessary pharmacological interventions, and repeat surgeries caused by AI failures.
Root Causes: Failures stem from training on narrow, homogeneous datasets, lack of subgroup analysis in reporting, and failure to account for out-of-distribution data (e.g., different MRI scanners, rare pathologies).
Recommendations for Safety: The paper argues that AI can only be safely integrated if:
- External Validation is performed across diverse demographics and equipment.
- Real-Time Error Monitoring and uncertainty quantification are implemented.
- Human-in-the-Loop oversight is maintained for high-stakes decisions.
- Researchers and journals must prioritize reporting failures and subgroup performance metrics with the same rigor as success metrics.

The review concludes that the benefits of AI will only outweigh the risks if the field moves beyond "success-only" reporting to a culture of rigorous error tracking, fairness auditing, and continuous human supervision.