Unverified Vendor Claims and Preventable Harms: A Mixed-Methods Longitudinal Independent Audit of Health AI System Performance in Nigeria

This mixed-methods longitudinal audit reveals that health AI systems deployed in Nigeria significantly underperform compared to vendor claims, resulting in substantial preventable patient harm and exacerbated inequities, thereby underscoring the urgent need for mandatory independent post-deployment verification in low- and middle-income countries.

The Gist

The Big Idea: The "Menu" vs. The "Meal"

Imagine you go to a fancy restaurant. The menu (the Vendor's Claim) promises a steak that is perfectly cooked, juicy, and 100% tender. It says, "Our chefs are the best in the world, and this steak is 95% perfect!" You pay for it, excited.

But when the waiter brings the plate (the Real-World Performance), the steak is actually cold, tough, and undercooked. It's only about 67% good.

This study is like a group of food critics who went into 73 different restaurants across Nigeria to taste the steaks. They found that while the menus promised perfection, the actual food was often disappointing, sometimes even dangerous to eat.

What Did They Do?

The researchers acted as independent "food critics" (auditors) for six different Health AI systems used in Nigerian hospitals. These systems were supposed to help doctors do things like:

• Read chest X-rays to find Tuberculosis (TB).
• Check if a pregnant woman is at risk.
• Triage patients (decide who needs help first).
• Chat with patients to understand their symptoms.

They looked at 52,000 patients and compared what the software companies said their tools could do against what the tools actually did in real Nigerian hospitals.

The Shocking Discovery: The "24-Point Gap"

The study found a massive gap between the promise and the reality.

• The Promise: The companies claimed their AI was 91.5% accurate.
• The Reality: In real life, the AI was only 67.3% accurate.

That is a 24-point drop. To use another analogy: If a GPS app claimed it could get you to your destination in 10 minutes, but it actually took you 14 minutes every time, you'd be annoyed. But if that GPS was guiding a surgeon or diagnosing a deadly disease, that "extra time" could cost lives.

Why Did This Happen? (The Three Types of "Bad Steaks")

The researchers figured out why the AI failed. They categorized the failures into three types:

1. The "Wrong Kitchen" Problem (Systematic Gaps):

   • Analogy: The AI was trained in a high-tech, air-conditioned kitchen in Europe or the US with perfect ingredients. But it was deployed in a busy, hot kitchen in a rural Nigerian village where the power flickers and the ingredients are different.
   • Result: The AI just didn't know how to handle the local environment.

2. The "One-Size-Fits-All" Problem (Context-Dependent Gaps):

   • Analogy: Imagine a shoe that fits perfectly in a city with smooth sidewalks but falls apart on a muddy, rocky path.
   • Result: The AI worked okay in big city hospitals with good internet and equipment, but it completely broke down in rural clinics with poor infrastructure.

3. The "Blind Spot" Problem (Population-Dependent Gaps):

   • Analogy: A security camera trained only on tall, light-skinned people might fail to recognize short people or people with darker skin tones.
   • Result: The AI was much worse at helping vulnerable people (the elderly, the poor, rural residents, or those with complex illnesses) because the AI had never "seen" enough people like them during its training.

The Human Cost: It's Not Just Numbers

This isn't just about statistics; it's about real people. Because the AI was less accurate than promised, the study estimated that every year, these systems caused:

• 1,247 missed cases of Tuberculosis (a deadly disease).
• 186 preventable deaths from TB.
• 342 high-risk pregnancies that were misclassified (meaning dangerous pregnancies were treated as safe, or vice versa).

It's like a smoke detector that claims to be 95% reliable but actually misses the fire 30% of the time. In a house, that's a risk. In a hospital, that's a tragedy.

The "Two-Tiered" Safety System

The paper argues that we have created a unfair world for technology:

• Rich Countries: They have strict rules. Before a new drug or AI is sold, it must be tested by independent experts (like the FDA in the US).
• Poor Countries (LMICs): They often get the "beta versions" of technology. Because they are desperate for solutions and lack strict regulators, they accept the companies' own promises without checking the work.

The study calls this a "Verification Paradox": The people who need the technology the most are the ones least likely to have it checked for safety.

The Solution: "Phase IV" for AI

The authors suggest we treat Health AI like Pharmaceutical Drugs.

• When a drug company makes a pill, they test it, sell it, and then keep watching it to make sure it doesn't have bad side effects later. This is called "Phase IV surveillance."
• Right now, AI companies sell their software and walk away.
• The Fix: We need a rule that says, "You can sell this AI, but an independent third party must check it every year to make sure it still works." If it fails, it gets pulled off the shelf.

The Bottom Line

Don't trust the brochure; check the engine.

The study concludes that we cannot just trust technology companies to tell the truth about how well their products work. In global health, especially in places with fewer resources, we must demand independent proof before we let AI make life-or-death decisions.

As the authors say: "Performance must be proven, not promised."

Technical Summary

1. Problem Statement

The rapid deployment of Artificial Intelligence (AI) in Low- and Middle-Income Countries (LMICs) is largely driven by vendor-reported performance metrics derived from controlled, retrospective, or idealized validation studies. There is a critical "verification paradox": while these systems are adopted based on high-accuracy claims, there is a lack of independent, real-world post-deployment auditing.

• The Core Issue: Vendors have a conflict of interest to report optimistic results to secure contracts, while LMICs often lack the regulatory infrastructure to verify these claims independently.
• The Risk: This reliance on unverified data creates a "two-tiered" safety system where LMIC populations are exposed to untested technologies, potentially leading to preventable clinical harms, algorithmic bias against vulnerable groups, and the exacerbation of existing health inequities.

2. Methodology

The study employed a mixed-methods longitudinal independent audit conducted between October 2024 and March 2026.

• Setting & Scope:
  • Locations: 73 public and private health facilities across six Nigerian states (Lagos, Gombe, Rivers, Enugu, Kaduna, Plateau).
  • Population: 52,000 patient interactions.
  • Systems Audited: Six distinct AI applications:
    1. Chest X-ray interpretation (Qure.ai qXR)
    2. Tuberculosis (TB) screening (Delft CAD4TB)
    3. Symptom triage
    4. Maternal health risk prediction
    5. Patient history intake
    6. Health chatbots
• Data Collection:
  • Vendor Claims: Abstracted from white papers, validation studies, and marketing materials.
  • Real-World Performance: Independently verified by Devsolve Africa Ltd (a third party with no financial ties to vendors) using system logs, patient records, and clinical outcomes.
  • Qualitative Data: 45 key informant interviews with clinicians and facility managers.
• Analytical Framework:
  • Performance Gap: Calculated as the absolute percentage-point difference between vendor claims and observed accuracy.
  • Equity Analysis: Used mixed-effects models with interaction terms to assess performance across five vulnerability dimensions (Age, Geography, Income, Clinical Complexity, Infrastructure).
  • Equity Harm Index (EHI): A metric calculated as (Harm_Vulnerable - Harm_Privileged) / Harm_Privileged × 100% to quantify disproportionate harm.
  • Harm Estimation: Clinical consequences (e.g., missed diagnoses) were estimated using patient volume, performance gaps, and bootstrap confidence intervals (n=10,000 iterations).

3. Key Contributions

• Empirical Evidence of the "Verification Gap": This is one of the first large-scale, independent audits in an LMIC setting that quantifies the discrepancy between vendor promises and real-world AI performance.
• New Classification of Performance Gaps: The study introduces a typology for AI failures:
  1. Systematic Gaps: Uniform failures across all populations due to domain shifts (e.g., training data vs. deployment environment).
  2. Context-Dependent Gaps: Failures driven by facility characteristics (e.g., infrastructure, workflow).
  3. Population-Dependent Gaps: Failures disproportionately affecting vulnerable groups due to data heterogeneity and algorithmic bias.
• Policy Framework: Proposes a "Phase IV" surveillance model for health AI, analogous to pharmaceutical post-marketing surveillance, mandating independent verification before and after deployment.

4. Key Results

A. Performance Discrepancies

• Overall Gap: Vendor-reported accuracy averaged 91.5%, while independently measured real-world accuracy was only 67.3%.
• Mean Performance Gap: 24.2 percentage points (95% CI: 21.5–26.9; p<0.001).
• System-Specific Gaps:
  • Patient History Intake: Largest gap (35 pp: 91% claimed vs. 56% observed).
  • Symptom Triage: 26 pp gap (89% vs. 63%).
  • Maternal Health: 24 pp gap (88% vs. 64%).
  • Chest X-Ray: 22 pp gap (92% AUC vs. 70% AUC).
  • Health Chatbot: 19 pp gap (94% vs. 75%).
  • TB Screening: 17 pp gap (95% vs. 78% sensitivity).

B. Clinical Consequences (Quantified Harms)

The performance gaps translated into significant preventable harm annually across the audited facilities:

• TB Screening: Estimated 1,247 missed TB cases, leading to approximately 186 preventable deaths (assuming a 15% mortality rate for missed cases).
• Maternal Health: 342 high-risk pregnancies misclassified and 156 delayed maternal referrals.
• Symptom Triage: Average emergency care delay of 4.7 hours due to intent recognition failures.

C. Equity and Bias

Performance gaps were not evenly distributed; they were significantly larger for vulnerable populations:

• Geography: Rural patients experienced a 38% higher Equity Harm Index (EHI) than urban patients (31.2 pp gap vs. 18.5 pp).
• Age: Patients >65 years had a 9.7 pp larger gap than younger patients.
• Income: The lowest income quintile had an 11.2 pp larger gap than the highest.
• Infrastructure: Low-capacity facilities saw gaps 11.6 pp larger than high-capacity facilities.

5. Significance and Implications

• Patient Safety Crisis: The study demonstrates that uncritical reliance on vendor claims in resource-constrained settings is not merely a technical oversight but a direct cause of patient mortality and morbidity.
• Regulatory Reform: It argues for a paradigm shift from "trust-based" deployment to "verification-based" deployment. Donors and governments should allocate 5–10% of AI project budgets specifically for independent post-deployment auditing.
• Global Health Equity: The findings highlight that without rigorous verification, AI risks becoming a tool that exacerbates health inequities, disproportionately harming the most vulnerable populations in LMICs.
• Future Direction: The paper calls for the establishment of mandatory, third-party verification protocols similar to the FDA's post-market surveillance for medical devices, ensuring that AI performance is proven rather than promised.

Conclusion: The study concludes that the current model of health AI deployment in LMICs is fundamentally flawed. The 24-percentage-point performance shortfall represents a systemic failure that requires immediate regulatory intervention to prevent further preventable harms.