Assessing diagnostic accuracy of Ov16 rapid diagnostic tests for onchocerciasis using Bayesian latent class models

By applying Bayesian latent class models to pooled data from Mozambique, Ghana, and Benin, this study demonstrates that while the novel GADx Ov16 rapid test achieves the sensitivity required for stopping mass drug administration, none of the evaluated tests meet the necessary specificity thresholds to support elimination decisions without confirmatory strategies.

The Gist

Imagine you are trying to find a very shy, invisible guest (a parasite called Onchocerca volvulus) who causes river blindness. To catch them, health workers use special "sniffer dogs" (rapid diagnostic tests) that look for evidence the guest has been there, specifically a scent marker called "Ov16."

For years, the only sniffer dog available was the SD Bioline test. But there was a problem: this dog was a bit unreliable. Sometimes it missed the guest (low sensitivity), and sometimes it barked at a squirrel instead of the guest (low specificity). Also, it was picky about the food it ate; it worked best on dried blood samples but struggled with fresh blood, making field work slow and complicated.

Recently, two new sniffer dogs were trained: the GADx test and the DDTD test. The big question was: Are these new dogs better? And can they be trusted to tell health officials, "Okay, the guest is gone, we can stop the search and stop giving medicine"?

The Challenge: No Perfect "Truth"

Usually, to test a new dog, you compare it to a "perfect" dog that never makes mistakes. But in this disease, there is no perfect dog. The old "gold standard" (looking at skin under a microscope) is painful for people and often misses the guest if the infection is low.

So, the researchers couldn't just say, "New Dog A is better than Perfect Dog B." Instead, they used a clever statistical trick called a Bayesian Latent Class Model.

Think of this like a detective solving a mystery without a confession. The detective has three witnesses (the three different tests) who all saw the same scene but might have different opinions. The detective doesn't know who is telling the absolute truth, but by listening to all three and seeing where they agree or disagree, they can mathematically guess who is the most reliable witness and how often they are right.

The Investigation

The researchers gathered data from three different countries (Mozambique, Ghana, and Benin) where the parasite is found. They tested thousands of people with all three "sniffer dogs" at the same time and ran them through their detective math.

Here is what they found:

1. The "Sniffing" Ability (Sensitivity)
This measures how good the test is at finding the parasite when it is actually there. The World Health Organization (WHO) says a test needs to be at least 89% good to be trusted for stopping medicine programs.

• GADx (The New Star): This test was the best. It found the parasite about 92% of the time. It consistently passed the 89% threshold.
• SD Bioline (The Old Guard): This one was hit-or-miss. In some calculations, it passed the 89% mark, but in others, it dropped to about 83%. It wasn't reliable enough to be counted on every time.
• DDTD (The Multi-Antigen Dog): This test looked for multiple scent markers. It did okay, finding the parasite about 87% of the time, but it consistently fell just short of the 89% goal.

2. The "False Alarm" Problem (Specificity)
This measures how good the test is at not barking when there is no guest. The WHO requires a test to be 99.8% perfect here. If a test barks at a squirrel (false positive), the program might keep giving medicine to a village that doesn't need it, wasting money and effort.

• The Bad News: None of the three tests reached the 99.8% goal.
• The best performer was GADx at 98.8%, followed closely by DDTD and SD Bioline.
• While 98.8% sounds high, in a large population, that small gap means some people who don't have the parasite will still test positive.

The Conclusion

The paper concludes that while the new GADx test is a significant improvement and is very good at finding the parasite (sensitivity), none of the tests are perfect enough to be used alone to decide when to stop giving medicine.

Because they all have a slightly higher "false alarm" rate than the WHO demands, using them alone could lead to the wrong decision: either stopping medicine too early (if the test misses the parasite) or continuing medicine too long (if the test falsely alarms).

The researchers say we need either a new, even better test or a strategy that uses these tests as a first step, followed by a second check, before making the final call to stop the fight against river blindness.

Technical Summary

Technical Summary: Assessing Diagnostic Accuracy of Ov16 Rapid Diagnostic Tests for Onchocerciasis Using Bayesian Latent Class Models

Problem Statement
Onchocerciasis (river blindness) elimination programs rely heavily on the Ov16 rapid diagnostic test (RDT) to detect antibodies against the Onchocerca volvulus antigen. However, the performance of currently available Ov16 RDTs remains uncertain, particularly regarding their ability to meet the World Health Organization's (WHO) Target Product Profile (TPP) benchmarks. These benchmarks require at least 89% sensitivity for decisions to stop mass drug administration (MDA) and at least 99.8% specificity to avoid false positives that could lead to premature cessation of treatment or unnecessary continued rounds.

The existing standard, the SD Bioline Ov16 RDT, demonstrates inconsistent sensitivity when used on whole blood compared to dried blood spots (DBS) and generally fails to meet the 89% sensitivity threshold. Furthermore, the "gold standard" for diagnosis (skin snip microscopy) lacks sensitivity in low-prevalence elimination settings and is invasive. Two novel Ov16 RDTs (GADx and DDTD) have been developed, but their accuracy has not been fully evaluated using methods that account for the lack of a perfect reference standard.

Methodology
To address the absence of a true gold standard, the authors pooled data from three field studies conducted in 2023 across Mozambique, Ghana, and Benin. The dataset included 3,256 test results, with 2,743 complete observations for all three index tests.

The study employed a Bayesian Latent Class Model (LCM) to simultaneously estimate the sensitivity and specificity of three Ov16 RDTs:

1. SD Bioline Ov16 RDT (manufactured by Abbott/Standard Diagnostic), tested on DBS.
2. GADx Ov16 RDT (manufactured by Global Access Diagnostics), tested on finger-prick whole blood.
3. DDTD multi-antigen Ov16 RDT (manufactured by Drug & Diagnostics for Tropical Diseases), tested on finger-prick whole blood. This test features two lines (T1 and T2); the manufacturer defines a positive result as requiring both lines to be visible.

The analysis utilized five distinct model specifications to ensure robustness:

• Models 1–3: Evaluated the primary performance of the novel RDTs against the SD Bioline test using different prior information (manufacturer data, field estimates, or non-informative priors).
• Models 4–5: Complementary models that tested alternative definitions of a positive result for the DDTD test (e.g., positive if either line is visible, or if only the T1 line is visible) to assess if changing the positivity criteria improved performance.

Sensitivity and specificity were estimated using Markov Chain Monte Carlo (MCMC) simulations, with results reported as posterior medians and 95% credible intervals (CrI).

Key Contributions

• Application of Bayesian LCM: The study provides a rigorous evaluation of Ov16 RDTs without assuming any single test is perfect, thereby avoiding the bias inherent in using an imperfect reference standard.
• Comparative Analysis of Novel Tools: It offers the first comprehensive comparison of the GADx and DDTD RDTs against the existing SD Bioline standard across multiple epidemiological contexts.
• Evaluation of Multi-Antigen Definitions: The paper investigates whether altering the positivity criteria for the multi-antigen DDTD test impacts its diagnostic accuracy.

Results

• Sensitivity:

  • GADx RDT: Consistently demonstrated the highest sensitivity, exceeding the WHO 89% threshold for stopping MDA across all models (posterior medians: 92.0%–92.8%).
  • SD Bioline RDT: Met the 89% threshold in two of the three primary models but fell below it in one model (Model 2: 83.2%).
  • DDTD RDT: Showed lower sensitivity overall, averaging between 86.6% and 88.4%, consistently falling slightly below the 89% threshold.
  • Alternative Definitions: Changing the definition of a positive DDTD result (e.g., accepting either line as positive) did not materially improve its sensitivity or specificity.

• Specificity:

  • All three tests demonstrated high specificity, but none achieved the WHO benchmark of ≥99.8%.
  • The highest posterior median specificity was observed for the GADx test (98.8%), followed by DDTD (98.7%) and SD Bioline (98.1%).

Significance and Claims
The paper concludes that while the novel GADx RDT shows improved sensitivity relative to the existing SD Bioline test and meets the threshold for stopping MDA decisions, none of the evaluated tests achieve the specificity required to support such decisions on their own.

The authors assert that the shortfall in specificity (all tests <99.8%) poses a significant risk of false-positive results, which could lead programs to incorrectly continue mass drug administration in areas where transmission has already been interrupted. Consequently, the findings highlight that these RDTs, including the new GADx test, are not yet suitable as standalone tools for stopping MDA. The paper emphasizes the critical need for either improved diagnostic tools with higher specificity or the implementation of confirmatory testing strategies as countries progress toward the 2030 elimination goals.