📊 statistics

A joint meta-analysis framework for the accuracy of two diagnostic tests accounting for varying study designs

This paper proposes a stable Bayesian hierarchical framework for the joint meta-analysis of two diagnostic tests that accounts for conditional dependence and accommodates varied study designs, including those lacking gold standards or joint classification data, to avoid biased accuracy estimates.

Original authors: Vera Hudak, Nicky J. Welton, Efthymia Derezea, Hayley E. Jones

Published 2026-06-29

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Vera Hudak, Nicky J. Welton, Efthymia Derezea, Hayley E. Jones

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery: How good are two different tools at spotting a specific problem?

In the world of medicine, these "tools" are diagnostic tests (like an ultrasound or a rapid swab), and the "problem" is a disease. Usually, scientists look at how well one test works by comparing it to a "Gold Standard"—the absolute best, perfect way to know if someone is sick.

But what happens when you want to know how two tests work together? Maybe doctors use them in a sequence (Test A, then Test B) or side-by-side. This is where things get tricky, and this paper introduces a new, smarter way to solve the puzzle.

The Problem: The "Silent Partner" Effect

Most old methods for combining test results make a big, risky assumption: They assume the two tests are strangers. They act as if Test A and Test B have no idea about each other. If Test A makes a mistake, Test A thinks Test B will definitely get it right, and vice versa.

The Reality: Tests are often "best friends" or "siblings." They often look at the same biological clues. If Test A gets confused by a specific type of disease, Test B is likely to get confused too. They tend to make the same mistakes at the same time.

The Analogy: Imagine two weather forecasters predicting rain.

The Old Way (Independence): You assume if Forecaster A is wrong, Forecaster B is definitely right. You combine their predictions and think you have a super-accurate forecast.
The Reality (Dependence): Both forecasters use the same barometer. If the barometer is broken, both of them will predict rain when it's sunny. They are "conditionally dependent." If you ignore this, you might think your combined forecast is 99% accurate when it's actually only 80%.

The Old Solutions vs. The New Solution

Before this paper, scientists tried to fix this "best friend" problem in two ways, both of which had flaws:

The "Fill-in-the-Blanks" Method: Some researchers would look at a study that only reported results for Test A and Test B separately, and they would guess (impute) what the combined results might have been.
- The Flaw: Guessing data is like trying to solve a jigsaw puzzle by drawing pieces that aren't there. It makes the final picture look more precise than it really is.
The "Math Trap" Method: Other methods tried to model the relationship using complex math that forced the numbers to stay within strict boundaries.
- The Flaw: The computer would often get stuck in a loop, unable to find a solution because the math was too rigid. It was like trying to fit a square peg into a round hole and expecting the hole to stretch.

The Paper's Innovation: The "Log-Odds Ratio" Compass

The authors (Hudak, Welton, Derezea, and Jones) propose a new framework that acts like a flexible compass.

Instead of guessing missing data or forcing numbers into rigid boxes, they use a mathematical tool called log-odds ratios.

Think of it this way: Instead of trying to measure the exact position of two moving cars (which is hard because they can't go outside the road), they measure the distance between the cars. This distance can be anything (positive, negative, huge, tiny) without breaking the rules of the road.
This approach allows the computer to handle the "best friend" relationship between tests naturally, without getting stuck or needing to guess missing data.

Handling the "Messy" Real World

Real-world medical studies are messy.

Some studies test everyone with both tools.
Some studies only test half the people with the second tool.
Some studies don't have a perfect "Gold Standard" and have to use a "Silver Standard" (a good, but not perfect, reference).

The old methods often threw these messy studies away or tried to force them into a perfect box. The new framework is like a universal adapter. It can plug in data from a perfect study, a messy study, or a study with a "Silver Standard" and still make sense of it all without assuming the "Silver Standard" is perfect.

What They Found (The Two Case Studies)

The authors tested their new compass on two real-world scenarios:

1. The Ultrasound Detectives (Down Syndrome Screening)

The Scenario: Two ultrasound markers (shortened humerus and shortened femur) used to screen for Down syndrome.
The Discovery: These two markers were very close friends. They made the exact same mistakes together.
The Result: When the authors ignored this friendship (using old methods), they wildly underestimated how often the tests would catch the disease if used together. They thought the combined test was very specific (rarely false alarms), but the new method showed it was actually less specific. The "independence" assumption led to a misleadingly optimistic view of the tests' combined power.

2. The Throat Swab Detectives (Strep Throat)

The Scenario: A clinical rule (McIsaac score) vs. a rapid antigen test (RADT) for Strep throat.
The Discovery: These two tests were strangers. They didn't really influence each other's mistakes.
The Result: Because they weren't "best friends," the old methods and the new method gave very similar answers. This proves the new method works well even when the tests don't depend on each other—it doesn't break the math just because the tests are independent.

The Bottom Line

This paper doesn't tell doctors to start using these tests differently tomorrow. Instead, it provides a better toolkit for scientists who review medical evidence.

No more guessing: You don't need to invent missing data.
No more math crashes: The computer won't get stuck.
Realistic results: If two tests are "best friends" (dependent), the new method catches it. If they are strangers, it handles that too.

By using this new framework, we can get a truer picture of how diagnostic tests perform when used together, ensuring that when doctors combine tests, they aren't relying on a mathematical illusion.

Technical Summary: A Joint Meta-Analysis Framework for the Accuracy of Two Diagnostic Tests Accounting for Varying Study Designs

Problem Statement
Meta-analyses of diagnostic test accuracy (DTA) typically synthesize evidence for a single test. However, clinical practice often requires evaluating multiple tests used in combination or sequence. Existing methods for jointly meta-analysing two tests frequently assume conditional independence between tests given the true disease status. This assumption is often unrealistic, particularly when tests share biological mechanisms or when disease severity influences test outcomes. Ignoring conditional dependence leads to biased estimates of joint accuracy (e.g., for "all positive" or "any positive" decision rules) and can result in invalid clinical conclusions.

Furthermore, existing models that attempt to account for conditional dependence face significant limitations:

Data Requirements: Many models (e.g., Trikalinos et al., 2014; Nikoloulopoulos, 2024) require full $2 \times 4$ joint classification tables (cross-classified results of both tests and the gold standard) from every study. Such data are often unavailable in published reports, which frequently provide only separate $2 \times 2$ tables.
Computational Instability: Models that treat joint true positive rates (TPR) and joint false positive rates (FPR) as random effects encounter boundary issues. These joint rates are mathematically constrained by the marginal sensitivities and specificities (e.g., joint TPR cannot exceed the minimum of individual TPRs). Modeling these constrained parameters as unconstrained multivariate normal random effects leads to estimation failures, divergent transitions, or invalid posterior draws where joint rates exceed marginal rates.
Study Design Heterogeneity: Real-world systematic reviews often include studies with varied designs, such as partial verification (where the gold standard is applied conditionally), imperfect reference standards, or missing data for one test in a subset of participants. Existing joint models often struggle to accommodate these designs without relying on data imputation or assuming reference standards are error-free.

Methodology
The authors propose a Bayesian hierarchical model for the joint meta-analysis of two binary diagnostic tests. The framework is designed to handle conditional dependence and varied study designs simultaneously.

Parametrization via Log-Odds Ratios: Instead of modeling joint rates or Pearson correlations as random effects, the authors model conditional dependence using study-specific log-odds ratios ( $\log(OR_{1i})$ for the diseased population and $\log(OR_{0i})$ for the disease-free population). Log-odds ratios are mathematically unconstrained, allowing them to be placed on a multivariate normal distribution without boundary issues. The joint probabilities are derived deterministically from the marginal sensitivities, specificities, and the log-odds ratios.
Hierarchical Structure: The model assumes a 6-dimensional multivariate normal distribution for the vector of logit-transformed marginal TPRs, logit-transformed marginal FPRs, and the log-odds ratios across studies. This structure allows for between-study heterogeneity and correlations between parameters.
Flexible Likelihood Construction: A key feature of the framework is its ability to accommodate diverse data structures by defining specific likelihood functions for each study based on its design and reporting:
- Full Joint Classification: Uses 4-dimensional multinomial distributions for diseased and disease-free populations.
- Separate Reporting (No Joint Data): Uses independent binomial distributions for each test, contributing no information on dependence but still informing marginal accuracy.
- Partially Paired/Missing Data: Accommodates designs where a subset of participants lacks results for one test (assuming Missing Completely at Random, MCAR) by combining multinomial and binomial likelihoods.
- Alternative Verification Designs: The framework extends to studies without a gold standard for all participants. This includes:
  - Imperfect Reference Standards: Modeling observed counts by marginalizing over the unobserved gold standard (latent class approach).
  - Conditional Verification (NMAR): Handling designs where the gold standard is applied only to those testing positive on an index test (e.g., "check the negatives" designs) by constructing likelihoods that account for the selection mechanism.
Inference: The model is implemented in Stan using Hamiltonian Monte Carlo. Vague priors are used for mean parameters, with weakly informative priors for between-study standard deviations and an LKJ prior for the correlation matrix.

Key Contributions

Computational Stability: By parametrizing dependence via log-odds ratios, the model avoids the boundary constraints inherent in joint rate parametrizations, ensuring valid probability spaces and stable estimation even when full joint data is missing.
No Data Imputation: The framework allows for the direct inclusion of studies reporting only separate $2 \times 2$ tables or partial joint data, eliminating the need for the initial data imputation steps required by previous methods (e.g., Trikalinos et al., 2014).
Symmetry: Unlike stratification approaches (e.g., Novielli et al., 2013) where one test acts as a risk stratifier for the other, this model treats both tests symmetrically, avoiding arbitrary choices in binary test scenarios.
Design Agnosticism: The framework naturally integrates studies with imperfect reference standards, partial verification, and missing data without assuming the reference standard is error-free.

Results
The framework was applied to two case studies:

Down Syndrome Screening (Ultrasound Markers): This dataset exhibited high conditional dependence (posterior median odds ratios of 21.83 for non-diseased and 61.44 for diseased). The results demonstrated that ignoring this dependence led to substantial bias: under an "all positive" rule, sensitivity was underestimated (12.3% vs. 28.3%) and specificity overestimated. Under an "any positive" rule, sensitivity was overestimated and specificity underestimated.
Group A Streptococcus Diagnosis (McIsaac Rule vs. RADT): This dataset showed weak to negligible conditional dependence (odds ratios close to 1). Consequently, the differences between the dependent model and models assuming independence were less pronounced, though shifts in point estimates for composite rules were still observed.

In both cases, the model successfully handled varied reporting structures (e.g., some studies reporting full joint tables, others only separate tables, and some with imperfect reference standards) without data imputation.

Significance and Claims
The paper claims that its primary significance lies in providing a robust, computationally stable framework for joint meta-analysis that does not rely on data imputation or the assumption of error-free reference standards. The authors emphasize that while the model retains the key features of standard DTA meta-analysis, it is essential for accurately estimating the performance of tests used in combination when conditional dependence is substantial.

The authors modestly note that the case studies serve as illustrative examples of the framework's capabilities rather than definitive guides for clinical decision-making, particularly given the age of the Down Syndrome data and the specific inclusion criteria of the original McIsaac review. They acknowledge limitations, including the complexity of extending the model to more than two tests and the need for manual coding of custom likelihoods for novel study designs, suggesting that user-friendly software development is a necessary future step for broader accessibility.