📊 statistics

Bayesian Inference for Multi-Source Binary Data with Selection and Measurement Error

This paper introduces a Bayesian modeling framework that jointly addresses misclassification and selection errors in multi-source binary data, demonstrating through simulations and empirical analysis that while composite error can be robustly identified, accurately decomposing it into specific components requires valid indicators, auxiliary variables, and informative priors.

Original authors: Santiago Gómez-Echeverry, Arnout van Delden, Ton de Waal, Dimitris Pavlopoulos

Published 2026-07-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Santiago Gómez-Echeverry, Arnout van Delden, Ton de Waal, Dimitris Pavlopoulos

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out how many people in a giant city love spicy pizza. You have two ways to ask. The first way is a scientific survey where you knock on doors at random; this is like a "probability sample," where everyone has a fair shot of being asked. The second way is checking a list of people who signed up for a spicy pizza club; this is like "administrative data" or a "non-probability sample." The club list might have millions of names, but it's biased because only pizza lovers joined, and maybe the club's sign-up sheet has typos.

In the world of statistics, researchers often try to mix these two lists to get a better answer. But there's a catch: the club list might miss people who love spicy pizza but hate signing up (that's selection error), and the sign-up sheet might accidentally write "not spicy" for someone who actually loves it (that's measurement error). Usually, statisticians try to fix these mistakes one at a time, like trying to untangle a knot by pulling on just one end. But what if the knot is too tight? What if you need to pull both ends at once to see the whole picture? This is the puzzle a team of researchers from the Netherlands and the U.S. decided to solve. They built a new mathematical tool to untangle these mixed-up errors all at once, helping us understand how much our data is lying to us and why.

The researchers, led by Santiago Gómez-Echeverry and colleagues, created a clever "detective kit" called a Bayesian modeling framework. Think of their method as a super-smart detective who doesn't just look at the clues (the data) but also brings in a team of experts (called "priors") to help guess what the truth might be. In their case, the detective uses a special "binary latent class mixture model." Imagine a magic box that takes in messy, imperfect answers from our pizza lists and tries to guess the real number of pizza lovers hidden inside.

Here is the magic trick: The model looks at the difference between the "true" population and the "observed" sample. It realizes that the total mistake (the composite error) is made up of two parts: the measurement error (the typos on the sign-up sheet) and the selection error (the fact that only pizza fans signed up). The authors tested their detective kit using computer simulations, creating fake worlds with different levels of typos and different levels of bias. They found that their model is incredibly good at spotting the total amount of error, no matter how messy the data gets. It's like the detective can always tell you, "Hey, this list is off by exactly 15%," with high confidence.

However, the story gets a bit more complicated when the detective tries to split that 15% mistake into "how much was due to typos" and "how much was due to bias." The simulations showed that while the total error is easy to find, separating the two causes is tricky. It's like trying to figure out how much of a cake's sweetness comes from sugar versus honey when they are mixed together perfectly. The model can only do this split successfully if the detective has very good clues: reliable indicators (a clean sign-up sheet), helpful extra information (like knowing the age of the signers), and strong expert guesses (informative priors). Without these, the model can tell you the total error is huge, but it can't always say exactly which part of the error is which.

To prove this works in the real world, the team applied their model to actual data from the U.S. government, linking the Current Population Survey (CPS) with the American Time Use Survey (ATUS). They were trying to figure out how many people earn "high income" (defined as over $40,000). They treated the CPS as their reliable random survey and the ATUS as the biased "club list." Just like in their simulations, the model successfully identified the total error in the income estimates. But, once again, pinning down exactly how much of that error came from the ATUS being biased versus the data being misrecorded required careful setup and expert knowledge.

The authors conclude that their new tool is a robust way to handle multi-source data, but it's not a magic wand that solves everything automatically. It suggests that to get the most granular, detailed understanding of why our data is wrong, we need to be very careful with how we prepare our data and bring in subject-matter experts to guide the model. If we just throw the data into the box without good clues, we might know the total mess, but we won't know exactly how to clean it up. Their work doesn't claim to have solved the problem of bad data forever, but it offers a much clearer map for navigating the messy, mixed-up world of modern statistics.

Technical Summary: Bayesian Inference for Multi-Source Binary Data with Selection and Measurement Error

Problem Statement
The integration of data from multiple sources, particularly linking probability survey samples with non-probability administrative records, offers significant potential for enhancing statistical inference. However, this integration introduces complex methodological challenges. While existing literature often addresses measurement error (misclassification) and selection error (bias due to non-random inclusion) in isolation, real-world multi-source data frequently suffers from both simultaneously. This fragmentation leads to an incomplete understanding of data quality. Specifically, there is a lack of methodological frameworks capable of modeling these errors jointly when dealing with binary target variables, where the distinction between a true latent state and observed indicators is obscured by both imperfect measurement and non-random selection mechanisms.

Methodology
The authors propose a Bayesian modeling framework designed to estimate population proportions from multi-source binary data while accounting for a "composite error" that jointly captures misclassification and selection biases.

Composite Error Decomposition: The core theoretical contribution is the decomposition of the total error ( $E_C$ ) into measurement error ( $E_M$ ) and selection error ( $E_S$ ).
- Measurement Error: Modeled using a binary latent class mixture model. The framework assumes conditional independence between observed indicators given the latent true status ( $X$ ). It utilizes sensitivity ($Seq$) and specificity ($Spq$) to estimate the validity of indicators.
- Selection Error: Building on the work of Meng (2018) and Dempsey (2023), the authors adapt the decomposition of the difference between sample and population means. They introduce a "data quality" component ( $\Delta^q_{R,Y}$ ), defined as the difference in inclusion probabilities between subpopulations with $Y=1$ and $Y=0$ , replacing the less interpretable correlation-based measures.
- Integration: The model incorporates a validity adjustment factor ( $1/J_q$ , where $J_q$ is Youden's J statistic) into the selection error term. This reflects the compounding effect of imperfect measurement on selection bias. The final composite error estimator is expressed as the sum of the measurement error term and the adjusted selection error term.
Estimation Strategy:
- The model is estimated within a fully Bayesian framework using Hamiltonian Monte Carlo (HMC) via the Stan interface.
- Identification: To address the label-switching problem inherent in mixture models, the authors impose structural constraints ensuring that the probability of a positive response is higher in the latent class $X=1$ than in $X=0$ .
- Selection Mechanism: Selection into the non-probability sample is modeled as a probit function of the latent status $X$ and an auxiliary continuous variable $Z$ . The model derives a closed-form solution for the marginal inclusion probability by integrating out the auxiliary variable, assuming a normal distribution for $Z$ conditional on $X$ .
- Priors: The framework allows for flexible prior specification, ranging from weakly informative to highly informative, enabling the incorporation of subject-matter expertise regarding data quality and selection mechanisms.

Key Contributions

Joint Modeling: The paper introduces the first known framework to simultaneously model measurement and selection errors for binary target variables in a multi-source context.
Composite Error Estimation: It provides a robust estimator for the total bias ( $E_C$ ) resulting from the sum of distortions, which the model can identify accurately even when the individual components are difficult to separate.
Interpretable Selection Metric: The authors propose $\Delta^q_{R,Y}$ as a more intuitive measure of selection bias compared to correlation coefficients, representing the difference in inclusion probabilities between groups.
Decomposition Conditions: The study clarifies that while the composite error is robustly identified, the successful decomposition into distinct measurement and selection components is contingent upon the availability of valid indicators, relevant auxiliary variables, and informative priors.

Results
The performance of the proposed framework was evaluated through simulation studies and an empirical application.

Simulation Study:
- The model demonstrated robustness in estimating the composite error ( $E_C$ ) across all scenarios, regardless of the levels of measurement or selection error.
- However, the decomposition of $E_C$ into $E_M$ and $E_S$ proved sensitive to identification conditions. When sensitivity was low or the auxiliary variable provided weak separation between latent classes, the posterior distributions for the individual error components were wide, and the true values often fell outside the credible intervals.
- The results indicated a trade-off: the model could precisely estimate the total bias, but distinguishing the specific contribution of measurement versus selection error required strong data support (high validity indicators, strong auxiliary variables) or informative priors.
Empirical Application:
- The framework was applied to linked microdata from the Current Population Survey (CPS) and the American Time Use Survey (ATUS) to estimate "High Income" status.
- The study utilized four prior specifications (uninformative, selection-informed, measurement-informed, and fully-informed) to test sensitivity.
- Model comparison via Bayesian Leave-One-Out Cross-Validation (LOO-CV) was used to assess predictive performance, though the paper notes that with similar predictive accuracies, standard errors of differences can be artificially low.

Significance and Claims
The paper claims that its primary significance lies in addressing the fragmented understanding of data quality in multi-source integration. By providing a unified Bayesian framework, it allows researchers to:

Accurately estimate the total bias (composite error) affecting population proportion estimates derived from integrated data.
Understand the conditions under which this total bias can be granularly decomposed into its constituent parts.

The authors modestly conclude that while the model robustly identifies the composite error, achieving a "fully granular interpretation of bias" (separating selection from measurement) is not automatic. It necessitates detailed data curation, the inclusion of valid indicators and auxiliary variables, and the incorporation of subject-matter expertise through informative priors. The work underscores that integrating multi-source data reshapes the underlying statistical structure, requiring new inferential frameworks that explicitly account for the interplay between selection and measurement distortions.

More like this