📊 statistics

Anchoring Convenience Survey Samples to a Baseline Census for Vaccine Coverage Monitoring in Global Health

This simulation study demonstrates that a hybrid survey approach, which anchors convenient non-probabilistic follow-up samples to a probabilistic baseline census, is a feasible and effective method for monitoring vaccine coverage in global health settings, even under conditions of moderate selection bias and varying response rates.

Original authors: Nathaniel Dyrkton, Shomoita Alam, Susan Shepherd, Ibrahim Sana, Kevin Phelan, Jay JH Park

Published 2026-03-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nathaniel Dyrkton, Shomoita Alam, Susan Shepherd, Ibrahim Sana, Kevin Phelan, Jay JH Park

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the head chef for a massive community kitchen in rural Chad and Niger. Your goal is to know exactly how many children in your region have eaten a specific, vital meal (the Measles vaccine).

The Problem: The "Gold Standard" is Too Heavy
Traditionally, to get an accurate count, you would send a team to every single house in every village to ask every parent, "Did your child eat?" This is called a probabilistic census. It's the "Gold Standard"—it's perfect, but it's incredibly expensive, slow, and exhausting. It's like trying to count every grain of sand on a beach by picking them up one by one.

The Shortcut: The "Convenience" Sample
Because resources are tight, health workers try a shortcut. Instead of visiting every house, they set up a training session in a village about measuring arm circumference (to check for malnutrition). They ask the parents who happened to show up to the training, "Did your child eat?"

This is a convenience sample. It's cheap and fast. But it has a big flaw: Selection Bias.

The Metaphor: Imagine you only ask parents who showed up to the training. Maybe the parents who showed up are the ones who are more health-conscious, have more free time, or live closer to the road. They are likely to have vaccinated their kids more than the parents who stayed home. If you just count the people at the training, you'll think everyone is vaccinated, but you'd be wrong. You've only sampled the "easy-to-reach" group.

The Solution: The "Anchor" Strategy
The authors of this paper asked: Can we use the cheap shortcut, but fix the math so it's still accurate?

They proposed a Hybrid Approach:

The Anchor (The Census): First, they did a massive, perfect count (a census) of the entire population in these villages. They knew exactly who lived there, their ages, and their vaccination status. Think of this as having a perfect map of the entire territory.
The Shortcut (The Survey): Later, they did the cheap "convenience" survey at the training sessions.
The Fix (The Calibration): They used the perfect map (the census) to "anchor" the shortcut survey. They looked at the people who showed up to the training and asked: "Do these people look like the whole population?"

If the training attendees were mostly young mothers from the city, but the census showed the village was mostly older fathers from the countryside, the math would say: "Okay, we are missing the older fathers. Let's mathematically 'stretch' our results to include them."

The Experiment: The Simulation Kitchen
Since they couldn't test this on real people without risking bad data, they built a virtual world (a simulation) in a computer.

They created a fake population of 381 villages.
They gave them fake vaccination records.
They created a "Selection Bias" monster: They made the "training attendees" slightly different from the real population (e.g., maybe 1.5 times more likely to be vaccinated just because they showed up).
They tested two different ways to fix the math:
1. Calibration Weights: Like adjusting the volume on a radio. If the signal is too quiet in one area, you turn that specific channel up until the music sounds balanced.
2. Logistic Regression (Imputation): Like a smart guesser. It looks at the people who showed up, learns the pattern (e.g., "Young moms usually vaccinate"), and then predicts what the people who didn't show up would have said.

The Results: What Did They Find?
They ran this simulation 1,000 times under different conditions (bad weather, low attendance, high bias).

The Good News: Both methods worked surprisingly well! Even when the "Selection Bias" was strong (people at the training were very different from the rest), the math could correct it.
The Secret Ingredient: The most important factor wasn't how many villages they visited, but how many people showed up to the training.
- Analogy: If you are trying to guess the flavor of a soup, it doesn't matter if you have a huge pot (many villages) if you only taste the spoonful from the top (low participation). You need a good mix of the soup in your spoon.
- When participation was high (80% of people at the training), the math fixed the bias almost perfectly.
- When participation was low (50%), the math struggled a bit, but was still better than just guessing.

The Conclusion
The paper concludes that you don't always need the expensive, perfect census every time.

If you have a "map" (a past census) and you can get a decent number of people to show up to your local event, you can use these statistical "anchors" to get a very accurate picture of vaccine coverage without breaking the bank.

In short: It's like using a GPS (the census) to correct your compass (the convenience survey). Even if your compass is slightly off because you're walking in a magnetic field, the GPS tells you exactly how to adjust your direction to get to the right destination.

1. Problem Statement

Monitoring vaccine coverage (specifically Measles-Containing Vaccine 1, or MCV1) is critical for global health but traditionally relies on probabilistic surveys, which are the gold standard but often prohibitively expensive and logistically difficult to implement in remote, resource-limited settings (e.g., rural Chad and Niger). Consequently, health organizations increasingly rely on convenience (non-probability) surveys. However, these surveys suffer from selection bias, leading to potentially invalid estimates of vaccine coverage.

The core challenge addressed in this paper is: How can non-probability convenience surveys be statistically adjusted to produce unbiased estimates of vaccine coverage when a baseline probability census exists? Specifically, the authors investigate the feasibility of correcting for non-ignorable selection bias (where the probability of being sampled depends on the outcome, even after adjusting for covariates) in a hybrid survey design.

2. Methodology

The authors conducted a simulation study based on a real-world two-stage cluster sampling design used by the Alliance for International Medical Action (ALIMA) in Ngouri, Chad, and Mirriah, Niger.

Study Design & Data Generating Mechanism (DGM)

Baseline: A full population census (probability sample) providing covariates (village demographics, distance to health centers, child/caregiver age/gender) and true vaccination status.
Follow-up: A convenience survey where caregivers attending a MUAC (malnutrition) training program are sampled.
Simulation Framework: The authors used the ADEMP framework (Aims, Data-generating mechanisms, Estimands, Methods, Performance measures).
- Outcome Model: A logistic mixed-effects model generating MCV1 status based on covariates and village-level random effects.
- Selection Model: A logistic mixed-effects model generating sample inclusion (attending training). Crucially, this model included a parameter $\xi$ (odds ratio) to induce non-ignorable selection bias, simulating scenarios where unvaccinated children are less likely to attend training.
Scenarios: 54 distinct scenarios varying:
- Selection Bias: Odds ratios (OR) from 1.0 (ignorable) to 1.5 (extreme non-ignorable).
- Village Sampling: 25%, 50%, and 75% of villages sampled.
- Participation Rate: 50%, 65%, and 80% of caregivers within sampled villages.

Statistical Adjustment Methods

Two methods were compared to estimate the finite population proportion ( $p$ ) of vaccinated children:

Calibration Weighting (Design-Based): A propensity score approach. It adjusts the initial design weights so that the weighted sample covariates match the known population totals from the baseline census. This method uses a $\chi^2$ -distance function to minimize the distance between design weights and calibrated weights.
Logistic Regression Imputation (Model-Based): A model-based prediction approach. A logistic regression model is fitted on the convenience sample to predict vaccination status. These predictions are then applied (imputed) to the entire baseline census population to estimate the total proportion.

Performance Metrics

The methods were evaluated based on:

Bias: Difference between the estimated mean and the true population proportion.
Coverage: The proportion of 95% confidence intervals containing the true parameter.
Equivalence Tolerance: The proportion of simulations where the estimate fell within a pre-specified margin (5% and 7.5%) of the true value, assessed via Two One-Sided Tests (TOST).

3. Key Results

The simulation results, based on 1,000 repetitions per scenario, yielded the following insights:

Impact of Participation Rate: The participation rate (response rate within sampled villages) was the most critical factor.
- At high participation rates (80%), both methods performed well even under high selection bias (OR 1.5), with biases <1% and coverage near 95%.
- At low participation rates (50%), performance degraded significantly. Under extreme bias (OR 1.5) and low participation, coverage dropped to ~70-80%, and bias increased to ~2%.
Impact of Village Sampling: Increasing the number of villages sampled (from 25% to 75%) did not significantly reduce bias but substantially improved coverage and equivalence rates by reducing estimator variance.
Comparison of Methods:
- Calibration Weighting: Generally showed slightly more conservative standard errors, leading to higher empirical coverage but slightly lower equivalence rates in some extreme scenarios. It is robust because it does not require individual-level covariate linkage.
- Logistic Regression: Performed similarly to calibration in terms of bias but struggled more with coverage when the number of sampled villages was low (25%) because it ignored the cluster structure in variance estimation.
Non-Ignorable Bias: Even under "worst-case" scenarios (OR 1.5, 25% villages, 50% participation), the empirical bias remained low (≤2.1%), and the proportion of estimates meeting a 7.5% equivalence margin remained above 90% in most cases.

4. Key Contributions

Hybrid Survey Validation: The study provides empirical evidence that anchoring non-probability convenience surveys to a baseline census is a feasible and robust strategy for global health monitoring, even when selection bias is non-ignorable.
Sensitivity Analysis: It moves beyond theoretical assumptions by explicitly testing the sensitivity of adjustment methods to varying degrees of non-ignorable selection bias (OR 1.0–1.5), a gap in previous literature.
Practical Guidance for Resource Allocation: The findings suggest that for convenience surveys, maximizing the participation rate within sampled units is more critical for accuracy than simply increasing the number of sampled villages or the total sample size.
Methodological Recommendation: The authors recommend Calibration Weighting as the primary analysis method for future implementations due to its simplicity, fewer assumptions (no need for individual-level data linkage), and robust performance.

5. Significance

This paper offers a practical solution to a major bottleneck in global health: the high cost of probabilistic surveys. By demonstrating that statistical adjustments can effectively correct for selection bias in convenience samples anchored to a census, the study enables health organizations (like ALIMA and WHO) to:

Monitor vaccine coverage more frequently and cost-effectively.
Make data-driven decisions in remote areas where traditional surveys are impossible.
Utilize existing census data to validate and correct rapid assessment tools.

The study concludes that while probability sampling remains the gold standard, a simulation-guided hybrid approach using calibration weighting provides a scientifically valid alternative for vaccine coverage monitoring under realistic constraints.