📊 statistics

Toward design-based inference for data integration

This paper proposes a design-based framework for integrating non-probability and probability samples by treating the former as a certainty stratum, enabling the development of consistent regression estimators that remain unbiased without relying on unverifiable missing-at-random assumptions.

Original authors: Andrius Čiginas, Ieva Burakauskaitė, Jae Kwang Kim

Published 2026-05-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Andrius Čiginas, Ieva Burakauskaitė, Jae Kwang Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to count the total number of apples in a massive orchard to report to the city council. You have two ways to get this data:

The "Volunteer" Pile: A group of people who love apples have already picked a huge pile of apples from the orchard and brought them to the gate. You know exactly how many apples are in this pile because they counted them. However, you don't know which trees they picked from. Maybe they only picked from the big, easy-to-reach trees, or maybe they only picked the reddest ones. You can't be sure if their pile represents the whole orchard.
The "Official" Survey: You have a strict, scientific plan to walk through the orchard, pick apples from specific trees using a random method, and count them. This is the gold standard for accuracy, but it's expensive and slow.

The Problem:
Traditionally, statisticians try to mix these two piles together. They assume the "Volunteer" pile is a fair representation of the whole orchard (a big assumption called "Missing at Random"). If that assumption is wrong—if the volunteers only picked red apples, for example—the final count will be wrong, and no amount of math can fix it.

The Paper's Solution: A "Two-Step" Strategy
The authors of this paper propose a clever, step-by-step way to combine these two sources without making risky guesses about the volunteers' habits.

Step 1: The "Certainty" Zone

First, you treat the "Volunteer" pile as a completed zone. You say, "Okay, we have counted every single apple in this specific group. We don't need to guess anything about them; we just accept them as a known fact."

Step 2: The "Complementary" Survey

Next, you send your official survey team out, but with a strict rule: They are forbidden from looking at the trees that are already in the Volunteer pile. They only survey the remaining trees that the volunteers missed.

Because the survey team is only looking at the "leftover" trees, and they are using a strict random method, their data is perfectly reliable. You now have two distinct, non-overlapping groups:

Group A: The known, fully counted volunteer pile.
Group B: The scientifically sampled remainder of the orchard.

The Two Ways to Combine the Data

Once you have these two groups, the paper suggests two ways to calculate the total:

The "Separate" Method: You calculate the average apple size for the volunteers and the average for the survey team separately, then add them up. This is the safest method. It works even if the volunteers were biased (e.g., if they only picked red apples) because you aren't trying to force their data to look like the survey data.
The "Combined" Method: You assume the volunteers and the survey team are basically the same type of people and mix their data together to get a single average. This is more efficient (gives a tighter estimate) only if the two groups are actually similar.

The "Pilot" Trick (Making the Survey Smarter)

Here is the clever part: Because you already have the huge "Volunteer" pile, you can use it as a pilot study to help your survey team work smarter.

Before the survey team goes out, you look at the volunteer pile to see how much the apple sizes vary. If you see that big trees have wildly different apple sizes, you can tell your survey team: "Hey, go check more trees that look like those big ones." This helps you design a better survey that gets the most accurate answer with the least amount of work. This is called optimal sampling.

Why This Matters (The Results)

The authors tested this idea with computer simulations and real data from Lithuanian government statistics (counting business sales and pharmacy drug sales).

It's Robust: Even when the "Volunteer" pile was heavily biased (picking only specific types of apples), the new method stayed accurate. Old methods that tried to "fix" the bias mathematically failed miserably in these cases.
It's Efficient: When the two groups were similar, mixing them together gave a slightly better answer. When they were very different, keeping them separate was safer.
No Magic Assumptions: The method doesn't require you to believe that the volunteers picked apples randomly. It just treats them as a known block of data and does the hard math on the rest.

The Bottom Line

Think of this paper as a new rulebook for mixing "messy, unverified data" with "clean, scientific data." Instead of trying to force the messy data to fit a perfect model (which often fails), it isolates the messy data as a known block, uses it to help design a better survey for the rest of the world, and then combines the results in a way that is mathematically guaranteed to be accurate, no matter how weird the messy data was.

Technical Summary: Toward Design-Based Inference for Data Integration

Problem Statement
The integration of non-probability samples (e.g., voluntary responses, administrative data) into finite-population inference is a growing challenge in official statistics. Traditional approaches typically rely on modeling unknown selection probabilities under a Missing-At-Random (MAR) assumption, often using inverse probability weighting (IPW) or doubly robust (DR) estimators. However, these methods face three critical limitations:

Unverifiable Assumptions: The MAR assumption and the positivity condition are difficult to verify in practice, particularly when selection depends on unobserved variables or the study outcome itself (Not-Missing-At-Random, or NMAR).
Redundancy and Efficiency: In independent sampling frameworks where probability and non-probability samples are drawn separately from the same population, overlap is unavoidable. This leads to duplicated observations and reduced estimation efficiency.
Fusion Complexity: Combining independent samples often requires a fusion tuning parameter (e.g., a weight $\alpha$ ) that is rarely chosen optimally and depends on estimated variances that are difficult to minimize in practice.

Methodology: A Sequential Design-Based Framework
The authors propose a sequential sampling framework that treats the non-probability sample ( $S_{np}$ ) as a fully observed certainty stratum. The probability sample ( $S_p$ ) is then drawn exclusively from the complementary set of units ( $U_1 = U \setminus S_{np}$ ) that were not included in the non-probability sample.

This approach eliminates the need to model the selection mechanism of $S_{np}$ entirely, as its inclusion probabilities are effectively 1. Within this framework, two Generalized Regression (GREG) estimators are developed:

Separate GREG Estimator ( $\hat{Y}_{sepDI}$ ): Fits the outcome regression model separately within the complementary stratum $U_1$ .
Combined GREG Estimator ( $\hat{Y}_{comDI}$ ): Pools both $S_{np}$ and $S_p$ to estimate a single regression coefficient, assuming homogeneity across strata.

To enhance efficiency, the non-probability sample serves as a pilot to estimate variance components. Under a working superpopulation model, these estimates are used to construct second-stage inclusion probabilities for $S_p$ that are proportional to the conditional standard deviation of the outcome (Isaki–Fuller optimality). A diagnostic test for coefficient homogeneity is proposed to guide the choice between the separate and combined estimators.

Key Contributions
The paper makes two distinct, layered theoretical contributions:

Robustness of Inference (Design Consistency):
The authors establish that both the separate and combined GREG estimators are design-consistent under standard regularity conditions on the second-stage probability design. Crucially, this consistency holds without any assumption regarding the selection mechanism of the non-probability sample. The estimators remain valid under both MAR and NMAR conditions. This robustness is structural: by treating $S_{np}$ as a certainty stratum, the unknown selection probabilities never enter the estimator, rendering the modeling of the non-probability mechanism unnecessary. Consistent plug-in variance estimators are also provided.
Asymptotic Optimality of the Design:
Under a working superpopulation model that holds in both strata (a condition strictly stronger than MAR), the pilot non-probability sample allows for the construction of second-stage inclusion probabilities that achieve Isaki–Fuller asymptotic optimality for the separate estimator. The authors emphasize that while this optimality relies on strong model assumptions, the failure of these assumptions does not invalidate the design-consistency or variance estimation results of the first contribution. Thus, the pilot-based design offers potential efficiency gains without compromising the validity of the inference.

Results
The paper validates these contributions through extensive simulation studies and two real-data applications using Lithuanian official statistics (service enterprise turnover and pharmacy drug sales).

Simulation Findings:
- Under MAR: Both sequential estimators and standard propensity-adjusted competitors (IPW, DR) perform well, though the sequential estimators generally achieve lower root mean squared error (RMSE) and avoid the redundancy of independent sampling.
- Under NMAR: The sequential estimators remain essentially unbiased with well-calibrated variance estimates and nominal 95% confidence interval coverage. In stark contrast, propensity-based competitors (IPW, DR, and fusion estimators) exhibit severe bias (exceeding 4.5% relative bias) when the selection mechanism depends on the outcome.
- Design Efficiency: The estimated optimal design (using pilot variance estimates) significantly outperforms equal-probability designs and is vastly superior to Probability Proportional to Size (PPS) designs, which showed instability and high RMSE.
Real-Data Applications:
- Turnover Data (High Heterogeneity): In an application where the administrative pilot sample over-represented large enterprises, the separate regression estimator was preferred. The combined estimator was less efficient due to strong heterogeneity between the strata.
- Pharmacy Data (Low Heterogeneity): In an application where the voluntary sample and the complement were similar, the combined estimator offered a modest efficiency gain over the separate estimator, despite the statistical test rejecting coefficient homogeneity (a common occurrence in large samples where even negligible differences are detected).

Significance and Claims
The paper claims that its sequential framework provides a robust alternative to model-dependent data integration methods. By shifting the focus from modeling unknown selection probabilities to a design-based stratified approach, the method ensures valid inference even when selection is NMAR.

The authors position the two contributions as "layered rather than coupled": the validity of the inference (design consistency) is guaranteed regardless of the selection mechanism, while the efficiency gains (Isaki–Fuller optimality) are conditional on a working model. The paper concludes that the sequential approach eliminates the need for unverifiable assumptions and tuning parameters, offering a transparent and implementable solution for integrating non-probability data into official statistics, particularly in scenarios where the non-probability source is large and the selection mechanism is unknown or complex.