📊 statistics

Small area estimation using incomplete auxiliary information

This paper proposes a two-step small area estimation method that integrates incomplete, non-probability auxiliary data with probability survey samples through model calibration and a Fay-Herriot framework, achieving improved domain-level precision without requiring explicit modeling of the non-probability sample's selection mechanism.

Original authors: Donatas Šlevinskas, Ieva Burakauskaitė, Andrius Čiginas

Published 2026-02-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Donatas Šlevinskas, Ieva Burakauskaitė, Andrius Čiginas

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a city planner trying to count how many people live in every neighborhood of a large city. You have a reliable, official census (a probability survey), but it's expensive and slow, so you only have a small sample of people in each neighborhood. In some tiny neighborhoods, your sample is so small that your guesses are wild and unreliable.

At the same time, you have access to massive, messy data streams (non-probability data). Maybe you have a list of people who bought coffee at a specific chain, or people who posted on a social media app. This data is huge, but it's incomplete (it only covers some people) and biased (it doesn't represent everyone equally). You can't just use it directly because it's "noisy."

This paper presents a clever two-step recipe to combine these two imperfect sources to get a perfect picture of every neighborhood, without needing to know why the messy data is missing certain people.

Here is the breakdown using simple analogies:

The Problem: The "Blind Spot"

The official census (the "Gold Standard") is accurate but has blind spots in small neighborhoods. The messy data (the "Big Data") covers the whole city but is full of errors and gaps.

The Challenge: How do you use the Big Data to fix the blind spots without trusting it blindly?

The Solution: A Two-Step "Translator and Smoother"

Step 1: The "Translator" (Model Calibration)

Imagine you have a translator who speaks both "Official Census" and "Big Data."

Find the Overlap: First, you look at the people who appear in both your official census and the Big Data list.
Learn the Connection: You ask the translator: "When the Big Data says 'Coffee Buyer,' what does that usually mean in terms of 'Official Resident'?" The translator builds a rule (a model) to predict the official number based on the messy data.
The Translation: You use this rule to translate the Big Data for everyone else in the city, even those not in the official census.
The Calibration: Now, you have a "predicted" number for the whole city. You adjust your official census weights so that your final count matches these predictions.
- The Magic: You don't need to know why the Big Data is missing people (e.g., maybe only rich people use that coffee chain). You just use the overlap to learn the relationship and correct the official numbers. This creates a much sharper, more reliable estimate for every neighborhood.

Step 2: The "Smoother" (Small Area Estimation)

Even after the translation, some tiny neighborhoods might still have a little bit of uncertainty.

The Neighborhood Party: Imagine all the neighborhoods are at a party. If one neighborhood is unsure about its population, it looks at its neighbors.
Borrowing Strength: If Neighborhood A is tiny and shaky, but Neighborhood B (right next door) is huge and stable, and they both have similar characteristics (like similar income levels or business types), Neighborhood A "borrows" some confidence from Neighborhood B.
The Result: The final numbers are smoothed out. The wild guesses for tiny areas are pulled closer to the truth by the reliable data from their neighbors.

Real-World Examples from the Paper

The authors tested this recipe on three real Lithuanian government surveys:

Business Turnover (The "Coffee Receipt" Analogy):
- Goal: Count how much money businesses make.
- Messy Data: Tax records (VAT).
- Result: The new method turned "unreliable" guesses into "good" ones for almost every business category. It was like turning a blurry photo into high-definition.
Investment in Assets (The "Zero-Inflation" Analogy):
- Goal: Count how much money companies spend on new machines.
- Messy Data: A different survey that only covers big companies.
- Challenge: Many companies spend $0. The data was full of zeros.
- Result: The method handled the zeros well, significantly improving the accuracy for the hardest-to-count industries.
Job Vacancies (The "Online Job Board" Analogy):
- Goal: Count open jobs in every town.
- Messy Data: Web-scraped online job ads.
- Result: This was the biggest win. The official survey was very shaky for small towns. By using the online job ads as a "translator," the accuracy jumped dramatically, turning "unreliable" town estimates into "good" ones.

Why is this better than the old way?

Old methods tried to use the messy data as a direct clue, but they had to make a big, risky assumption: "We assume the people missing from the messy data are just like the people who are there." (This is called the "Missing at Random" assumption). If that assumption is wrong, the whole estimate fails.

This new method says: "We don't need to guess why the data is missing. We just need to find the people who are in both lists, learn how the two lists relate, and use that to fix the official numbers." It's a more robust, "fail-safe" way to use big data.

The Bottom Line

The paper shows that by acting as a translator (Step 1) and then a neighborhood peacemaker (Step 2), statisticians can use messy, incomplete big data to make official government statistics much more accurate, especially for small areas where data is usually scarce. It's a way to turn "noisy" information into a clear signal without needing to understand the noise perfectly.

1. Problem Statement

In official statistics, probability sample surveys are the gold standard for estimating population parameters. However, for small population domains (small areas), direct design-based estimators often suffer from high variance due to small sample sizes. While auxiliary information from administrative registers or big data sources (non-probability samples) is increasingly available, it presents two major challenges:

Incompleteness: The auxiliary data often covers only a subset of the population.
Non-probability Origin: The selection mechanism of the auxiliary source is unknown and often biased, making standard calibration or propensity weighting methods (which rely on Missing-At-Random assumptions) difficult or invalid to apply directly.

Furthermore, the auxiliary variable ( $y^*$ ) is often a "noisy proxy" of the study variable ( $y$ ), differing due to definition mismatches or measurement errors. The paper addresses how to integrate such incomplete, non-probability proxy data with probability survey data to improve small area estimation without modeling the selection mechanism of the non-probability source.

2. Methodology

The authors propose a two-step framework that combines model calibration with area-level modeling.

Step 1: Model-Calibration Estimation (Unit-Level)

This step aims to produce approximately design-unbiased domain totals by integrating the non-probability data.

Setup: Let $A$ be the probability sample and $B$ be the non-probability sample. They overlap at $A \cap B$ . For units in $B$ , a proxy $y^*$ is observed; for units in $A$ , the true study variable $y$ is observed.
Measurement Error Model: A working model $y_i = g(y^*_i, x_i) + \epsilon_i$ is fitted on the overlap $A \cap B$ . This model can be linear, nonlinear, or nonparametric (e.g., k-nearest neighbors).
Prediction: The fitted model $\hat{g}$ predicts $y$ for all units in $B$ (including those not in $A$ ).
Calibration Constraints: Calibration weights $w_i$ $w_{i}$ for the probability sample $A$ $A$ are derived by minimizing the distance to design weights subject to three constraints per domain:
1. Match the population size of the "missing" stratum (units in $U$ but not in $B$ ).
2. Match the population size of the "big data" stratum (units in $B$ ).
3. Match the domain total of the predicted values $\sum \hat{y}_i$ derived from the non-probability source.
Result: This yields a Model-Calibration (MC) estimator ( $\hat{t}^{MC}_m$ ) that is approximately design-unbiased and incorporates the proxy information without assuming the non-probability sample is representative (i.e., no MAR assumption).
Variance Estimation: A design-based Rao–Wu–Yue bootstrap is used. Crucially, the prediction model $\hat{g}$ is treated as fixed during resampling to isolate design variability.

Step 2: Area-Level Modeling (Fay–Herriot)

The MC estimators and their bootstrap variances serve as inputs for a second stage to further reduce variance.

Transformation: Domain totals are log-transformed to stabilize variance and handle skewness.
Fay–Herriot (FH) Model: A mixed-effects model is fitted:
$\log(\hat{t}^{MC}_m) = z_m^\top \beta + u_m + e_m$
where $z_m$ are exactly known auxiliary covariates (error-free), $u_m$ are random area effects, and $e_m$ are sampling errors.
Prediction: Empirical Best Linear Unbiased Predictors (EBLUP) are calculated and back-transformed to the original scale with bias correction.

Benchmark Comparison

The authors compare their approach against a Ybarra–Lohr (YL) modification of the FH model. The YL approach treats the proxy total as an error-prone covariate in the FH model, requiring an Inverse Probability Weighting (IPW) step to estimate the proxy total. This benchmark relies on the Missing-At-Random (MAR) assumption, which the proposed MC method avoids.

3. Key Contributions

Novel Integration Framework: The paper develops a design-based method to integrate non-probability data with probability samples using model calibration rather than propensity weighting. This allows for valid inference without the unverifiable MAR assumption regarding the non-probability sample's selection mechanism.
Flexibility in Proxy Modeling: The framework accommodates general relationships between the proxy and the study variable (linear, hurdle models for zero-inflated data, and nonparametric kNN), rather than being restricted to linear measurement error models.
Two-Step Efficiency: It demonstrates that a unit-level calibration step effectively stabilizes domain totals, which can then be further refined by area-level modeling, outperforming direct estimation and standard FH benchmarks.
Empirical Validation: The methodology is rigorously tested on three real-world Lithuanian official statistics datasets, covering diverse data structures and proxy types.

4. Empirical Results

The study applied the method to three scenarios:

Monthly Turnover: Integrated VAT data (proxy) with survey data.
- Result: The MC estimator reduced the median Relative Root Mean Squared Error (RRMSE) by 57% compared to the direct GREG estimator. The number of "unreliable" domains (RRMSE > 0.333) dropped from 2.1 to 0.2 per month.
Annual Investment in Tangible Assets: Integrated a cut-off sample from a structural business survey.
- Result: Despite a smaller auxiliary sample ( $|B|/|A| \approx 1.1$ ) and zero-inflated data, the MC approach increased "good" domains from 37 to 53 and reduced unreliable domains from 4 to 3.
Quarterly Job Vacancies: Integrated web-scraped online job ads (non-probability) with survey data.
- Result: This showed the most dramatic improvement. The number of "good" municipalities increased from 6.2 to 30.8, and unreliable ones dropped from 10.8 to 3.0.

Comparative Findings:

MC vs. Direct: The model-calibration step provided the majority of the precision gains, significantly stabilizing the most variable domains (reducing the upper tail of RRMSE distributions).
MC vs. YL (Benchmark): The proposed MC approach consistently outperformed the Ybarra–Lohr benchmark. The YL method, which relies on MAR, performed similarly to direct estimators in turnover and vacancy cases and only showed marginal gains in the investment case.
Impact of Data Size: Gains were proportional to the relative size of the non-probability source ( $|B|/|A|$ ) and the strength of the proxy relationship.

5. Significance

This paper offers a robust solution for modern official statistics where big data and administrative records are abundant but imperfect.

Theoretical Significance: It challenges the reliance on the MAR assumption for non-probability data integration, offering a design-based alternative that remains valid even if the non-probability sample selection is unknown or biased.
Practical Significance: It provides a practical, two-step workflow that statisticians can implement to produce reliable small-area estimates using incomplete auxiliary data. The results suggest that integrating proxy data at the unit level via calibration is more effective than treating it as an error-prone covariate at the area level.
Policy Impact: By significantly reducing the number of "unreliable" small-area estimates, this method enables more accurate policy-making and resource allocation in sectors like employment, business turnover, and investment.