Doubly robust integration of nonprobability and probability survey data

Technical Summary: Doubly Robust Integration of Nonprobability and Probability Survey Data

Problem Statement
Probability surveys remain the gold standard for estimating population means or prevalences, but they face increasing challenges regarding cost and feasibility. Conversely, nonprobability surveys offer accessible data but suffer from undefined sampling frames and potential selection bias. A prominent approach to integrating these data sources involves using outcome data from a nonprobability sample (Sample B) and covariate data from a probability sample (Sample A).

While Doubly Robust (DR) estimators have been established for this integration—combining Inverse Probability Weighting (IPW) and Mass Imputation (MI)—a specific gap remains: how to optimally combine these DR estimators with a standard Horvitz-Thompson (HT) or Hajek estimator derived solely from the probability sample's outcome data. Previous work, such as Gao and Yang (2023), addressed this combination but relied on the assumption that both the selection and outcome models are correctly specified and focused primarily on specific DR estimator formulations. Furthermore, existing methods often lacked generalization for domain (subpopulation) estimation and robust variance estimation when models are misspecified.

Methodology
The authors propose a framework for estimating the mean of an outcome $Y$ in a specific domain (subpopulation) by integrating data from a probability sample (Sample A) and a nonprobability sample (Sample B).

1. Generalization of DR Estimators:
   The paper extends previously proposed DR estimators (specifically $\hat{\mu}_{DR1}$ and $\hat{\mu}_{DR2}$ by Chen et al., 2020) to handle domain estimation.

   • $\hat{\mu}_{DR1}$ and $\hat{\mu}_{DR2}$: These estimators utilize outcome data only from Sample B. The authors introduce a Hajek-style variant ($\hat{\mu}_{KH2}$) that does not require the domain size ($N_s$) to be known a priori.
   • Nuisance Models: The method employs a selection model (for the probability of inclusion in Sample B) and an outcome model (for the conditional expectation of $Y$ given covariates). The estimators are "doubly robust," meaning they remain consistent if either the selection model or the outcome model is correctly specified.
   • Variance Estimation: The authors derive asymptotic variance formulae for these estimators under both design-based and model-design-based inference. Crucially, they provide variance estimators that remain valid (double robust) even if one of the nuisance models is misspecified.

2. Efficient Combination of Estimators:
   The core contribution is the development of a combined estimator that weights a standard HT or Hajek estimator (using only Sample A outcomes) and a DR estimator (using Sample B outcomes and Sample A covariates).

   • Weighting Scheme: The optimal weight $w$ is derived to minimize the variance of the linear combination $(1-w)\hat{\mu}_H + w\hat{\mu}_{DR}$. This weight depends on the variances of the individual estimators ($V_H, V_D$) and their covariance ($C$).
   • Covariance Derivation: The paper explicitly derives the covariance between the HT/Hajek estimator and the DR estimator, correcting the assumption in some prior literature that this covariance is zero. The authors show that because both estimators utilize the same probability sample covariate data, they are generally not independent.
   • Robustness: The proposed combination allows for the use of IPW estimators (a special case of DR with a misspecified outcome model) and provides variance formulae that do not require both nuisance models to be correctly specified.

3. Asymptotic Relative Efficiency (ARE):
   The authors derive the ARE of the combined estimator compared to its component estimators. They demonstrate that the efficiency gain depends on the ratio of the variances of the component estimators and the correlation between them (which is driven by how predictive the covariates are of the outcome).

Key Results

• Theoretical Efficiency: The paper proves that the relative efficiency of the combined estimator compared to the more efficient of its two components is maximized when the two components have equal variance. However, this gain is bounded; the combined estimator cannot be more than twice as efficient (200%) as the best single component estimator.
• Simulation Study: A simulation study involving 36 scenarios (varying outcome distributions, sample sizes, and sampling designs) was conducted.
  • Efficiency Gains: The combined estimators ($\hat{\mu}_{HT,PL1}$ and $\hat{\mu}_{Haj,PL2}$) consistently outperformed their individual components. The gains were most pronounced when the variances of the component estimators were similar (i.e., when Sample A and Sample B sizes were comparable).
  • Model Misspecification: The combined estimators maintained robustness when the outcome model was misspecified, confirming the theoretical properties of the DR framework.
  • Variance Estimation: The proposed variance estimators yielded standard errors that closely matched empirical standard errors (ratios ranging from 0.94 to 1.05). The coverage of 95% confidence intervals was generally close to the nominal level (93%–96.5%), with slight under-coverage observed in scenarios with very small Sample B sizes.
  • KH vs. PL: The Kim-Haziza (KH) method and Maximum Pseudo Likelihood (PL) method produced similar results, though the KH method occasionally faced convergence issues in small samples.

Significance and Claims
The paper claims to address a limited attention area in survey methodology: the efficient integration of outcome data from both probability and nonprobability samples. Its primary significance lies in:

1. Generalization: Extending DR estimators to domain estimation and introducing a Hajek-style variant ($\hat{\mu}_{KH2}$) that functions without known domain sizes.
2. Robust Combination: Providing a rigorous framework for combining HT/Hajek and DR estimators that accounts for their covariance and remains valid under model misspecification.
3. Practical Utility: Demonstrating through simulation that the combined approach yields tangible efficiency gains, particularly when the two data sources provide complementary information (i.e., when their individual variances are comparable).

The authors modestly note that while they have not explored pre-testing for consistency between estimators (as suggested by Gao and Yang), their framework could theoretically be extended to accommodate such approaches. They also identify the need for future exploration of their variance estimators against bootstrap-based approximations.