Bias Correction for Semiparametric Regression Models

Technical Summary: Bias Correction for Semiparametric Regression Models (SABRE)

1. Problem Statement

The paper addresses a critical gap in the semiparametric literature regarding models where the conditional distribution of a response $Y$ given covariates $(x, z)$ takes the form $f\{Y | x^T\beta + m(z), \phi\}$. In this class of models:

• $\beta$ is a parametric component of dimension $p$, which is allowed to diverge with the sample size $n$.
• $m(\cdot)$ is an unknown smooth nonparametric function.
• $\phi$ is a dispersion parameter.

While existing literature has focused heavily on achieving semiparametric efficiency for $\beta$ (often treating $\phi$ and $m(\cdot)$ as nuisance parameters), it has largely ignored finite-sample bias. The authors argue that in finite samples—particularly when the ratio $p/n$ is large, dispersion is high, or data is subject to misclassification (e.g., in electronic health records or label-only differential privacy)—standard estimators exhibit substantial bias. This bias undermines the accuracy of confidence intervals (CIs) and hypothesis tests for $\beta$ and leads to unreliable estimation of $\phi$, which is often of direct scientific interest.

2. Methodology: SABRE

The authors propose SABRE (SemipArametric Bias-Reduced Estimation), a unified, simulation-based bias correction framework. The methodology proceeds in three main stages:

A. Parametric Approximating Model
Since the true model contains an unknown function $m(\cdot)$, SABRE first approximates $m(\cdot)$ using a B-spline expansion:
$$m(z) \approx B(z)^T \alpha = \sum_{j=1}^K \alpha_j B_j(z)$$
where $B_j(z)$ are B-spline basis functions and $\alpha$ is a vector of coefficients. This transforms the semiparametric model into a parametric approximating model with parameter vector $\gamma = (\beta^T, \alpha^T)^T$.

B. Initial Estimator (sMLE)
An initial estimator, denoted as the B-spline Maximum Likelihood Estimator (sMLE), is obtained by solving the score equations for $\gamma$ and $\phi$ based on the observed data. Let $\hat{\gamma}$ and $\hat{\phi}$ denote these initial estimates.

C. Simulation-Based Correction
SABRE defines the final estimator $(\tilde{\gamma}, \tilde{\phi})$ as the solution to an estimating equation that equates the observed initial estimator to its expected value under the parametric approximating model:
$$\begin{pmatrix} \tilde{\gamma} \\ \tilde{\phi} \end{pmatrix} = \text{argzero}_{\gamma, \phi} \begin{pmatrix} \hat{\gamma} - E_{\gamma, \phi}[\hat{\gamma}^*(\gamma, \phi)] \\ \hat{\phi} - E_{\gamma, \phi}[\hat{\phi}^*(\gamma, \phi)] \end{pmatrix}$$
Here, $\hat{\gamma}^*(\gamma, \phi)$ and $\hat{\phi}^*(\gamma, \phi)$ are the sMLEs computed on synthetic data generated from the B-spline approximating model with parameters $(\gamma, \phi)$. The expectations $E_{\gamma, \phi}[\cdot]$ are approximated via Monte Carlo simulation. To handle computational burden in high dimensions, the authors employ stochastic approximation methods to solve these equations without explicitly computing expectations at every step.

3. Key Contributions

The paper outlines four primary contributions:

1. First Framework for this Class: SABRE is presented as the first bias correction framework specifically designed for this broad class of semiparametric models with diverging dimension $p$, a functional component $m(\cdot)$, and a dispersion parameter $\phi$.
2. Bias Reduction for $\beta$ and $\phi$: Unlike existing methods that assume fixed $p$ or focus solely on $\beta$, SABRE achieves bias reduction for both the parametric component $\beta$ and the dispersion parameter $\phi$ without relying on sparsity assumptions or penalized methods. It allows $p$ to diverge at a rate of $p^2 \log(n) n^{-1} \to 0$.
3. Asymptotic Efficiency: The authors establish that the SABRE estimator for $\beta$ retains the same asymptotic variance as the initial sMLE. Consequently, the bias correction is achieved without inflating variance, preserving semiparametric efficiency.
4. Theoretical Development: Within the subclass of Generalized Partially Linear Models (GPLMs), the paper provides a rigorous asymptotic theory proving:
   • Convergence rates for $\tilde{\beta}$ and $\tilde{\phi}$.
   • Reduced bias orders: The bias of $\tilde{\beta}$ is shown to be of order $O(h^q) + O((p+N)^{3/2}n^{-3/2})$, which is smaller than the $O(n^{-1})$ bias of standard estimators and faster than the $O(n^{-1})$ bias of adjusted-score methods in fixed-dimensional parametric settings.
   • Asymptotic normality for $\tilde{\beta}$, justifying standard inference procedures (CIs and hypothesis tests).

4. Empirical Results

The authors validate SABRE through comprehensive simulation studies and a real-data application:

• Simulation Studies:

  • Partially Linear Logistic Regression (Misclassified Data): In settings with misclassified binary outcomes and varying $p/n$ ratios, SABRE maintained negligible bias across all scenarios. In contrast, the standard sMLE exhibited substantial bias as $p/n$ or misclassification rates increased, leading to poor CI coverage. The adjusted-score method (sMLE-BR) was inconsistent under misclassification. SABRE consistently produced CIs with accurate coverage and shorter lengths.
  • Dispersion Estimation (Inverse Gaussian & Negative Binomial): For models with high dispersion and large $p/n$, the standard sMLE showed substantial bias in estimating $\phi$. SABRE significantly reduced this bias and achieved lower Root Mean Squared Error (RMSE) compared to competitors.

• Real Data Application (Early-Stage Diabetes):

  • The authors analyzed a dataset of 520 individuals to model the association between symptoms and diabetes status, accounting for potential misclassification (false negatives).
  • SABRE produced CIs approximately 20% shorter than the sMLE, corresponding to a roughly 56% increase in effective sample size.
  • Crucially, SABRE identified a statistically significant association between alopecia and diabetes (consistent with clinical literature), an association that the standard sMLE failed to detect due to bias.

5. Significance and Claims

The paper claims that finite-sample bias is a central issue in modern semiparametric inference, particularly in high-dimensional settings and those involving noisy or privatized data. The significance of SABRE lies in its ability to:

• Correct Bias without Sacrificing Efficiency: It reduces bias for both $\beta$ and $\phi$ while maintaining the optimal variance properties of the initial estimator.
• Handle Complex Modern Settings: It is applicable to models with diverging dimensions, nonparametric components, and complex data mechanisms like misclassification and label-only differential privacy.
• Improve Scientific Inference: By providing more accurate CIs and reducing bias, SABRE enhances the reliability of downstream inference and the detection of scientifically meaningful effects, as demonstrated in the diabetes application.

The authors note that while the full asymptotic theory is developed for the GPLM subclass, the methodology is applicable to the broader model class, and numerical experiments suggest promising performance beyond GPLMs. They acknowledge that a complete formal theory for the full class remains a subject for future work due to the distribution-specific nature of the derivations.