Two-stage Estimation of Latent Variable Regression Models: A General, Root-N Consistent Solution

Technical Summary: Two-Stage Estimation of Latent Variable Regression Models

Problem Statement
Latent variable (LV) models are standard in psychological research for operationalizing unobservable constructs. While one-stage maximum likelihood (ML) estimation is the "gold standard," it often faces practical limitations, including numerical instability (e.g., non-convergence, Heywood cases), computational intractability due to complex integrals in multidimensional models, software constraints, and data availability issues (e.g., only composite scores are available). Consequently, researchers frequently resort to two-stage factor score regression (FSR). In FSR, a measurement model is fitted first to predict factor scores, which are then treated as observed variables in a second-stage structural regression. However, naive application of FSR yields biased estimates of structural parameters due to measurement error and the discrepancy between the covariance structure of factor scores and true latent variables. Existing bias-correction methods are often limited to specific model classes (e.g., linear SEMs) or require specific types of factor scores, lacking a general, computationally efficient solution for broader LV models.

Methodology
The authors propose a general bias-correction framework for two-stage estimation applicable to parametric statistical models with multiple focal and nuisance parameters. The method is tailored specifically to FSR but is designed to be a "blackbox" solution requiring minimal analytical derivation.

1. Statistical Framework: The parameter vector $\theta$ is partitioned into nuisance parameters ($\nu$, e.g., measurement parameters) and focal parameters ($\phi$, e.g., structural regression coefficients). The two-stage procedure estimates $\hat{\nu}$ first, then estimates $\hat{\phi}$ conditional on $\hat{\nu}$.
2. Bias-Correction Strategy: Drawing on Leung and Wang (1998), the authors define a function $h(\phi; \nu) = E[\hat{\phi}(Y; \nu)]$, representing the expected value of the initial (biased) estimator. The bias-corrected estimator $\hat{\phi}_{BC}$ is obtained by inverting this relationship: $\hat{\phi}_{BC} = h^{-1}(\hat{\phi})$.
3. Theoretical Foundation: The paper establishes the $\sqrt{n}$-consistency of the bias-corrected estimator under mild regularity conditions. A key proposition demonstrates that if the initial estimators are jointly $\sqrt{n}$-consistent, the bias-corrected focal parameter estimator is also $\sqrt{n}$-consistent for the true parameter $\phi^*$, with a derived asymptotic covariance matrix (ACM).
4. Computational Algorithms: To avoid complex analytical derivations of $h(\phi; \nu)$ and its derivatives, the authors introduce two stochastic algorithms:
   • Point Estimation (Algorithm 1): A Robbins-Monro stochastic approximation (SA) algorithm. It iteratively updates the focal parameters by simulating data from the model using the current parameter estimates and comparing the resulting initial estimator to the observed data's initial estimator.
   • Variance Estimation (Algorithm 2): A Monte Carlo-based procedure using simultaneous perturbation (SP) to numerically approximate the Jacobian matrices required for the ACM. This allows for the calculation of valid standard errors (SEs) that account for the uncertainty carried over from the first stage.

Key Contributions

• Generalizability: Unlike previous solutions restricted to linear SEMs or specific factor score types (e.g., Bartlett or ARM scores), the proposed method applies to a broad class of LV models, including those with nonlinear structural relationships and non-normal manifest variables (e.g., Item Response Theory models).
• Computational Efficiency: The method relies on stochastic approximation and Monte Carlo simulation rather than closed-form analytical solutions, making it adaptable to complex models where one-stage ML is computationally prohibitive.
• Robustness to Factor Score Choice: The framework does not require the user to select specific types of factor scores (e.g., regression vs. Bartlett) to achieve consistency; the bias correction works regardless of the initial scoring method used.
• Theoretical Rigor: The paper provides a formal proof of $\sqrt{n}$-consistency and a method for deriving valid large-sample standard errors for the corrected estimators.

Results
Three simulation studies were conducted to evaluate the proposed bias-corrected FSR (BC-FSR) against naive FSR and one-stage ML (and DWLS for Study 3):

1. Study 1 (Simple Latent Regression): In models with continuous manifest variables, naive FSR exhibited significant attenuation bias (up to 19% for slopes) and variance bias. The BC-FSR reduced relative bias to near zero (<5%) across all conditions and factor score types (mean, Bartlett, regression), performing comparably to one-stage ML in both accuracy and efficiency.
2. Study 2 (Latent Moderation): In a bilinear interaction model with non-normal outcomes, naive FSR showed substantial bias (up to 33% for interaction effects). BC-FSR successfully recovered parameters with negligible bias, matching the performance of one-stage ML (computed via numerical integration in Mplus).
3. Study 3 (Multidimensional IRT): In a complex model with dichotomous items and multiple predictors, naive FSR showed bias ranging from 8% to 33% depending on the parameter. BC-FSR eliminated this bias, performing on par with one-stage ML (via MH-RM algorithm) and diagonally weighted least squares (DWLS).
   In all studies, the Monte Carlo-based SE estimates (Algorithm 2) were accurate, with relative bias in SEs generally under 10%.

Significance and Claims
The authors claim that their approach transforms the intuitive "measure-then-structural" strategy (FSR) from a heuristic prone to bias into a statistically valid, $\sqrt{n}$-consistent estimation method. The significance of the work lies in:

• Practical Utility: It offers a straightforward alternative to one-stage ML for researchers facing computational or data constraints, without sacrificing asymptotic accuracy.
• Methodological Shift: It challenges the conventional view that naive FSR is inherently flawed and unrecoverable, demonstrating that bias can be systematically removed via stochastic approximation.
• Accessibility: By minimizing reliance on analytical derivations, the method lowers the barrier for applying rigorous two-stage estimation to complex, customized LV models that exceed the capabilities of standard software packages.

The paper concludes that the proposed bias-corrected FSR enjoys the implementation convenience of two-stage methods while achieving the theoretical rigor and efficiency of one-stage methods, potentially empowering researchers to examine a wider array of complex relationships among latent constructs.