📈 economics

A Neyman-Orthogonalization Approach to the Incidental Parameter Problem in Likelihood Models

This paper proposes a method to construct likelihood-based estimating equations that are orthogonal to nuisance parameters up to an arbitrary order $q$ , thereby mitigating the incidental parameter problem in models with fixed effects and enabling robust inference even when nuisance parameters are imprecisely estimated.

Original authors: Stéphane Bonhomme, Koen Jochmans, Martin Weidner

Published 2026-08-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Stéphane Bonhomme, Koen Jochmans, Martin Weidner

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of economics, researchers often try to understand how specific factors influence outcomes, such as how a worker's skill affects their wage or how a team's composition impacts its productivity. To do this, they build mathematical models that separate the factors they care about from other, unwanted variables that clutter the picture. These unwanted variables are called nuisance parameters. In many cases, these nuisance factors are specific to individuals, like a unique talent for a particular teacher or a specific trait of a single researcher. When data is limited, estimating these individual traits becomes difficult, and the errors in estimating them can distort the final answer about the main factors of interest. This is a persistent problem known as the incidental parameter problem. For decades, statisticians have used a technique called orthogonality to try to fix this. This method involves designing the calculation so that small errors in estimating the nuisance factors do not immediately throw off the main result. However, this standard approach often fails when the data is sparse or the individual traits are very hard to pin down, leaving researchers with biased conclusions.

A team of economists from the University of Chicago, Toulouse School of Economics, and the University of Oxford has developed a more powerful solution to this problem. They created a method that goes beyond the standard fix by making the calculation robust to estimation errors, provided those errors shrink at a specific rate as the sample size grows. Instead of just ensuring the first step of the math is clean, their approach cleans up the second, third, and even higher steps of the calculation. They call this higher-order orthogonalization. By doing this, they can use data where the individual traits are estimated imprecisely and still arrive at an accurate answer for the main question, as long as the estimation error decreases fast enough relative to the sample size. The researchers tested this new method on a real-world problem involving scientific research teams. They looked at data from thousands of academic papers to understand how researchers work together. Specifically, they wanted to know if working with a partner improves the quality of a paper and whether certain researchers complement each other's skills.

The researchers applied their new technique to a dataset containing over 91,000 articles published in the field of economics between 1990 and 1999. The data included information on who wrote each paper and the quality of the journal where it appeared. In this setting, every author has a unique, unobserved ability that affects the quality of their work. Because many authors only published a few papers during this period, it was impossible to measure their individual ability with high precision using standard methods. When the researchers used the old, standard way of handling these individual abilities, the results were skewed. The standard method suggested that working with a partner increased paper quality, but the estimate of how researchers interacted was biased. Specifically, the uncorrected estimate of the substitution parameter was close to the Cobb-Douglas case, which implies a specific type of production relationship, but the bias obscured the true nature of the interaction.

When the researchers applied their new, higher-order method, the picture changed significantly. The new estimates showed that the boost in quality from having a co-author was more moderate than the standard method suggested. More importantly, the new method revealed that researchers are complements rather than substitutes. This means that when two researchers with different strengths work together, they produce better work than the sum of their individual parts. The data indicated that the specific combination of authors matters deeply. The researchers also simulated what would happen if authors were randomly paired into teams instead of choosing their partners. They found that random pairing would lead to a noticeable drop in the average quality of articles. This suggests that the current system, where researchers choose their collaborators, is effective because it allows for positive sorting, where skilled researchers find other skilled researchers to work with.

The study also demonstrated that the new method is stable. As the researchers increased the complexity of their correction, the results settled down to a consistent value, indicating that the bias had been successfully removed. In contrast, the standard method produced results that were far from this stable value. The researchers confirmed their findings through extensive computer simulations that mimicked their real-world data. These simulations showed that their approach could recover the true underlying relationships even when the individual abilities were estimated with very little data. The work provides a new tool for economists and other social scientists who deal with complex data involving many individual characteristics. It allows them to draw more reliable conclusions from datasets that were previously too messy to analyze accurately, ensuring that the stories they tell about human behavior are based on solid ground rather than statistical artifacts.

Technical Summary: A Neyman-Orthogonalization Approach to the Incidental Parameter Problem in Likelihood Models

1. Problem Statement and Motivation
The paper addresses the incidental parameter problem in likelihood models, specifically in settings where the dimension of nuisance parameters is large relative to the sample size, such as panel data with fixed effects or network data with node-specific heterogeneity. In these contexts, standard plug-in estimators for target parameters (e.g., structural parameters $\theta_0$ or average effects $\mu_0$ ) suffer from substantial bias because the nuisance parameters (e.g., fixed effects $\eta_{i0}$ ) are estimated at a rate slower than the parametric rate required for valid inference.

The authors note that while first-order Neyman-orthogonalization (where the expected first derivative of the estimating equation with respect to the nuisance parameter is zero) allows for valid inference if the nuisance parameter converges faster than $n^{-1/4}$ , this condition often fails in high-dimensional settings. For instance, in panel data with $N$ units and $T$ periods, the fixed effects converge at rate $T^{-1/2}$ . First-order orthogonality requires $N = o(T)$ , a condition rarely met in microeconometric applications. Consequently, first-order orthogonalization alone does not resolve the incidental parameter problem in these settings.

2. Methodology
The authors propose a higher-order generalization of Neyman-orthogonalization within the conditional-likelihood framework.

Higher-Order Orthogonality: An estimating equation $u(Z; \theta, \eta, \mu)$ is defined as Neyman-orthogonal to order $q$ if the expected value of its first $q$ partial derivatives with respect to the nuisance parameter $\eta$ are zero:
$E\left[ \nabla^q_\eta u(Z; \theta_0, \eta_0, \mu_0) \right] = 0$
where $\nabla^q_\eta$ represents the vector of all partial derivatives up to order $q$ . When combined with sample splitting (or cross-fitting), this property ensures that the estimation error in the nuisance parameter $\hat{\eta}$ affects the target parameter estimator at a rate of $O_p(|\hat{\eta} - \eta_0|^{q+1})$ . This relaxes the convergence requirement for $\hat{\eta}$ to $o_p(n^{-1/2(q+1)})$ , allowing for valid inference even when $N = o(T^q)$ .
Construction via Bhattacharyya Basis: The paper provides a constructive method to generate such higher-order orthogonal functions. Leveraging the conditional likelihood $\ell(y|x; \theta, \eta)$ , the authors utilize the Bhattacharyya basis functions $v_p$ , defined as the $p$ -th order derivatives of the log-likelihood normalized by the likelihood itself. These basis functions have zero conditional mean.
Theorem 1 establishes that a new moment function $u^*_q$ can be constructed by projecting the original moment function $u$ onto the space spanned by the first $q$ Bhattacharyya basis functions ( $w_q$ ). Specifically:
$u^*_q(z; \theta, \eta, \mu) = u(z; \theta, \eta, \mu) - A(x; \theta, \eta, \mu)^\top w_q(z; \theta, \eta)$
where $A$ is derived from the conditional covariance matrices of the basis functions and the moment function. This construction generalizes the projected score of Small and McLeish (1989) and Waterman and Lindsay (1996) to arbitrary order $q$ and general parameters of interest (not just $\theta$ ).
Estimation Procedure: The target parameter is estimated using Generalized Method of Moments (GMM) based on the orthogonalized functions $u^*_q$ and preliminary estimators $\hat{\eta}_i$ obtained via sample splitting. The authors emphasize that the conditional-likelihood structure is crucial, as it allows the construction of these higher-order equations without introducing additional nuisance parameters (which would otherwise be required to solve the system of equations for higher orders).

3. Key Contributions

General Construction: The paper provides a general, closed-form procedure to construct estimating equations orthogonal to any chosen order $q$ in conditional-likelihood models.
Relaxation of Convergence Rates: By increasing the order of orthogonality $q$ , the method significantly relaxes the required convergence rate of nuisance parameters, making valid inference feasible in settings where $N$ is large relative to $T$ (e.g., $N = o(T^q)$ ).
Handling Overlapping Fixed Effects: The methodology accommodates complex data structures, such as "overlapping fixed effects" in network data, where a single nuisance parameter appears in multiple observations.
Diagnostic Tool: The authors propose a diagnostic statistic based on the difference between orthogonal moment functions of order $q$ and $q+1$ to assess whether the chosen order of orthogonality is sufficient to eliminate bias.

4. Empirical Results and Illustrations
The paper illustrates the method through two primary examples:

Neyman-Scott Model: The authors demonstrate that while the standard score is first-order orthogonal, it remains biased. A second-order orthogonalized score is derived, which yields the degrees-of-freedom corrected estimator (dividing by $T-1$ instead of $T$ ), fully removing the $O(T^{-1})$ bias.
Team Production (Network Data): The authors apply the method to estimate a Constant Elasticity of Substitution (CES) production function using data on economic research collaborations (Ductor et al., 2014). The model includes researcher-specific fixed effects.
- Findings: Standard plug-in estimators and first-order orthogonalized estimators yield unstable or biased results (e.g., a substitution parameter $\gamma$ near 0.13). Higher-order orthogonalized estimators ( $q \geq 2$ ) converge to a stable estimate of $\gamma \approx 0.41-0.47$ , indicating supermodularity (complementarity) among researchers.
- Counterfactuals: The method is used to simulate a random reallocation of researchers. The results suggest that random pairing would decrease average article quality compared to the observed allocation, driven by positive sorting and complementarity.
- Stability: Estimates for parameters and standard errors stabilize as $q$ increases from 2 to 6, with no evidence of a sharp bias-variance trade-off in this specific application.

5. Significance and Claims
The paper claims that higher-order Neyman-orthogonalization offers a robust solution to the incidental parameter problem in likelihood models where nuisance parameters are estimated imprecisely. Unlike bias-correction methods that require explicit estimation and removal of bias terms (e.g., Hahn and Newey, 2004), this approach achieves bias reduction by modifying the estimating equations themselves.

The authors position this work as a bridge between semiparametric inference and high-dimensional fixed-effects models. They note that while the method relies on the conditional-likelihood structure (specifically the existence of Bartlett identities and the information matrix equality), it successfully handles complex network structures where alternative solutions are currently unavailable. The paper concludes modestly, acknowledging that the approach requires the likelihood structure to hold and that extending these results to general semi-parametric moment condition models (without likelihoods) is a subject for future work.

More like this