A Beta-Based Heteroskedasticity-Consistent Covariance Matrix Estimator

Technical Summary: A Beta-Based Heteroskedasticity-Consistent Covariance Matrix Estimator

Problem Statement
In ordinary least squares (OLS) regression, the assumption of homoskedasticity is frequently violated in cross-sectional applications. While the OLS estimator remains unbiased and consistent under heteroskedasticity, the conventional covariance matrix estimator becomes biased and inconsistent, rendering standard inferential procedures (hypothesis tests and confidence intervals) invalid. To address this, heteroskedasticity-consistent (HC) covariance matrix estimators have been developed to provide asymptotically valid standard errors without modeling the variance function explicitly.

However, existing HC estimators (e.g., HC2, HC3, HC4, HC4m) rely on leverage-based adjustments that scale squared residuals using powers of $(1 - h_t)$, where $h_t$ is the leverage of the $t$-th observation. The paper identifies a critical flaw in these methods, termed "overshooting." In samples with high-leverage observations, particularly in moderate sample sizes, the aggressive functional forms of existing estimators can induce explosive growth in adjustment factors. This leads to excessively inflated standard errors and unstable inference, where the correction for leverage becomes disproportionate to the actual data structure.

Methodology: The HC$\beta$ Estimator
The authors propose a new estimator, denoted HC$\beta$, which replaces the fixed or piecewise-defined leverage adjustments of traditional HC estimators with a data-driven correction derived from a fitted Beta distribution.

1. Conceptual Framework: The method generalizes the interpretation of $(1 - h_t)$. In existing estimators, this term acts as the cumulative distribution function (CDF) of a Uniform(0,1) distribution. HC$\beta$ replaces this with the CDF of a Beta distribution, $F_{Beta}(1 - h_t; a, b)$, which is estimated directly from the data.
2. Parameter Estimation: The shape parameters $a$ and $b$ of the Beta distribution are estimated using the method of moments applied to the transformed leverage values $w_t = \max\{0.01, \min\{1 - h_t, 0.99\}\}$. Truncation is applied to $w_t$ to prevent numerical instability caused by leverage values approaching 0 or 1, which would otherwise lead to degenerate parameter estimates or extreme adjustment factors.
3. Regularization: To ensure stability in small samples, the estimated parameters are shrunk toward the uniform distribution ($a=b=1$) using a shrinkage factor $\zeta = n/(n+50)$. This ensures that for small $n$, the estimator borrows strength from the stable uniform baseline, while for large $n$, it retains full flexibility.
4. Adjustment Factor: The adjustment factor $g_t$ for the $t$-th squared residual is defined as:
   $$g_t = \frac{n}{n-p} \left( \frac{1}{F_{Beta}(w_t; \tilde{a}, \tilde{b})} \right)^{c_1/n^{c_2}}$$
   where $\tilde{a}$ and $\tilde{b}$ are the regularized parameter estimates. The constants $c_1=7$ and $c_2=0.75$ were determined via Monte Carlo simulations to balance finite-sample correction intensity with asymptotic convergence. As $n \to \infty$, the exponent $c_1/n^{c_2} \to 0$, causing $g_t \to 1$ and ensuring the estimator converges to the standard HC0 structure asymptotically.

Key Contributions

• Novel Adjustment Mechanism: HC$\beta$ introduces a flexible, parametric adjustment based on the Beta distribution, allowing the correction to adapt to the specific empirical distribution of leverage values in a sample, rather than relying on rigid functional forms.
• Mitigation of Overshooting: By modeling the leverage distribution, HC$\beta$ avoids the "overshooting" phenomenon where aggressive corrections lead to unstable, excessively large standard errors in the presence of influential observations.
• Asymptotic and Finite-Sample Properties: The estimator is designed to be asymptotically equivalent to traditional HC estimators under standard regularity conditions while offering superior finite-sample performance.
• Software Implementation: The authors developed and released the open-source R package hcinfer, which implements HC$\beta$ alongside classical HC estimators, providing a unified framework for covariance estimation, hypothesis testing, and diagnostic visualization.

Results
The performance of HC$\beta$ was evaluated through extensive Monte Carlo simulations and empirical applications.

• Simulation Results:

  • Size Distortion: In scenarios with strong heteroskedasticity and high leverage, HC$\beta$ demonstrated smaller size distortions (rejection rates closer to the nominal 5% level) compared to HC3, HC4, and HC4m. While traditional estimators often exhibited liberal behavior (over-rejection) or erratic convergence, HC$\beta$ provided more controlled inference.
  • Convergence: The estimator showed rapid convergence to nominal levels as sample size increased.
  • Coverage: Confidence intervals based on HC$\beta$ achieved coverage rates closer to the nominal 95% level than competing estimators, particularly in small samples ($n=50$) with strong heteroskedasticity, where traditional methods suffered from severe undercoverage.
  • Power: The power of quasi-t tests based on HC$\beta$ was comparable to other HC estimators, indicating that the improved size control did not come at the cost of significant power loss.

• Empirical Applications:

  • The paper applied HC$\beta$ to four datasets: public school spending, Boston house prices, 2009 crime data, and orthorexia nervosa/exercise addiction.
  • High-Leverage Scenarios: In datasets with extreme leverage points (e.g., Washington D.C. in the crime data, specific outliers in the house price data), traditional estimators (HC3, HC4, HC4m) produced massive adjustment factors (e.g., $g_t > 2000$), leading to inflated standard errors and failure to reject null hypotheses that were otherwise significant. HC$\beta$ produced moderate, controlled adjustment factors (e.g., $g_t < 7$), yielding stable standard errors and coherent inference that aligned with results obtained after removing the influential observations.
  • Moderate-Leverage Scenarios: In datasets without extreme outliers, HC$\beta$ provided slightly more conservative (larger) standard errors than HC0, reflecting its adaptive mechanism to prevent underestimation of variability, while maintaining inferential stability.

Significance
The paper posits that the "overshooting" of residual-based corrections under extreme leverage is a significant, previously underappreciated source of inferential instability in heteroskedasticity-consistent procedures. The significance of HC$\beta$ lies in its ability to balance flexibility and stability. It offers a data-driven mechanism that accommodates heterogeneous leverage patterns without resorting to the explosive corrections seen in higher-order HC estimators.

The authors claim that HC$\beta$ provides a more robust framework for inference in linear regression, particularly in empirical settings characterized by unknown heteroskedasticity, moderate sample sizes, and the presence of influential observations. By avoiding the need for computationally intensive resampling methods (like bootstrapping) while outperforming analytical HC estimators in finite samples, HC$\beta$ offers a practical and theoretically grounded alternative for applied researchers. The availability of the hcinfer package facilitates the immediate adoption of this methodology in standard statistical workflows.