Model Checking for Regressions Based on Weighted Residual Processes with Diverging Number of Predictors

1. Problem Statement

The paper addresses the critical challenge of model specification testing (goodness-of-fit) in high-dimensional regression settings where the number of predictors ($d$) and the number of parameters ($p$) diverge with the sample size ($n$).

• Context: Researchers often need to verify if a parametric regression model $Y = m(X, \beta) + \varepsilon$ correctly specifies the conditional mean function.
• Limitations of Existing Methods:
  • The classical Integrated Conditional Moment (ICM) test (e.g., Bierens, 1982) performs well in fixed-dimensional settings but fails in high dimensions.
  • Degeneracy: As $d \to \infty$, the limiting null and alternative distributions of the ICM statistic degenerate to fixed constants due to the "curse of dimensionality" (specifically, the concentration of inter-point distances in the kernel weights).
  • Bootstrap Failure: The standard wild bootstrap, used to approximate the null distribution of the ICM test, also converges to the same degenerate constant in high dimensions. This leads to severe size distortion (incorrect Type I error rates) and substantial power loss.

2. Methodology

The authors propose a novel specification test based on weighted residual processes that avoids the curse of dimensionality by shifting the integration domain from the high-dimensional covariate space to a one-dimensional space.

A. The Proposed Test Statistic ($WICM_n$)

Instead of integrating over the high-dimensional covariate vector $X$ (as in ICM), the proposed method utilizes a real-valued weight function $g(X)$ and integrates over a scalar parameter $t$.

1. Residual Process: Define the centered weight function $g_0(X) = g(X) - E[g(X)]$. The test constructs a residual empirical process:
   $$\hat{U}_n(t) = \frac{1}{\sqrt{n}} \sum_{i=1}^n \{g(X_i) - \bar{g}\} [\cos(t\hat{\varepsilon}_i) + \sin(t\hat{\varepsilon}_i)]$$
   where $\hat{\varepsilon}_i$ are the fitted residuals.
2. Aggregation: The test statistic aggregates this process over $t$ using an even weight function $\phi(t)$ (e.g., standard normal density):
   $$WICM_n = \int_{\mathbb{R}} |\hat{U}_n(t)|^2 \phi(t) dt$$
3. Dimensionality Reduction: By using $g(X)$ as a scalar projection, the integration is reduced to one dimension, avoiding the exponential decay of weights associated with high-dimensional distances.

B. Smooth Residual Bootstrap

Since the limiting null distribution of $WICM_n$ depends on the unknown error distribution and is not asymptotically pivotal, the authors propose a smooth residual bootstrap to approximate critical values:

1. Generate bootstrap errors $\varepsilon^*_{i,j}$ by resampling centered residuals and adding a small noise term with a smoothing parameter $v_n$.
2. Construct bootstrap responses $Y^*_{i,j}$ and re-estimate parameters.
3. Compute the bootstrap test statistic $WICM^*_n$.
4. Repeat to form an empirical distribution for critical value determination.

C. Weight Function Selection ($g(X)$)

To maximize power against local alternatives, the paper proposes data-driven strategies for selecting $g(X)$:

• Directional Alternatives: If the alternative belongs to a specific parametric class, $g(X)$ is constructed based on the difference between the null and alternative score functions.
• Nonparametric Alternatives: A Fourier expansion approach is used, projecting the regression function onto an orthonormal basis derived from the score function of the null model.

3. Key Contributions

1. Resolution of Degeneracy: The proposed test maintains a non-degenerate limiting distribution under both the null and alternative hypotheses, even as $p \to \infty$.
2. Theoretical Validity in High Dimensions:
   • Established asymptotic normality under the null hypothesis with a relaxed rate condition: $p^3 \log(n) / n \to 0$.
   • Proved consistency against fixed alternatives.
   • Demonstrated the ability to detect local alternatives deviating from the null at the parametric rate $1/\sqrt{n}$.
3. Valid Bootstrap Approximation: Proved that the smooth residual bootstrap consistently approximates the null distribution in high-dimensional settings, correcting the failure of the wild bootstrap.
4. Improved Rate Conditions: The theoretical conditions for validity are weaker than those in recent literature (e.g., Tan et al., 2025), broadening the applicability of the theory.

4. Results

Simulation Studies

The authors conducted extensive simulations comparing their proposed tests ($WICM^{(1)}_n$ for directional, $WICM^{(2)}_n$ for nonparametric) against the classical ICM test and the PCvM test.

• Size Control: The proposed tests maintained the nominal significance level (0.05) across various dimensions ($p$) and sample sizes ($n$). In contrast, the ICM test suffered from severe size distortion (rejecting the null too rarely) as $p$ increased.
• Power:
  • The proposed tests exhibited significantly higher power than the ICM test in high dimensions.
  • $WICM^{(1)}_n$ (directional) outperformed others when the alternative had a specific structure.
  • $WICM^{(2)}_n$ (nonparametric) showed robust performance, particularly for complex, non-smooth alternatives where dimension reduction methods failed.
  • The PCvM test showed moderate power but was generally inferior to the proposed methods in high-dimensional, non-structured settings.

Real Data Application

The method was applied to the Geographical Origin of Music dataset (1,059 observations, 68 audio features).

• Task: Test if a linear model adequately predicts latitude based on audio features.
• Result: The proposed test yielded a p-value $\approx 0$, strongly rejecting the linear model.
• Visualization: Scatter plots confirmed non-linear relationships, validating the test's conclusion that a more flexible model is required.

5. Significance

This paper provides a rigorous solution to a fundamental gap in modern statistical methodology: valid model checking in high-dimensional regimes.

• Practical Impact: It offers a reliable tool for practitioners who cannot rely on classical ICM tests when $d$ is large, preventing misleading scientific conclusions due to model misspecification.
• Theoretical Advancement: By establishing the asymptotic properties of weighted residual processes in diverging dimensions and validating a smooth bootstrap, the paper extends the frontier of specification testing beyond fixed-dimensional constraints.
• Robustness: The method does not require the underlying regression to possess a low-dimensional structure (e.g., single-index), making it applicable to general parametric models in the "Big Data" era.