📊 statistics

New Confidence Regions for Linear Regression Parameters with Stationary-Ergodic Dependent Errors

This paper proposes a novel method for constructing joint confidence regions for linear regression coefficients under stationary-ergodic dependent errors by employing random smoothing with an auxiliary sample, which achieves asymptotic normality and reliable finite-sample performance without requiring explicit long-run variance estimation or parametric dependence modeling.

Original authors: Mous-Abou Hamadou, Martial Longla, Mathias Nthiani Muia, Mahmud Hasan

Published 2026-05-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mous-Abou Hamadou, Martial Longla, Mathias Nthiani Muia, Mahmud Hasan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to draw a map of a hidden treasure (the true values of your data's relationships) based on a series of clues. In statistics, this is called linear regression. Usually, statisticians assume that each clue is independent, like rolling a die where the next roll doesn't care about the previous one.

But in the real world, clues often come in clusters. If it's windy today, it's likely windy tomorrow. If a stock price drops, it might keep dropping. This is called dependent error. When these clues are "sticky" or connected, the standard maps statisticians use often get distorted, leading to confidence regions (the area where we think the treasure is) that are either too small (missing the treasure) or too big (useless).

This paper introduces a new, more robust way to draw that map when the clues are sticky, without needing to know exactly how they are connected.

The Problem: The "Sticky" Clues

Think of the standard method (like the famous Newey-West approach) as trying to measure the "stickiness" of the clues by counting how many times they repeat. It's like trying to predict the weather by looking at a single, complex formula. If you guess the formula wrong, or if the weather is unusually chaotic (long memory), your map becomes unreliable.

The Solution: Random Smoothing with a "Ghost" Sample

The authors propose a clever trick called Random Smoothing. Here is the analogy:

Imagine you are trying to find the center of a crowded, noisy room (your data).

The Standard Way: You ask everyone in the room to shout their location, average the voices, and hope the noise cancels out. If the people are shouting in a coordinated pattern (dependent errors), the average is skewed.
The New Method: You bring in a second, completely silent, independent group of people (an "auxiliary sample"). You don't ask them for data; you just use them as a "ruler."
- You take a person from the noisy room and a person from the silent group.
- If the silent person is standing very close to the noisy person, you give that noisy person's voice a "high weight" (you listen closely).
- If they are far apart, you ignore that noisy person's voice.
- You do this randomly for everyone, using a "shrinking ruler" (bandwidth) that gets more precise as you get more data.

This technique, called Random Smoothing, effectively "smooths out" the noise without needing to calculate the complex, sticky connections between the data points. It's like using a magic lens that blurs out the chaotic patterns automatically, revealing the true shape underneath.

The "Smoother" and the "Safety Net"

The authors realized that just using this smoothing trick has a few bumps in the road:

The Bandwidth Dilemma: How big should the "ruler" be? If it's too big, you blur the picture too much. If it's too small, the noise wins. The paper introduces a scaled estimator (a modified version of the ruler) that finds the perfect balance automatically, like a camera that auto-focuses based on the lighting.
The "Near-Miss" Safety Net: Sometimes, the math gets stuck if the data points are too similar (like trying to divide by zero). The authors add a mild truncation, which is like a safety valve. If the math gets too unstable, the valve opens slightly to keep the calculation moving without breaking the final result.

What They Found (The Results)

The authors tested their new map-drawing tool against the old standard tools using various types of "sticky" data:

Short-term stickiness: (Like a mild echo).
Long-term stickiness: (Like a memory that lasts for years, known as "long memory").
Chaotic stickiness: (Non-linear patterns that don't follow simple rules).

The Verdict:

The Old Tools: Worked well when the stickiness was simple and short, but often failed (gave wrong maps) when the stickiness was strong, long-lasting, or weird.
The New Tool: It was more reliable. It didn't always produce the smallest map (which is nice), but it was much more likely to actually contain the true treasure, especially when the data was messy or had long memories. It didn't break down when the standard tools got confused.

Real-World Test: Beijing's Winter Air

To prove it works in the real world, they applied it to Beijing's winter PM2.5 (air pollution) data.

The Setup: They tried to predict pollution levels based on weather (temperature, wind, pressure).
The Issue: Pollution doesn't change randomly; if it's bad today, it's likely bad tomorrow (dependent errors).
The Result: The new method successfully identified that wind speed and atmospheric pressure were the main drivers of pollution changes, while temperature was less certain. Crucially, it gave a reliable "confidence region" (a map of certainty) that accounted for the fact that the pollution data was "sticky," whereas standard methods might have been overconfident or misleading.

Summary

In short, this paper offers a new, "plug-and-play" way to analyze data where the errors are connected. Instead of trying to reverse-engineer the complex connections (which is hard and error-prone), it uses a random smoothing trick with a helper sample to naturally filter out the noise. It's a more robust, stable way to say, "We are 95% sure the answer is in this area," even when the data is behaving unpredictably.

Technical Summary: New Confidence Regions for Linear Regression Parameters with Stationary–Ergodic Dependent Errors

Problem Statement
Constructing joint confidence regions for linear regression coefficients under serial dependence remains a significant challenge when the dependence structure is unspecified. While standard approaches rely on estimating the long-run covariance matrix (e.g., Newey–West HAC estimators), these methods can be sensitive to tuning choices in finite samples, particularly under persistent dependence or moderate sample sizes. Alternative methods, such as fixed-b asymptotics or self-normalized statistics, offer different limiting theories but often require specific weak dependence conditions. Furthermore, in models with long-memory errors, inference frequently relies on parametric or semiparametric descriptions of the serial dependence, limiting applicability when the structure is unknown. This paper addresses the need for a robust inference procedure that does not impose a parametric model on the error process nor directly estimate the long-run covariance matrix.

Methodology
The proposed approach extends the random-smoothing technique introduced by Longla and Peligrad (2021) from mean estimation to linear regression coefficients. The core methodology involves the following steps:

Moment Vector Reformulation: The regression coefficient vector $\beta$ is re-expressed as a smooth transformation $g(\mu)$ of a finite-dimensional population moment vector $\mu$ . This vector $\mu$ consists of the lower triangular entries of the regressor covariance matrix and the cross-moments between regressors and the response.
Random Smoothing Estimator: A smoothed estimator $\hat{\mu}_n$ is constructed by integrating the observed sample with an independent auxiliary sample using kernel weights driven by a shrinking bandwidth $h_n$ . This estimator avoids direct long-run variance estimation.
Asymptotic Normality: Under assumptions of stationarity, ergodicity, and finite second moments, the paper establishes that the smoothed moment vector is asymptotically normal. By applying the multivariate delta method to the transformation $g$ , the regression coefficient estimator $\hat{\beta}_n$ inherits asymptotic normality.
Implementation Refinements:
- Scaled Estimator: To address the lack of a standard interior optimum in the mean-squared error expansion of the direct estimator, a scaled variant $\tilde{\beta}_n$ is introduced. This restores a tractable bias–variance tradeoff, enabling data-driven bandwidth selection.
- Bandwidth Selection: An adaptive bandwidth rule is proposed. For short-memory processes, an MSE-optimal rate is derived. For long-memory processes (where standard conditions may fail), a robust logarithmic rate ( $h_n \propto \log n / n$ ) is used, calibrated via the Geweke–Porter-Hudak (GPH) estimator of the memory parameter.
- Truncation: A mild truncation device is incorporated to stabilize the procedure when the estimated design matrix determinant is near zero, ensuring the estimator is well-defined in finite samples while maintaining asymptotic negligibility.

Key Contributions and Results
The paper provides theoretical guarantees and empirical evidence for the proposed method:

Theoretical Results: The authors prove that under stationarity, ergodicity, and finite second moments, the proposed estimator yields asymptotically valid Wald-type confidence regions and simultaneous confidence intervals without requiring a parametric dependence model or direct long-run covariance estimation.
Simulation Performance: Extensive simulations were conducted under various dependence structures, including ARMA, ARFIMA, copula-based Markov models, and fractional Gaussian noise, with both Gaussian and heavy-tailed margins.
- Joint Coverage: The proposed method demonstrates near-nominal coverage for joint confidence ellipsoids across a broad range of dependence structures.
- Comparison with HAC: Compared to Newey–West HAC and MAC procedures, the proposed method exhibits more stable joint calibration, particularly in settings with nonlinear dependence (extended FGM copulas) and strong persistence (ARFIMA, fractional Gaussian noise). While Newey–West methods may be more economical in region size under short-memory linear dependence, they tend to show less balanced marginal coverage and greater sensitivity to parameter configurations in persistent or nonlinear settings.
- Long Memory: In long-memory scenarios, the competing OLS-based procedures often undercover in smaller samples, whereas the random-smoothing approach maintains reliable joint performance as sample sizes increase.
Real-Data Application: The procedure was applied to a regression analysis of winter Beijing PM2.5 concentrations on weather covariates. The analysis confirmed the presence of serial dependence in residuals and provided joint confidence regions that remained stable across chronological blocks, illustrating the method's practical utility in realistic settings with dependent errors.

Significance
The paper claims that the primary significance of this work lies in providing a credible basis for regression inference when the serial dependence structure is unspecified. The random-smoothing framework offers a robust alternative to traditional HAC-based inference, particularly excelling in joint inference for settings characterized by persistent or nonlinear dependence. The method achieves reliable calibration without the need for explicit long-run variance estimation or parametric modeling of the error process, addressing a gap in the literature where existing procedures often rely on specific structural assumptions or suffer from sensitivity to tuning parameters in finite samples.