📊 statistics

Renewable estimation in linear expectile regression models with streaming data sets

This paper proposes a novel online renewable expectile regression method for streaming data that leverages the smoothness of the expectile loss function to achieve superior computational efficiency and minimal storage requirements while maintaining statistical consistency and asymptotic normality comparable to oracle estimators based on full data.

Original authors: Wei Cao, Shanshan Wanga, Xiaoxue Hua

Published 2026-02-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Wei Cao, Shanshan Wanga, Xiaoxue Hua

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to predict the weather, but the weather data is pouring in like a firehose. It never stops, it's massive, and you can't possibly save every single drop of rain that falls in your memory bank. You need a way to learn from the data as it arrives, update your robot's brain in real-time, and throw away the raw data to save space.

This is the challenge of Streaming Data.

The paper you provided introduces a new, smarter way to teach this robot using a method called ReER (Renewable Expectile Regression). Here is the breakdown in simple terms, using some everyday analogies.

1. The Problem: The "Checklist" vs. The "Smooth Curve"

Traditionally, statisticians used a method called Quantile Regression to look at the "tails" of data (like predicting extreme storms or record-breaking heat, not just the average temperature).

The Old Way (Quantile Regression): Imagine trying to draw a line through a jagged, rocky mountain path using a ruler. It's possible, but the path is so bumpy (mathematically "non-smooth") that the ruler keeps getting stuck. It takes a long time to calculate, and it's computationally expensive.
The New Way (Expectile Regression): The authors suggest using a different tool. Instead of a ruler on rocks, imagine using a smooth, flexible rubber sheet. You can stretch this sheet over the data, and it naturally settles into the best shape without getting stuck on the bumps. This is much faster and easier to calculate.

2. The Challenge: The "One-Shot" vs. The "Continuous Update"

There are other methods that try to handle streaming data, but they have a flaw.

The "One-Shot" Method: Imagine a student who takes a test every day. At the end of the week, they throw away all their daily notes and just look at the final test score to guess what they learned. They miss the nuance of how they improved day by day. This is what previous methods did; they treated each batch of data as a separate island and didn't connect the dots well enough.
The "Divide and Conquer" Method: This is like asking 100 people to solve a puzzle, then averaging their answers. It works okay, but if one person gets a small piece of the puzzle (a small batch of data), their answer might be wild and throw off the average.

3. The Solution: ReER (The "Smart Notebook")

The authors propose ReER, which acts like a smart, self-updating notebook.

Here is how it works:

Don't hoard the data: When a new batch of data arrives (a new day of weather), the robot doesn't save the raw numbers. It saves a tiny "summary" (like the average wind speed and a measure of how much the wind changed).
The "Renewable" Update: When the next day arrives, the robot doesn't start from scratch. It looks at its summary notebook from yesterday and combines it with the new summary from today.
The Magic Trick (Taylor Expansion): This is the technical part, but think of it like this: Instead of re-reading the entire history book to find the answer, the robot uses a mathematical shortcut (a "Taylor expansion") to guess how the old answer should change based on the new information. It's like adjusting a GPS route based on a new traffic report without needing to re-map the whole city.

4. Why is this better?

Speed: Because it uses the "smooth rubber sheet" (Expectile) instead of the "jagged ruler" (Quantile), it calculates answers much faster.
Memory: It only remembers the "summary" (a tiny list of numbers), not the millions of raw data points. It's like remembering the gist of a conversation rather than recording every word spoken.
Accuracy: Even if the new data comes in small, messy chunks (like a sudden storm), ReER is robust. It doesn't get confused by small batches because it constantly refines its understanding of the whole picture, not just the current moment.

5. Real-World Proof

The authors tested this on two real-life scenarios:

Beijing Air Quality: They predicted pollution levels using data from 12 different monitoring stations. ReER was just as accurate as if they had saved all the data, but it was lightning fast.
Electricity Usage: They predicted power consumption for a household. Even when they chopped the data into tiny, frequent chunks, ReER stayed accurate, while other methods got shaky.

The Bottom Line

If you have a river of data that you can't stop, and you can't save every drop, ReER is the best way to build a dam that captures the water's energy perfectly. It's fast, it doesn't need a massive warehouse to store water, and it gives you the most accurate prediction possible, even when the water flow is choppy or unpredictable.

In short: It's a super-efficient, memory-saving, and highly accurate way to learn from data that never stops flowing.

1. Problem Statement

The paper addresses the challenges of statistical modeling in streaming data environments, characterized by:

Volume and Velocity: Data arrives continuously in large volumes, making it impractical to store all historical raw data due to limited storage capacity.
Heterogeneity: Streaming data often exhibits heteroscedastic variances (non-constant variance) and inhomogeneous covariate effects.
Limitations of Existing Methods:
- Quantile Regression (QR): While effective for capturing distributional heterogeneity, QR relies on a non-smooth "check" loss function, making it computationally expensive and difficult to optimize in online settings.
- Existing Online ER Methods: Previous approaches for Expectile Regression (ER) in streaming data (e.g., Pan et al., 2024) often rely on "one-shot" strategies. These methods aggregate local estimators but fail to fully leverage cross-batch information, leading to inefficiency, especially when data arrives in small or unevenly sized batches. They also impose restrictive assumptions (e.g., batch sizes must diverge to infinity).

2. Methodology: ReER (Renewable Expectile Regression)

The authors propose ReER, a novel online renewable estimation strategy for linear expectile regression.

Core Concept

Instead of storing raw data or performing full re-optimization on every new batch, ReER updates the parameter estimates using:

The current data batch.
Summary statistics (sufficient statistics) from all previous batches.

Algorithmic Derivation

Loss Function Approximation: The method utilizes a Taylor expansion of the global loss function around the previous renewable estimator ( $\tilde{\beta}_{b-1}$ ). This allows the historical loss component to be approximated using the Hessian matrix and the previous estimate.
Iterative Update: The estimator $\tilde{\beta}_b$ for batch $b$ is derived by minimizing an approximated loss function that combines the current batch's loss with the historical curvature (Hessian).
Closed-Form Approximation: Since the exact solution involves the unknown current parameter, the method approximates the current parameter in the weighting terms with the previous step's estimate ( $\tilde{\beta}_{b-1}$ ). This yields a computationally efficient update rule:
$\tilde{\beta}_b = \left[ \sum_{t=1}^{b-1} W_{nt}(\tilde{\beta}_t) + W_{nb}(\tilde{\beta}_{b-1}) \right]^{-1} \left[ \sum_{t=1}^{b-1} W_{nt}(\tilde{\beta}_t)\tilde{\beta}_{b-1} + U_{nb}(\tilde{\beta}_{b-1}) \right]$
Where $W$ and $U$ are weighted sums of covariates and responses based on the asymmetric squared loss of ER.

Storage Efficiency

The algorithm requires storing only:

A $p \times p$ cumulative Hessian matrix ( $H_{Nb}$ ).
A $p \times 1$ vector of the current parameter estimate ( $\tilde{\beta}_b$ ).
This is in stark contrast to storing the full $N_b \times p$ data matrix, making it highly scalable.

3. Key Contributions

Novel Estimation Framework: Unlike "one-shot" aggregation methods, ReER employs a Taylor expansion-based update mechanism. This allows it to refine historical parameter estimates using new data, effectively integrating information across batches rather than just weighting local estimates.
Computational Efficiency: By leveraging the smoothness of the asymmetric squared loss function (unlike the non-smooth QR loss), ReER avoids complex non-parametric density estimation and achieves superior computational speed.
Robustness to Batch Size: The method does not require the assumption that individual batch sizes ( $n_t$ ) must diverge to infinity. It only requires the first batch size ( $n_1$ ) to be sufficiently large for a consistent initial estimator. This makes it suitable for real-world scenarios with small or variable batch sizes.
Theoretical Guarantees: The authors prove that under mild regularity conditions:
- The ReER estimator is consistent.
- It possesses asymptotic normality.
- It is asymptotically equivalent to the "Oracle" estimator (which has access to all raw data), achieving the same statistical efficiency.

4. Experimental Results

Simulation Studies

The authors compared ReER against:

Oracle: Full-data estimator (Gold Standard).
DCER: Divide-and-conquer ER (Song et al., 2021).
PAER: One-shot online ER (Pan et al., 2024).

Findings:

Accuracy: ReER achieved Mean Square Error (MSE) and Bias comparable to the Oracle estimator across various scenarios (homogeneous/heterogeneous variance, normal/heavy-tailed errors).
Small Batch Performance: In scenarios with small batch sizes ( $n_k$ ), DCER and PAER showed significant performance degradation (high MSE). ReER remained stable and accurate, demonstrating its ability to aggregate information across batches effectively.
Heavy-Tailed Data: ReER showed superior robustness compared to DCER when errors followed a Student's $t$ -distribution.
Computation Time: ReER was the fastest among the renewable methods, reducing computation time by approximately 2% compared to PAER and DCER, while being orders of magnitude faster than the Oracle method.

Real-Data Applications

Beijing Air Quality Dataset: Analyzed PM2.5 levels against covariates (temperature, pressure, pollutants) across 12 monitoring sites (treated as 12 streams). ReER matched the Oracle's predictive performance (MPE) while significantly reducing computational cost.
Household Electricity Dataset: Predicted active power based on reactive power over 2 million observations. ReER maintained stable prediction errors regardless of batch size variations, whereas competing methods fluctuated significantly.

5. Significance and Impact

Bridging the Gap: The paper successfully bridges the gap between the computational demands of streaming data and the statistical need to model distributional heterogeneity (via expectiles).
Practical Applicability: The method is implemented in an R function (ReER), making it accessible for practitioners dealing with high-velocity data in finance, sensor networks, and environmental monitoring.
Theoretical Advancement: It relaxes the strict assumptions of previous streaming literature regarding batch sizes, offering a more flexible framework for perpetual data streams where data arrival patterns are irregular.
Future Directions: The authors identify high-dimensional streaming ER and handling dynamic model heterogeneity (concept drift) as key areas for future research.

In summary, ReER provides a computationally efficient, storage-light, and statistically robust solution for analyzing streaming data with heteroscedasticity, outperforming existing online methods particularly in scenarios with small or irregular data batches.