🤖 machine learning

End-to-end Early Classification of Time Series in Non-Stationary Environments

This paper challenges the limitations of existing separable designs for Early Classification of Time Series (ECTS) in non-stationary environments by introducing DQeND, a reinforcement learning-based end-to-end framework that jointly optimizes representation, classification, and triggering, demonstrating superior robustness and adaptability across various drifting scenarios compared to state-of-the-art baselines.

Original authors: Aurélien Renault, Alexis Bondu, Antoine Cornuéjols, Vincent Lemaire

Published 2026-08-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Aurélien Renault, Alexis Bondu, Antoine Cornuéjols, Vincent Lemaire

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of data science, time is often the most critical variable. Imagine a system that watches a stream of information unfold, second by second, like a doctor monitoring a patient's heartbeat or a security camera scanning a crowd. The goal is to make a decision as soon as the evidence is clear enough, rather than waiting for the entire story to finish. This is the challenge of early classification: predicting the outcome of a sequence before it is complete. For decades, researchers have built systems that treat this as two separate jobs. First, a machine learns to recognize patterns in the data. Second, a different mechanism decides when to stop watching and make a call. This approach works well when the world is stable, but it struggles when the rules of the game change. In the real world, data is rarely static; the patterns we rely on can shift, fade, or disappear entirely as time passes. When the environment evolves, a system that learns the "what" and the "when" separately often fails to adapt quickly enough, leaving it stuck with outdated rules.

A team of researchers at Orange Research and AgroParisTech set out to test a different idea: what if the machine learned to recognize patterns and decide when to stop at the same time? They built a new system called DQeND, which treats the entire process as a single, unified task. Instead of training one part to find the signal and another to pull the trigger, they trained a single brain to do both simultaneously. To see if this approach could handle a changing world, they put their system through a rigorous test. They created thousands of synthetic time series, like digital streams of numbers representing a digit being drawn, and then deliberately altered the rules of the game as the system learned. In some tests, the important part of the signal slowly moved from the beginning of the stream to the end. In others, the noise in the data grew louder, or the very definition of the categories changed unexpectedly. They compared their new, unified system against the best existing methods, which kept the recognition and decision-making parts separate.

The results showed a clear advantage for the unified approach. When the data changed gradually, the new system adapted smoothly, shifting its strategy to wait longer for clues that had moved later in the sequence, or to make quicker guesses when the signal became too noisy to trust. The separate systems, by contrast, often stumbled. Because they were trained to optimize the recognition and the timing independently, they could not easily coordinate a change in strategy. When the patterns moved, the recognizer was still looking in the wrong place, and the decision-maker was still pulling the trigger too soon. The unified system, however, learned that the best time to stop was tied directly to what it was seeing, allowing it to relearn the optimal moment to act as the environment shifted.

The researchers also tested how the system handled sudden, jarring changes, where the rules flipped overnight. Here, the unified system again proved more robust, quickly abandoning its old habits and learning the new timing required. They found that the key to this success was the ability to update both the way the system saw the data and the way it made decisions at the same time. When they froze one part of the system and only let the other learn, performance dropped significantly. This confirmed that the two skills are deeply intertwined; you cannot simply update the decision-maker if the way it sees the world has not also changed. The study suggests that for systems operating in dynamic, unpredictable environments, the old method of separating the "what" from the "when" is a limitation. By learning them together, machines can become more flexible, maintaining their accuracy even as the world around them evolves.

Technical Summary: End-to-end Early Classification of Time Series in Non-Stationary Environments

Problem Statement
Early Classification of Time Series (ECTS) involves making accurate class predictions as early as possible in an online setting, balancing the trade-off between predictive accuracy and earliness. While significant progress has been made in ECTS, most existing methods operate under the implicit assumption of a stationary data-generating process. In practical deployments, however, time series distributions often evolve due to covariate shifts or concept drifts. Current ECTS literature predominantly utilizes a separable design, where the classification module (predicting the label) and the triggering module (deciding when to predict) are optimized independently. The authors argue that this decoupling limits adaptability in non-stationary environments, as the optimal decision boundaries and stopping times may evolve in a coupled manner that independent optimization cannot capture.

Methodology: DQeND
To address these limitations, the authors propose DQeND, a unified, end-to-end architecture based on Reinforcement Learning (RL) that jointly learns representation, classification, and triggering decisions.

Architecture:
- Encoder: The pipeline utilizes an Extreme Learning Machine (ELM) inspired encoder. It employs a single-layer convolutional network with frozen, randomly initialized weights (similar to the ROCKET family) to extract features. To handle the time-dependent nature of ECTS, the encoder concatenates these signal features with an explicit time-step embedding.
- Decision Module: A Deep Q-Network (DQN) acts as the decision agent. Unlike separable approaches that rely on handcrafted features or classifier confidence scores, the DQN operates directly on the learned latent representations. The action space is unified to include "wait" and "predict class $k$ " options, allowing the agent to simultaneously determine the stopping time and the class label.
Training and Adaptation:
- Offline Training: The model is trained on a fixed collection of labeled time series using a reward function that penalizes both misclassification and delay costs. The gradient of the Bellman error propagates through the entire pipeline, updating both the encoder and the decision module.
- Online Deployment: The framework supports online adaptation via an $\epsilon$ -greedy strategy. Both the encoder and decision modules are jointly updated based on feedback from incoming data, allowing the representation itself to evolve in response to distribution shifts.

Experimental Protocol
The study evaluates DQeND against state-of-the-art separable baselines (specifically variants of the Alert framework and MiniROCKET-based classifiers) and other end-to-end methods (ELECTS, EARLIEST).

Dataset: The experiments use a synthetic MNIST-1D dataset (binarized) to allow precise control over drift characteristics.
Drift Scenarios: Four controlled non-stationary scenarios were tested:
1. I_loc: Incremental shift of discriminative patterns from the start to the end of the series.
2. I_noise: Incremental increase in Gaussian noise intensity.
3. A_loc: Abrupt shift of patterns from the first quarter to the last quarter of the series.
4. A_class: Abrupt reshuffling of sub-classes defining the meta-classes.
Metrics: Performance is measured via cumulative cost, hold-out AvgCost on drifted data, and hold-out AvgCost on the original distribution (to assess the stability-plasticity trade-off).

Key Results

Robustness to Drift: Across both incremental and abrupt drift scenarios, end-to-end approaches (particularly DQeND) consistently outperformed separable baselines in terms of cumulative cost and stability under drift.
Joint Adaptation: In scenarios like I_loc and A_loc, separable methods (especially those with frozen triggers) struggled to adapt their stopping policies to the new temporal locations of discriminative patterns. DQeND successfully adjusted its triggering policy to delay predictions as patterns shifted later in the series.
Ablation Study: The study confirmed that jointly updating the representation and decision modules is critical. Freezing the decision module resulted in the most significant performance degradation, while freezing the encoder had a more limited impact in certain abrupt drift settings (e.g., A_class), suggesting that a stable representation can suffice if the decision policy adapts.
Trade-offs: While DQeND demonstrated superior adaptation, it exhibited higher computational costs compared to LSTM-based end-to-end methods (ELECTS, EARLIEST) due to the sample inefficiency of value-based RL. In the A_class scenario, DQeND showed some instability in its triggering policy under repeated abrupt shifts, whereas ELECTS maintained more stable earliness profiles.

Significance and Claims
The paper claims to provide the first systematic comparison between separable and end-to-end ECTS architectures under controlled non-stationary conditions. The primary contribution is demonstrating that end-to-end learning offers improved adaptation capabilities for ECTS in dynamic environments compared to traditional separable designs.

The authors argue that the assumption of independent optimization for classification and triggering is fundamentally limiting when distributions evolve. By unifying these tasks within a single RL framework, DQeND achieves a better balance between adaptation (responding to new data distributions) and retention (maintaining performance on the original concept). The results motivate a shift away from separable designs for ECTS in non-stationary settings and highlight the importance of jointly learning prediction and stopping decisions. The authors note that while their framework is robust, future work should explore theoretical analyses of the stability-plasticity trade-off and additional forms of non-stationarity, such as drifting cost functions.

Technical Summary: End-to-end Early Classification of Time Series in Non-Stationary Environments

More like this