🤖 machine learning

Weighted Contrastive Learning for Anomaly-Aware Time-Series Forecasting

The paper proposes Weighted Contrastive Adaptation (WECA), a novel method that aligns normal and anomaly-augmented representations to significantly improve multivariate time-series forecasting accuracy under anomalous conditions while maintaining performance on regular data.

Original authors: Joel Ekstrand, Tor Mattsson, Zahra Taghiyarrenani, Slawomir Nowaczyk, Jens Lundström, Mikael Lindén

Published 2026-06-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Joel Ekstrand, Tor Mattsson, Zahra Taghiyarrenani, Slawomir Nowaczyk, Jens Lundström, Mikael Lindén

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Oscillating Thermostat"

Imagine you are trying to predict how much cash an ATM will need every day. Usually, the pattern is predictable: people withdraw money on Fridays, less on Tuesdays, and maybe a little extra on holidays.

Modern AI models are great at learning these "normal" patterns. They are like a thermostat that keeps a house at a perfect 70°F. But real life isn't always normal. Sometimes, a crisis happens (like a bank outage or a local festival), and suddenly, everyone rushes to the ATM. The demand spikes wildly.

The Dilemma:

The "Stiff" Model: If you train the AI only on normal days, it stays calm and accurate during normal times. But when a crisis hits, it ignores the chaos and keeps predicting "normal" numbers. The ATM runs out of cash, and people get angry.
The "Over-Reactive" Model: If you try to fix this by showing the AI examples of crises and telling it to "learn from them," it often goes too far. It starts treating every small change as a crisis. It forgets what "normal" looks like. Now, even on a quiet Tuesday, it predicts a massive rush, causing the bank to waste money keeping too much cash in the machine.

The paper calls this the Stability–Sensitivity Trade-off. You want a model that is stable enough to ignore minor bumps but sensitive enough to react when a real earthquake happens.

The Solution: WECA (The "Smart Filter")

The authors propose a new training method called WECA (Weighted Contrastive Anomaly-Aware Adaptation).

Think of the AI's learning process as a student studying for a test.

Standard Training: The student studies only normal days. They fail the "Crisis" section of the test.
Fine-Tuning (The old fix): The teacher shows the student a few crisis examples and says, "Memorize these!" The student memorizes them so well that they forget everything else. They fail the "Normal" section of the test.
WECA (The new approach): The teacher gives the student a special filter.
- When the student sees a small, harmless variation (like a slightly busier Friday), the filter says, "This is just a normal day. Ignore the noise and keep your prediction steady." (High Stability).
- When the student sees a huge, scary spike (a real crisis), the filter says, "This is different! Don't ignore it. Adjust your prediction immediately." (High Sensitivity).

How It Works (The "Weighted" Part)

The magic of WECA is in how it treats different types of data. It uses a mathematical "weight" (a score from 0 to 1) to decide how much the AI should learn from a specific example.

Mild Variations (Weight = 1.0): If the AI sees a day that is just a little different from normal, it forces the AI to treat it exactly like the normal day. This keeps the model stable. It doesn't get confused by small noise.
Severe Anomalies (Weight = 0.5 or lower): If the AI sees a massive, unrealistic spike (simulating a crisis), it tells the AI, "Don't force this to look like a normal day. Keep the difference." This preserves the signal needed to predict the crisis correctly.

It's like a dance instructor. If a student trips slightly, the instructor says, "Keep dancing, that was just a stumble." But if the music stops and the lights go out, the instructor says, "Stop dancing! Something big has changed!"

The Results: The ATM Test

The researchers tested this on a real-world dataset from 1,300 ATMs across Sweden over two years. They simulated crises by injecting "fake" demand spikes that looked like real-world disasters.

Here is what happened:

The "Stiff" Model: Did great on normal days but failed miserably when the fake crisis hit.
The "Over-Reactive" Model (Fine-Tuning): Did okay on the crisis but made the model terrible at predicting normal days. It forgot how to behave.
The WECA Model: It was the goldilocks solution.
- It stayed almost perfectly accurate on normal days (only a tiny, negligible drop in performance).
- It improved its accuracy on crisis days by a huge margin (6.1 percentage points better than the standard model).

The Takeaway

The paper proves that you don't have to choose between a model that is reliable on normal days and one that reacts to emergencies. By using this "weighted" approach, you can train a single AI that knows the difference between a "stumble" and an "earthquake," keeping the ATM stocked when it's needed most without wasting money when it's not.

Technical Summary: Weighted Contrastive Learning for Anomaly-Aware Time-Series Forecasting

Problem Statement
Deep time-series forecasters achieve high accuracy on stationary, routine data but frequently fail in production environments when data distributions shift abruptly due to anomalies (e.g., crises, outages, or sudden demand spikes). In critical applications like ATM cash logistics, such failures lead to service outages or capital inefficiency. The core challenge identified is a stability–sensitivity trade-off: a model must remain accurate under routine variations (stability) while correctly responding to genuine anomalies that alter the forecast trajectory (sensitivity).

Standard approaches struggle to resolve this:

Fine-tuning on anomaly-augmented data often leads to catastrophic forgetting, where the model recovers performance on anomalous data but degrades significantly on normal data.
Standard Contrastive Learning aligns normal and augmented samples to learn robust representations. However, enforcing strict alignment between normal and anomaly-augmented samples can suppress the very anomaly signals required for accurate forecasting during shifts.

Methodology: Weighted Contrastive Anomaly-Aware Adaptation (WECA)
The authors propose WECA, a training objective designed to learn representations that are invariant to benign variations but sensitive to significant anomalies. The method operates within a standard encoder-decoder forecasting framework ( $g_\phi$ and $h_\psi$ ) and introduces a weighted contrastive loss to the training objective.

Weighted Alignment: Instead of enforcing full invariance between an original input $x_i$ $x_{i}$ and its anomaly-augmented counterpart $\tilde{x}_i$ $\tilde{x}_{i}$ , WECA scales the contrastive alignment based on perturbation severity.
- Mild variations: Assigned a high similarity weight ( $w_{(i,t)} \approx 1$ ), enforcing strong alignment to ensure stability.
- Severe anomalies: Assigned a reduced weight ( $w_{(i,t)} < 1$ ), allowing the representations to diverge. This preserves the distributional signal necessary to forecast the shifted trajectory accurately.
Loss Function: The method utilizes an InfoNCE-style objective where the contrastive loss $\ell_{WECA}$ is scaled by the weight $w_{(i,t)}$ . The total training objective combines the standard forecasting loss (MAE) with the weighted contrastive term, controlled by a hyperparameter $\lambda$ .
Anomaly Injection: The model is trained on a dataset augmented with synthetic anomalies parameterized by amplitude, duration, and decay, calibrated to replicate high-impact historical events (e.g., demand spikes/dropouts).

Key Contributions

Formalization: The paper formalizes anomaly-aware forecasting as a dual objective: learning representations invariant to benign variations while remaining sensitive to anomalies.
Algorithmic Innovation: It introduces a weighted contrastive loss that dynamically controls alignment strength, enabling a tunable invariance–sensitivity trade-off without architectural changes.
Empirical Validation: It demonstrates that this approach improves accuracy under distribution shifts on real-world cash-demand data without degrading performance during normal operation.

Experimental Results
The method was evaluated on a nationwide ATM transaction dataset covering ~1,300 locations over two years. The backbone model used was TimesNet.

Baselines: The study compared WECA against:
- NT (Normally Trained): Trained only on normal data.
- FT (Fine-Tuning): Adapted on anomaly-augmented data.
- CL-IL (Standard Contrastive Learning): Instance-level contrastive learning with uniform weights ( $w=1$ ).
Performance Metrics (SMAPE):
- Fine-Tuning (FT): Improved performance on anomaly-affected data (AD) by 7.22 points but degraded normal data (ND) performance by 2.77 points, confirming catastrophic forgetting.
- Standard Contrastive (CL-IL): Improved AD performance by 4.82 points with a negligible impact on ND (-0.11 points).
- WECA: Achieved the optimal balance. It improved AD performance by 6.13 percentage points (reducing SMAPE from 37.91% to 31.78%) while leaving ND accuracy virtually unchanged (SMAPE change of only 0.03 percentage points).
Contrastive Axis Analysis: The authors found that Instance-level contrastive learning (aligning entire sequences) provided a stronger inductive bias for this task than Temporal contrastive learning (aligning individual timesteps), as the latter risked over-smoothing abrupt anomaly onsets.

Significance and Claims
The paper claims that WECA resolves the stability–sensitivity trade-off at training time, eliminating the need for post-deployment adaptation or architectural modifications. The results suggest that anomaly robustness need not be traded off against normal-regime reliability. By weighting the contrastive alignment according to perturbation severity, the model learns to ignore noise while retaining the capacity to react to genuine distribution shifts. This is particularly significant for industrial deployments where forecasting errors directly translate to operational failures or capital inefficiency.

Limitations
The authors note that the anomaly injection used is synthetic, calibrated to historical events rather than drawn from observed deployment failures. Future work is suggested to validate the method against real-world anomalies (e.g., gradual drift, sensor failures) and to explore learned or task-adaptive weighting schemes.