🤖 AI

Differencing the Diffusion Trajectory toward Uncertain Components for Time Series Forecasting

DiffDiff is a novel diffusion framework for time series forecasting that embeds predictability asymmetry directly into the diffusion trajectory by progressively shifting the noisy state from the target to its second-order differences, thereby enabling the model to focus its generative effort on uncertain components while leveraging historical continuity for anchored content.

Original authors: Chen Su, Yuanhe Tian, Yan Song

Published 2026-07-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chen Su, Yuanhe Tian, Yan Song

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Art of Guessing the Future

Imagine you are trying to predict the weather for next week. You look at the sky, check the wind, and see that it's been raining steadily for three days. You know with near certainty that it will probably rain tomorrow, too. That's the easy part: the "slowly changing" stuff. But predicting exactly when a sudden, chaotic thunderstorm might pop up? That's the hard part. In the world of science, this is called time series forecasting. It's the art of looking at a string of past data—like electricity usage, stock prices, or heart rates—and guessing what comes next.

For a long time, computers have been getting better at this, but they often struggle with the "uncertainty" part. They try to guess the whole future picture all at once, treating a predictable trend (like a steady rise in temperature) the same way they treat a wild, unpredictable spike (like a sudden market crash). Recently, a new type of AI called Diffusion Models has become a superstar at this. Think of these models like a sculptor who starts with a block of noisy, static-filled marble and slowly chips away the noise to reveal a perfect statue. In forecasting, the "statue" is the future data, and the "noise" is the randomness we can't predict. The goal is to teach the AI how to chip away the noise to reveal the most accurate future possible. But here's the catch: if you try to chip away the noise from the entire block at the same rate, you might accidentally smooth out the exciting, unpredictable details while wasting time on the boring, predictable parts.

The New Sculptor: DIFFDIFF

This paper introduces a new method called DIFFDIFF (Differencing Diffusion) that changes how the sculptor works. The authors realized that in time series, the future isn't just a blank slate; it's a mix of things we can easily guess based on the past and things that are pure chaos. If you look at a river, the water level might rise slowly and predictably (anchored by the past), but the ripples on the surface are wild and random. Standard diffusion models treat the whole river the same way, trying to "denoise" the slow rise and the wild ripples simultaneously. This is inefficient, like trying to fix a blurry photo by sharpening the sky and the moving birds with the exact same brush strokes.

DIFFDIFF changes the game by making the "chipping away" process smarter. Instead of treating the whole future equally, it uses a special mathematical trick called differencing. Imagine you are watching a car drive down a road. If you just look at where the car is, it's moving smoothly. But if you look at how fast it's speeding up or slowing down (its acceleration), you see the real, unpredictable changes. DIFFDIFF shifts its focus. As the AI tries to remove the noise, it gradually stops caring about the smooth, predictable parts of the future (because the past already tells us what those will be) and starts focusing intensely on the "jagged" parts—the rapid changes and fluctuations that are truly uncertain.

The paper shows that by doing this, the AI stops wasting its brainpower on things it already knows. It saves all its energy for the hard stuff: the unpredictable spikes and dips. To make sure it doesn't forget the smooth parts entirely, it uses a clever "gate" system. This gate acts like a traffic controller, letting the AI know when to pay attention to the past's smooth trends and when to ignore them to focus on the chaos.

What They Found

The researchers tested DIFFDIFF on seven different real-world datasets, including electricity usage, traffic flow, and weather patterns. They compared it against six other top-tier diffusion models. The results were clear: DIFFDIFF consistently outperformed the others.

Specifically, the paper found that:

It's better at the hard parts: The model excelled at predicting long-term futures (up to 720 time steps ahead), where uncertainty is highest.
It's efficient: It didn't need more computer power or time to run; it just used the power it had more wisely.
The "Gate" works: The system that decides when to focus on the past and when to focus on the chaos actually adapts to different types of data. For example, on electricity data (which is very regular and periodic), the gate stays open high throughout the process to keep the "differencing" focus active, but on wind data (which is highly stochastic and chaotic), the gate stays suppressed because the differential information there is mostly just noise.

The authors also ran experiments to prove that their method works by shifting the focus, not just by adding more complexity. They showed that if you remove the "differencing" trick, the model gets worse. If you remove the "gate," it gets worse. But when you put them together, the model concentrates its effort exactly where it's needed: on the parts of the future that are truly uncertain.

In short, DIFFDIFF teaches the AI to stop trying to guess the obvious and start focusing on the surprising. It's a bit like telling a student, "Don't waste time memorizing the definition of 'gravity' (which we already know); instead, spend your time figuring out how a specific, wobbly apple will fall." By doing this, the model produces forecasts that are not only more accurate but also better at showing us the range of possible futures, which is crucial for things like managing power grids or preparing for extreme weather.

Technical Summary: DIFFDIFF – Differencing the Diffusion Trajectory toward Uncertain Components for Time Series Forecasting

1. Problem Statement

Probabilistic time series forecasting aims to model the conditional distribution of future values given a historical observation window. While Denoising Diffusion Probabilistic Models (DDPMs) have emerged as a powerful framework for this task by modeling complex predictive distributions without restrictive parametric assumptions, they face a fundamental asymmetry in time series data that standard diffusion processes fail to address.

In time series forecasting, the future is not a blank slate; it continues the observed history. Consequently, predictive uncertainty is not uniformly distributed. Components strongly correlated with the history (e.g., smooth trends, seasonal patterns) carry little remaining uncertainty, whereas components the history under-determines (e.g., rapid fluctuations, regime shifts) concentrate the residual uncertainty. Existing diffusion-based forecasters often attempt to handle this by decoupling the target into deterministic and uncertain components before the diffusion process using external rules (e.g., freezing a point predictor for trends). However, this approach localizes the asymmetry outside the diffusion pipeline, leaving the corruption trajectory "blind" to which parts of the target are already anchored by history. Consequently, the diffusion model's capacity is wasted on reconstructing history-anchored content that could be supplied deterministically, rather than focusing on the high-uncertainty components where generative effort is most needed.

2. Methodology: DIFFDIFF

The authors propose DIFFDIFF (Differencing Diffusion), a framework that embeds the predictability asymmetry directly into the diffusion trajectory itself. Instead of external decoupling, DIFFDIFF makes the forward operator step-dependent, progressively shifting the noisy intermediate state from the target values toward their second-order differenced structure.

2.1 Step-Dependent Forward Process

The core innovation lies in the forward process $q(x_t | x_0)$ . Unlike standard diffusion which adds isotropic Gaussian noise, DIFFDIFF defines a generalized forward marginal:
$q(x_t | x_0) = \mathcal{N}(\sqrt{\bar{\alpha}_t} A_t x_0, (1 - \bar{\alpha}_t)I)$
where $A_t$ is a step-dependent transition matrix.

Progressive Differencing: $A_t$ is defined as $(1 - \lambda_t)I + \lambda_t D^2$ , where $D^2$ is a second-order differencing operator and $\lambda_t$ increases monotonically with the diffusion step $t$ .
Effect: Early steps ( $t \approx 0$ ) remain close to the original target $x_0$ . As diffusion progresses, $A_t$ increasingly suppresses low-frequency components (levels and trends) which are easily anchored by history, while preserving high-frequency components (fluctuations and curvature) which carry the residual uncertainty.
Extended Target: To ensure cross-boundary continuity, the target $x_0$ is defined as an extended window $[x_{label}, x_{fut}]$ , including a short overlap from the observed history. This allows $D^2$ to encode curvature at the history-future boundary.

2.2 Stage-Adaptive Conditional Denoiser

Since the forward process suppresses value-domain content in later steps, the reverse process requires a conditioning pathway that can re-supply this information. DIFFDIFF employs a stage-adaptive conditional denoiser that fuses two views of the history:

Value Domain: Encodes the raw history $x_{hist}$ (preserving level, scale, and trend).
Difference Domain: Encodes the first-order differences $\Delta x_{hist}$ (capturing local velocity and short-term fluctuations).

These two representations are fused via a timestep-adaptive gate $g$ :
$c = h_{val} + g \odot h_{diff}$
where $g$ is a sigmoid function conditioned on the diffusion timestep. This gate dynamically adjusts the contribution of the differential branch: it remains open when the noisy state retains more differenced structure (later steps) and suppresses the differential branch when the value-domain content alone suffices (early steps).

2.3 Training and Sampling

Training: The model minimizes a position-weighted reconstruction loss in the value domain ( $x_0$ -prediction), alongside auxiliary losses to predict instance normalization statistics (mean and variance) to recover absolute scales.
Sampling: The terminal distribution approaches a standard Gaussian $\mathcal{N}(0, I)$ , preserving compatibility with existing accelerated samplers (e.g., DDIM). The reverse process uses the step-dependent matrices $A_t$ to map predictions back to the correct signal representation at each step.

3. Key Contributions

Internalizing Asymmetry: DIFFDIFF is the first diffusion framework to embed the deterministic-uncertain asymmetry directly into the diffusion trajectory via a step-dependent forward operator, rather than relying on external pre-decomposition.
Frequency-Selective Corruption: By utilizing a second-order differencing operator in the forward process, the method creates a non-uniform Signal-to-Noise Ratio (SNR) profile. It attenuates low-frequency (history-anchored) components faster than high-frequency (uncertain) components, effectively reallocating the model's generative capacity to the most uncertain parts of the target.
Stage-Adaptive Conditioning: The introduction of a timestep-adaptive gating mechanism allows the denoiser to dynamically balance value-domain and differential history information, compensating for the suppression caused by the forward operator.
Theoretical Consistency: The authors prove that despite the non-Markovian nature of the step-dependent forward process, the reverse process maintains marginal consistency with the forward trajectory, ensuring valid sampling.

4. Experimental Results

The authors evaluated DIFFDIFF on seven diverse time series benchmarks (Electricity, ETTm2, Exchange Rate, Traffic, Solar-Energy, Weather, Wind) across four prediction horizons ( $H \in \{96, 192, 336, 720\}$ ).

Performance: DIFFDIFF outperformed six state-of-the-art diffusion baselines (including CSDI, TimeDiff, TMDM, NsDiff, MA-TSD, and D3U) on 53 out of 84 settings. It achieved the best Continuous Ranked Probability Score (CRPS) in the majority of cases.
Ablation Studies:
- The forward process (differencing) was identified as the primary source of improvement; variants without it performed significantly worse.
- The stage-adaptive gate was crucial; removing it or replacing it with fixed fusion mechanisms led to performance degradation, particularly on datasets with varying degrees of stochasticity.
- Second-order differencing ( $D^2$ ) proved more robust than first-order ( $D^1$ ) or third-order ( $D^3$ ) operators, striking a balance between capturing curvature and avoiding amplification of extrapolation errors.
Analysis: Spectral analysis confirmed that the forward process successfully shifts the retained signal energy toward high-frequency components as diffusion progresses. Furthermore, the gate dynamics were shown to adapt to dataset characteristics, keeping the differential branch open for periodic series (Electricity, Traffic) and suppressing it for highly stochastic series (Solar, Wind).

5. Significance and Claims

The paper claims that DIFFDIFF demonstrates that probabilistic time series forecasting benefits significantly from encoding predictability asymmetry directly into the diffusion trajectory. By reshaping the corruption path to progressively suppress history-anchored content, the model avoids wasting generative capacity on reconstructing what is already known. Instead, it focuses its representational budget on the "hard" components—the history-under-determined dynamics—while relying on the conditioning pathway to supply the anchored content.

The authors emphasize that this approach allows a single end-to-end diffusion process to be "aware" of the target's structure, achieving superior calibration and sharpness (lower CRPS) without the need for external decomposition rules or complex multi-stage pipelines. The method preserves compatibility with standard diffusion sampling techniques, making it a practical and efficient upgrade for existing diffusion-based forecasting frameworks.