🤖 machine learning

NVExplain: Explaining Time Series Forecasting with Latent Trajectory Analysis and Structure-Preserving Surrogates

NVExplain is a model-agnostic framework that enhances time series forecasting interpretability by modeling predictions as latent trajectories to generate horizon-specific, temporally coherent explanations via semantic flow and structure-preserving surrogates, achieving competitive faithfulness and stability with improved computational efficiency.

Original authors: Muyan Anna Li, Manikandan Ravikiran, Aditi Gautam

Published 2026-08-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Muyan Anna Li, Manikandan Ravikiran, Aditi Gautam

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern world, computers are increasingly tasked with predicting the future based on patterns found in the past. From forecasting electricity demand for a city to anticipating stock market shifts or tracking the spread of disease, these systems analyze long streams of data points that change over time. While these models have become remarkably accurate, they often operate as black boxes. They produce a number for tomorrow or next week, but they do not explain why they chose that number. This lack of transparency is a significant problem in high-stakes fields where a wrong guess can have serious consequences. To trust these predictions, experts need to understand which specific moments in the past history actually influenced the forecast. They need to know if the prediction for next week is based on what happened yesterday, or if it is relying on a pattern from three months ago.

A team of researchers at NVIDIA has developed a new way to open these black boxes, specifically for time-based predictions. They created a system called NVExplain, which does not require changing the original forecasting model or retraining it. Instead, the system watches how the model's internal "thoughts" change as it processes new information. Imagine the model as a traveler moving through a landscape of data; as the traveler steps forward in time, their internal state shifts. The researchers measure the size of these shifts to determine how much new information is altering the model's understanding. By tracking these changes, they can map out exactly which past moments are driving the prediction for any specific future time.

The core of this method involves treating the model's internal processing as a journey. As the model looks at a window of past data and moves it forward one step at a time, the researchers record the model's internal state at each step. They then measure the distance between one state and the next. A large jump indicates that the new piece of information being added to the window caused a significant change in the model's perspective. By collecting these measurements, the researchers build a map that shows how much influence each past moment has on the prediction for the near future versus the distant future. This map reveals that the factors influencing a prediction for tomorrow are often different from those influencing a prediction for next month.

To make these findings easy for humans to read, the researchers also tested a technique that creates slightly altered versions of the past data. They changed the data in ways that respected the natural flow of time, such as keeping seasonal patterns intact while shifting specific values. They then asked the model to predict the future for these altered versions. By comparing the results, they could fit a simple, clear rule that explains the model's behavior for that specific situation. This step acts as a translator, turning the complex internal measurements into a straightforward list of which past days mattered most.

When the team tested this approach on real-world data, including electricity usage, financial exchange rates, and weather patterns, the results were compelling. The method proved to be highly reliable in identifying the true causes behind the predictions. In tests involving electricity data and weather records, the new system correctly identified the influential past moments nearly 100 percent of the time, outperforming existing methods that rely on different mathematical tricks. It also worked much faster than some of the older techniques, which can take minutes or even hours to analyze a single prediction, while the new method completed the task in seconds.

However, the researchers also discovered that this clarity comes with a caveat. While the method of tracking internal changes worked consistently well across all datasets, the step that creates the simple, readable rules sometimes struggled when the data was highly complex and unpredictable. In these difficult cases, the simple rules were less accurate, suggesting that while the system can always see the internal shifts, translating those shifts into a simple story is harder when the underlying patterns are chaotic. The study concludes that the best approach is to use the internal tracking method as the primary tool for understanding the model, while using the simple rules only when the data behaves in a predictable, linear way.

This work provides a crucial step toward making artificial intelligence more trustworthy in critical fields. By showing exactly how a model connects the past to the future, it allows experts to verify that the system is using sensible logic rather than hidden biases. The researchers found that for many important datasets, the model's internal changes are stable and consistent, meaning the explanations they generate are robust. This gives decision-makers in energy, finance, and health a way to look under the hood of their forecasting tools, ensuring that the predictions they rely on are not just accurate, but also understandable.

Technical Summary: NVExplain

Problem Statement

Time series forecasting models are critical for high-stakes decisions in energy, finance, transportation, and industrial monitoring. However, despite the strong performance of modern deep learning and foundation-model forecasters, their internal reasoning remains opaque, limiting trust and reliable deployment. Existing post-hoc explainability methods (e.g., LIME, SHAP, Integrated Gradients) face unique challenges when applied to forecasting:

Temporal Violations: Simple perturbations often violate temporal continuity and seasonality.
Horizon Specificity: Multi-step forecasting requires horizon-specific reasoning, as the lags influencing near-term predictions differ from those affecting longer horizons.
Latent Evolution: Most methods operate in input space, failing to capture how information evolves through latent representations over time.

Current approaches like TimeSHAP and ShapTime estimate input-space importance via perturbation or sequence removal but do not explicitly model the geometry of latent temporal evolution.

Methodology

The authors propose NVExplain, a model-agnostic framework that reframes forecasting explanation as analyzing semantic evolution in latent space. The framework consists of four stages:

1. Latent Trajectory Construction

The model is treated as inducing a trajectory in latent space by embedding rolling context windows. For a context window $X_t$ of length $L$ , the model produces a forecast $\hat{Y}_t$ . The framework extracts a latent state $Z_\tau = \text{embed}(x_{\tau-L+1:\tau})$ for each rolling window.

Extension: The trajectory is extended using the model's own forecast to allow analysis over both historical and predicted segments.
Access: For encoder-based models, this uses encoder outputs or hidden states. For decoder-only or API systems without exposed states, the method functions as a grey-box component, relying on prediction queries for the surrogate stage while requiring a stable proxy for latent-flow analysis.

2. Semantic Flow Computation

To quantify how information evolves as the temporal window advances, the framework measures changes between consecutive latent states. The semantic flow magnitude at step $\tau$ is defined as:
$m_\tau = \|Z_{\tau+1} - Z_\tau\|_2$
This captures the rate of semantic change, indicating how strongly new information influences the representation. The trajectory can be smoothed via an exponential moving average to reduce noise sensitivity.

3. Lag–Horizon Attribution Matrix

To explain multi-step forecasting, the framework aggregates semantic flow signals to construct a lag–horizon attribution matrix. This matrix quantifies how each historical lag contributes to each specific forecast step (horizon).

Aggregation: A score $S(j, h)$ is computed by summing semantic flow magnitudes over temporal history, weighted by a horizon-dependent exponential kernel $w_h(a)$ . This kernel allows longer-term predictions to incorporate broader history.
Normalization: A horizon-wise softmax converts scores into a distribution over lags for each forecast horizon, enabling horizon-resolved temporal explanations.

4. Structure-Preserving Local Surrogates

To generate human-readable local summaries, the framework fits sparse local surrogate models using temporally coherent perturbations.

Perturbations: Generated via block-bootstrap resampling combined with Fourier-domain amplitude perturbations to preserve continuity and seasonal structure.
Surrogate Model: A sparse linear model $g_{c,h}(u) = \beta_0 + \beta^T u$ is fitted for each channel and horizon using locality-weighted $\ell_1$ regularization. Weights are computed based on the distance of perturbed samples from the original in latent space.

Key Contributions

The paper claims four primary contributions:

Semantic Flow: Introducing semantic flow as a primitive for latent-space explanation, quantifying changes between consecutive latent states.
Lag–Horizon Framework: Proposing a model-agnostic framework that decomposes forecasting behavior into horizon-resolved lag attributions.
Structure-Preserving Perturbations: Designing perturbation strategies that maintain temporal continuity and seasonal structure for time series explanations.
Specialized Diagnostics: Evaluating explanation quality using forecasting-specific faithfulness and stability diagnostics.

Experimental Results

The method was evaluated on the MOMENT forecasting model across four datasets (ETTh1, Exchange, ILI, Weather) covering diverse domains. It was compared against Random, Attention, Integrated Gradients (IntGrad), and TimeSHAP.

Faithfulness

Pass Rate (FPR): The semantic-flow variant ("Ours (flow)") achieved competitive or superior faithfulness. It matched TimeSHAP on ETTh1 (99.1%) and outperformed all baselines on Exchange (88.9%) and Weather (96.3%). On ILI, IntGrad scored highest (76.1%), but semantic flow remained competitive (67.1%) at a fraction of the computational cost.
AOPC & Gap: Semantic flow yielded the largest forecast degradation (AOPC) and strongest separation from control perturbations on ETTh1, Exchange, and Weather, indicating the identified lags carry substantial predictive signal.
Efficiency: The semantic-flow variant was substantially more computationally efficient than IntGrad and TimeSHAP (e.g., 1.95s vs 52.4s on ETTh1).

Stability

Embedding Stability: Datasets like Exchange, ILI, and Weather showed low embedding sensitivity (<0.02), indicating stable trajectories suited for semantic flow. ETTh1 showed higher sensitivity but still achieved high faithfulness.
Trajectory Smoothness: Weather exhibited the smoothest latent evolution, while ETTh1 showed the highest volatility. The authors note that smoother dynamics yield more reliable semantic flow estimates.

Surrogate Performance

The full pipeline ("Ours (full)"), which includes the sparse surrogate stage, remained strong on ETTh1 but degraded on Exchange, ILI, and Weather. This suggests that in highly nonlinear temporal regimes, sparse local surrogates may introduce approximation errors, even when the underlying semantic-flow attribution is faithful.

Significance and Claims

The paper asserts that NVExplain offers a forecasting-specific decomposition that converts rolling latent-state changes into horizon-resolved lag attributions. Its significance lies in:

Horizon-Resolved Explanations: Unlike global attribution methods, it provides explanations specific to each forecast step, addressing the unique nature of multi-step forecasting.
Latent Dynamics: By operating in latent space, it captures the evolution of information over time, which input-space methods miss.
Reliability: The inclusion of stability diagnostics allows users to identify regimes where interpretation is robust versus those requiring caution (e.g., unstable latent dynamics).
Efficiency: It achieves competitive faithfulness with significantly lower computational cost than standard post-hoc baselines.

The authors conclude that while semantic flow drives strong faithfulness, the surrogate stage involves a trade-off between explainability and fidelity in nonlinear settings. They recommend using the full surrogate explanation when local linearity diagnostics are acceptable, and relying on the semantic-flow matrix with stability diagnostics when the surrogate fit is unreliable.