🤖 machine learning

Global Explanations for Multivariate Time Series Forecasting Models via $K$ -Order Markov Approximations

The paper introduces KARMA, a novel method for generating global explanations for multivariate time-series forecasting models by constructing a $K$ -order Markov surrogate model that captures temporal dependencies and recovers causal structures more effectively than existing attribution techniques.

Original authors: Amadeo Tunyi

Published 2026-06-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Amadeo Tunyi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart, but completely opaque, crystal ball that predicts the future based on a stream of data (like weather, stock prices, or heart rates). You know it works well, but you have no idea why it makes the predictions it does. It's a "black box."

Most current methods for trying to peek inside this box are like asking, "What would happen if I magically erased this one piece of data?" The problem is, in time-series data (like a weather forecast), you can't just erase yesterday's temperature without breaking the laws of physics; today's weather is deeply connected to yesterday's. Erasing it creates a fake scenario that never happens in reality, leading to confusing or wrong answers.

Enter KARMA.

The paper introduces a new method called KARMA (K-Order Markov Approximation for Multivariate Time Series). Instead of trying to erase data, KARMA asks a simpler, more natural question: "If the system was in this specific state for the last few steps, what does the model expect to happen next?"

Here is how KARMA works, broken down into simple concepts:

1. The "Memory Limit" Detective (Pillar 1)

Imagine you are trying to guess the next word in a sentence. Do you need to remember the whole paragraph, or just the last few words?
KARMA acts like a detective to find the model's "memory limit." It tests the model to see: "How far back in history does this model actually need to look to make a good prediction?"

It finds a specific number, let's call it K.
If the model only needs the last 4 hours of data to predict the next hour, KARMA proves that looking back 100 hours is useless.
The Result: It tells you, "This model only cares about the last 4 steps. Everything before that is noise."

2. The "Certified Zero" Baseline (Pillar 2)

Once KARMA knows the model only looks back K steps, it does something powerful. It creates a "baseline" (a default starting point) that is mathematically proven to be the best fit.

Because the model ignores anything older than K, KARMA can say with 100% certainty: "The importance of any data older than K is exactly zero."
Most other methods say, "This old data has a tiny, tiny influence." KARMA says, "No, it has no influence." This is a "certified zero," which is a huge deal for trust.

3. The "Five-Layer" Explanation Map (Pillar 3)

Once KARMA has built this simplified "memory map" (called a Markov Surrogate), it generates a five-level report to explain the model's behavior, like peeling an onion:

Level 1: Who Matters? (Variable Importance)
- Analogy: In a band, who is the lead singer?
- KARMA ranks the variables (e.g., Temperature, Wind, Pressure) to see which ones drive the predictions the most. It tells you, "Temperature is the star; Wind direction is just background noise."
Level 2: When Does It Matter? (Lag Profiles)
- Analogy: How far back does the lead singer's voice echo?
- It shows how the influence of a variable fades over time. Does the model care about the temperature from 1 hour ago, or 3 hours ago? It reveals if the model is looking for "momentum" (things keep going up) or "mean reversion" (things bounce back).
Level 3: What are the "Special Moments"? (Regime Explanation)
- Analogy: When does the band switch from a ballad to a rock song?
- It identifies specific patterns in the past (histories) where the model behaves strangely or uniquely. For example, "When it's hot AND windy, the model predicts rain, but when it's hot AND calm, it predicts sun."
Level 4: What If We Change Things? (Interventional Effects)
- Analogy: If we force the drummer to play a different beat, how does the song change?
- It simulates changing a specific past value (e.g., "What if yesterday's wind was stronger?") and measures how much the prediction shifts. This builds a map of cause-and-effect relationships learned by the model.
Level 5: How Sure Are We? (Uncertainty)
- Analogy: Is the band playing a song they know by heart, or are they improvising?
- It measures two types of uncertainty:
  1. Aleatoric: Is the future inherently chaotic? (The model is unsure because the data is noisy).
  2. Epistemic: Has the model never seen this situation before? (The model is unsure because it lacks data).
- This acts as a "reliability badge," telling you when to trust the explanation and when to be skeptical.

How It Was Tested

The authors tested KARMA in two main ways:

Synthetic Data (The "Fake World"): They created a fake world where they knew the exact rules (the "true" cause and effect). KARMA was able to perfectly reconstruct these rules, finding every true connection and ignoring the fake ones. Other methods got confused by the complexity.
Real-World Data (The "Real World"): They used real weather data (Beijing PM 2.5) and financial data. KARMA performed as well as or better than the best existing methods at identifying which time steps were actually important for the prediction.

The Bottom Line

KARMA is a new way to explain time-series models that respects the fact that time flows in a sequence. Instead of breaking the timeline to see what matters, it builds a simplified, probabilistic map of how the model connects the past to the future. It gives you a ranked list of important factors, tells you exactly how far back the model looks, and even gives you a "trust score" for its own explanations.

Limitations mentioned in the paper:
The method works best when the number of variables and the "memory length" aren't too huge. If you have thousands of variables, the math gets very heavy (exponential growth), though the authors use tricks to keep it manageable for most real-world cases.

Technical Summary: KARMA – Global Explanations for Multivariate Time Series Forecasting via K-Order Markov Approximations

1. Problem Statement

The deployment of deep learning models (e.g., RNNs, TCNs, Transformers) for multivariate time series forecasting in high-stakes domains (finance, clinical monitoring, industrial sensing) faces significant opacity. Existing Explainable AI (XAI) methods, such as LIME, SHAP, and gradient-based attributions, are primarily designed for static, independent and identically distributed (i.i.d.) data. When applied to time series, these methods often violate fundamental temporal properties:

Independence Assumption: They implicitly assume timestamp features are independent, ignoring temporal autocorrelation and inter-variable Granger causality.
Off-Manifold Perturbations: Random perturbations in SHAP or LIME break sequential structures, creating counterfactuals that do not exist in the data distribution.
Local vs. Global: Gradient-based methods describe local geometry at a single point rather than the systematic conditional structure learned globally by the model.
Baseline Selection: Methods like TimeSHAP struggle with the "baseline selection problem" in sequential contexts, where the choice of baseline is non-trivial.

The paper argues that explanations for time series must respect the sequential and causal structure of the data, moving away from perturbation-based or gradient-based local approximations toward a probabilistic, global surrogate approach.

2. Methodology: The KARMA Framework

The authors propose KARMA (K-Order Markov Approximation), a framework that explains a black-box time-series predictor $f$ by constructing a K-order Markov surrogate model. The core premise is that the behavior of a complex predictor on a multivariate time series can be faithfully approximated by a Markov chain of a specific order $K$ .

The methodology is organized around three pillars:

Pillar 1: Markov Surrogate and Order Selection

Discretization: Continuous time series variables are discretized into $N$ bins using quantile boundaries derived from training data, creating a discrete state space $S = [N]^D$ .
Order Selection ( $K^*$ ): The method identifies the minimal history length $K^*$ that is predictively sufficient. This is achieved via a surrogacy predictive validity stopping rule. The algorithm tests increasing lag lengths $K$ by replacing the prefix of an input window with a baseline $b$ and measuring the prediction discrepancy $\hat{\Delta}_{pred}(K, b)$ .
Stopping Criterion: $K^*$ is the smallest $K$ where the minimum discrepancy falls below a tolerance $\epsilon$ . This certifies that the model is insensitive to inputs beyond lag $K^*$ .

Pillar 2: Compression and Certified Attribution

Model Compression: If $K^* < W$ (the model's input window), the model is provably insensitive to inputs beyond lag $K^*$ . This yields a compression ratio $W/K^*$ .
Certified Baseline ( $b^*$ ): The baseline $b^*$ is not analyst-chosen but model-certified; it is the specific prefix that minimizes prediction discrepancy when substituted.
Certified-Zero Attributions: For all lags $k > K^*$ , attributions are mathematically zero (within approximation error), resolving the ambiguity of "small" vs. "zero" importance found in other methods.

Pillar 3: Five-Level Global Explanation Hierarchy

Once the $K^*$ -order Markov transition kernel $\hat{T}^f_{K^*}$ is estimated, KARMA derives five layers of global explanation without additional oracle queries:

Level 1 (Variable Importance): Computes the total distributional influence of source variables on target predictions across all lags, ranking variables by their marginal total-variation influence.
Level 2 (Lag Profiles): Resolves influence by specific lag $k$ , revealing temporal dynamics (e.g., momentum, mean-reversion) for each variable.
Level 3 (Regime Distinctiveness): Identifies distinctive historical regimes (paths) where the model's conditional forecast deviates significantly from its typical behavior, using a marginal interdependence index.
Level 4 (Interventional Effects): Computes average interventional effects (AIE) and edge contributions to populate a model-induced causal graph, connecting directly to Level 1 importance.
Level 5 (Uncertainty Quantification): Provides reliability certificates:
- Aleatoric Uncertainty: Entropy of the predictive distribution (genuine model uncertainty).
- Epistemic Uncertainty: Variance of kernel estimates (flags histories with insufficient data coverage).

Kernel Estimation: To avoid the exponential complexity of estimating the full joint kernel ( $N^{DK^*}$ ), KARMA estimates $D$ factored marginal kernels. It employs a Strategy B (b-prefix Monte Carlo)* by default, which restricts estimation to the observed history support $H^+_{K^*}$ , scaling linearly with series length $T$ rather than exponentially with dimension.

3. Key Results

Synthetic Data (VAR Oracle)

Causal Structure Recovery: On synthetic Vector Autoregressive (VAR) data with known ground-truth causal edges, KARMA achieved perfect recall (1.00) in all but the most complex configuration (xlarge, where recall was 0.71). It maintained high precision, effectively excluding spurious edges.
Feature Ranking: KARMA was the sole top-ranked method (highest Kendall's $\tau$ ) on medium, large, and xlarge configurations. Competing methods (TimeSHAP, WinIT, Feature Occlusion) degraded as graph density increased.
Comparison: Integrated Gradients (IG) was excluded from synthetic tests as it reduces to exact coefficients on linear VARs, making the comparison degenerate.

Real-World Benchmarks

Experiments were conducted on three datasets (ETTh1, Beijing PM2.5, Exchange Rate) across three architectures (GRU, LSTM, TCN) using a temporal-aware lag-based AUC score ( $AUClag@25\%$ ).

ETTh1: KARMA achieved the highest scores across all architectures, notably outperforming the next-best method on TCN (0.71 vs. 0.49).
Beijing PM2.5: KARMA was competitive, matching or slightly trailing the strongest baselines.
Exchange Rate: All methods collapsed to near-zero scores. The authors interpret this correctly: the dataset approximates a random walk with little learnable temporal structure, meaning no method can identify timesteps whose removal shifts the forecast. This validates the metric's behavior rather than indicating a failure of the explainer.

4. Significance and Claims

The paper positions KARMA as a principled alternative to existing XAI methods for time series by:

Respecting Temporal Geometry: Unlike perturbation-based methods that break sequential structure, KARMA's Markov approximation operates within the native language of the domain (conditional dependencies).
Providing Statistical Guarantees: It offers certified-zero attributions for irrelevant lags and explicit reliability certificates (Level 5) for the explanations, a feature absent in prior work.
Global vs. Local: It provides a global explanation hierarchy derived from a single estimated kernel, rather than instance-level local approximations.
Model-Agnostic Validity: The selection of $K^*$ and the baseline $b^*$ are derived directly from the model's behavior, not analyst assumptions.

Limitations Acknowledged:
The authors note that the state space grows exponentially with $N^{DK^*}$ , making full kernel estimation challenging in high-dimensional settings. KARMA mitigates this via marginal kernels and sampling strategies, but scaling to very high-dimensional systems remains an open direction.

The paper concludes that the Markov approximation paradigm offers a robust, underexplored direction for XAI in sequential domains, applicable to finance, neuroscience, and genomics.