🤖 machine learning

Information Bottleneck Learning for Faithful Time Series Forecasting Explanations

The paper introduces IB-Forecast, an inherently interpretable multivariate time-series forecasting framework that utilizes a budget-constrained information bottleneck to generate faithful, sparse explanations matching the accuracy of black-box models while significantly outperforming existing post-hoc and interpretable baselines.

Original authors: Xu Zheng, Wei Cheng, Zhuomin Chen, Mo Sha, Jingchao Ni, Dongsheng Luo

Published 2026-07-31

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Xu Zheng, Wei Cheng, Zhuomin Chen, Mo Sha, Jingchao Ni, Dongsheng Luo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but instead of a crime scene, you are looking at a stream of numbers that predict the future. This is the world of time series forecasting, a branch of science used to guess what will happen next in things like electricity usage, traffic jams, or weather patterns. To make these guesses, computers use complex "black box" models. These models are like brilliant but silent wizards: they can predict the future with incredible accuracy, but they won't tell you why they made that guess. They just give you the answer.

In recent years, scientists have tried to build "interpretable" wizards—models that show their work. However, there's a catch. Just because a model looks like it's showing its work (like pointing to a few numbers and saying "I used these!"), doesn't mean those numbers actually caused the prediction. It's like a magician waving a red scarf to distract you while the real trick happens in their other hand. We need a way to know if the explanation is faithful—meaning, if we actually remove the parts the model says it used, does the prediction fall apart? If the prediction stays the same, the explanation was a lie. This paper tackles the problem of building a forecasting model that is not only accurate but also guarantees that its explanation is the real reason for its prediction, without needing to run extra tests to prove it.

The Magic Trick: IB-Forecast

The authors of this paper, a team from universities and research labs, have built a new system called IB-Forecast. Think of it as a super-smart weather forecaster who doesn't just guess the temperature; they hand you a map showing exactly which clouds and wind gusts they looked at to make that guess. And here's the kicker: the map isn't drawn after the guess; the map is the very tool they used to make the guess in the first place.

The Problem with Current "Explanations"

Right now, if you want to know why a black-box model predicted a heatwave, you have to use "post-hoc" methods. These are like asking the wizard to explain themselves after the spell is cast. You might ask, "What if I change this number?" or "What if I hide that part of the data?" The wizard then recalculates and tells you how much the answer changed. This is slow, expensive, and often unreliable. It's like asking a chef, "Why did you add salt?" and them saying, "Well, if I didn't add salt, the soup would taste different," without actually telling you if the salt was the main reason the soup was good or if they just added it because they felt like it.

Other models try to be "interpretable by design," meaning they are built to show their work. But the authors found that these models often have a "fake" explanation. They might point to a pattern, but if you actually remove that pattern, the model still predicts the same thing. The explanation is just a decoration, not the engine.

The Solution: A Budget and a Mask

IB-Forecast solves this by using a concept called the Information Bottleneck. Imagine you are packing for a trip, but you have a strict rule: you can only fit 20% of your clothes in your suitcase. You have to choose the absolute most important items. If you pack the wrong things, you won't survive the trip.

In this paper, the "suitcase" is the model's memory, and the "clothes" are the data points from the past. The model is forced to look at a long history of data (like the last 96 hours of electricity usage) but is only allowed to "open" a small budget of data points—about 14% to 20% of them—to make its prediction.

Here is how it works, step-by-step:

The Background Noise (The Context): First, the model strips away the boring, predictable stuff. It knows that electricity usage goes up every morning and down every night. It also knows the average temperature for this time of year. It writes these "rules" down in a separate notebook called the Structural Context. This is like knowing the sun always rises in the east; you don't need to check the sky every time to know it's morning.
The Mystery (The Deviation): Once the model removes the predictable patterns, it's left with the "weird" stuff—the unexpected spikes, the sudden drops, the unique events that happened just now. This is the "deviation."
The Selection (The Mask): Now comes the magic. The model has to predict the future based only on these weird deviations. But it can't look at all of them; it has a strict budget. It uses a special "gate" (a digital switch) to decide which specific moments in the past were important enough to keep. It closes the gates on the boring stuff and opens them only for the critical moments.
The Prediction: The model makes its prediction using only the moments where the gates are open. If a gate is closed, that part of the past is treated as if it followed the normal rules (the Structural Context).

Because the model is forced to ignore the closed gates, the explanation (the list of open gates) is guaranteed to be faithful. If you take away the open gates, the model literally cannot make the prediction. The explanation isn't a guess; it's the only path the model has left.

What They Found

The authors tested IB-Forecast on real-world data, including electricity grids, weather stations, and traffic flows. They compared it to the best "black box" models and other "interpretable" models.

Accuracy: IB-Forecast was just as accurate as the top black-box models. It didn't sacrifice precision for the sake of being explainable. In fact, it often matched the best models in the world.
Faithfulness: This is where it shined. When the authors tested how "honest" the explanations were, IB-Forecast crushed the competition. Other models, even the ones designed to be interpretable, had explanations that were often wrong. If you removed the "important" parts they pointed to, their predictions barely changed. IB-Forecast's predictions, however, fell apart if you removed its selected parts, proving the explanation was real.
Efficiency: Unlike other methods that need to run hundreds of extra calculations to generate an explanation, IB-Forecast gives you the explanation for free. It happens in the same split-second it takes to make the prediction.

The Takeaway

The paper suggests that we don't need to choose between a smart model and a transparent one. By forcing the model to be "lazy" (using only a small budget of data) and by separating the predictable patterns from the unique events, we can build a system that tells the truth about how it thinks.

The authors show that accurate forecasts often only need a tiny fraction of the history—sometimes as little as 14% of the data points—to be spot on. The rest is just noise. IB-Forecast is the first tool that can find those critical 14% and prove, without a shadow of a doubt, that they are the reason the prediction was made. It turns the black box into a clear window, showing us exactly which pieces of the past shaped our future.

Technical Summary: Information Bottleneck Learning for Faithful Time Series Forecasting Explanations

Problem Statement

As time-series forecasting increasingly drives critical decisions in energy, transportation, and healthcare, understanding the historical data behind predictions has become as vital as the predictions themselves. Existing approaches face a dichotomy:

Interpretable-by-design models (e.g., N-BEATS, TFT, DLinear) reveal internal structures but offer no guarantee that these structures faithfully reflect the evidence driving predictions. Their interpretability is often evaluated primarily on forecasting error, with the faithfulness of their explanations rarely assessed systematically.
Faithfulness-oriented post-hoc methods (e.g., gradient-based saliency, occlusion, optimization-based masks) explicitly verify model behavior but are predominantly designed for classification tasks. When applied to forecasting, they decouple the explanation from the actual computation, often requiring the evaluation of perturbed inputs outside the training distribution. Furthermore, direct transfer of classification-oriented attribution is insufficient for forecasting, which must model persistent levels, scales, recurring temporal structures, and input-dependent biases across variables and future horizons.

The core challenge is to bridge this gap by creating a forecasting framework that is inherently interpretable, guarantees faithfulness by construction, and maintains state-of-the-art accuracy without incurring additional inference costs.

Methodology: IB-Forecast

The authors propose IB-Forecast, a self-interpretable multivariate time-series forecasting framework. The method decomposes forecasting into a learned periodic component and a residual component computed with explainable masks over input tokens.

1. Structural Context and Decomposition

Following classical time-series decomposition, the input $X$ is separated into a structural context $C$ and a deviation $D$ :

Structural Context ( $C$ ): Includes the per-channel level ( $\mu$ ) and scale ( $\sigma$ ) via instance normalization, and a learned seasonal profile ( $Q$ ) representing recurring patterns (e.g., daily or weekly cycles). This context is disclosed and fixed for the forecast.
Deviation ( $D$ ): The residual $D = (X - \mu)/\sigma - Q_{per}$ captures window-specific variations, local trends, and residuals that deviate from the seasonal baseline.

2. The Information Bottleneck Mechanism

The forecast depends on the history window through the disclosed context $C$ and a sparse set of deviation tokens selected by a binary mask $m$ .

Tokenization: The deviation $D$ is partitioned into patches across channels.
Stochastic Gating: A transformer-based gate network scores each token's relevance. During training, hard binary gates are sampled using a noisy relaxation (Concrete distribution) with a straight-through estimator to enable gradient flow.
Budget Constraint: An explicit information budget $\rho$ (controlled by a KL-divergence term in the loss) limits the fraction of tokens the model can "open." This forces the model to prioritize observations carrying the most forecasting information.
Gated Deviation Readout: The forecast is generated solely from the masked deviations ( $\bar{m} \odot D$ ) via a base network $f_{base}$ . If a token is closed, its contribution is zero in deviation space, effectively resetting it to the expected seasonal pattern at the current level and scale.

3. Training Objective

The model is trained end-to-end to minimize a composite loss:
$\mathcal{L} = \| \hat{Y} - Y \|_2^2 + \beta \sum \text{KL}(\text{Bern}(p) \| \text{Bern}(\pi)) + \lambda_b (p - \rho)^2 + \lambda_{tv} \text{TV}(m)$

Forecasting Loss: Standard MSE.
Sparsity Regularization: Encourages gates to be sparse (Bernoulli prior).
Budget Penalty: Enforces the user-defined sparsity budget $\rho$ .
Temporal Continuity: Encourages contiguous selections via Total Variation (TV) on the mask.

4. Faithfulness by Construction

The explanation is generated within the forward pass. The architecture enforces the Markov chain $X \to E \to \hat{Y} | C$ , where $E$ (the mask and selected deviations) is the only carrier of window-specific predictive information once $C$ is fixed. This ensures that the explanation is not a post-hoc approximation but the actual mechanism of the forecast.

Key Contributions

Problem Formulation: The authors formalize self-interpretable multivariate forecasting (Problem 1), requiring a single forward pass to jointly produce a forecast and a compact explanation under a predefined budget and disclosed structural context.
IB-Forecast Architecture: A novel framework that decomposes predictions into a learned periodic profile and a gated deviation readout. The gate acts as an information bottleneck, retaining forecasting-relevant information while limiting dependence on the full historical input.
Empirical Superiority: Extensive experiments demonstrate that IB-Forecast matches the accuracy of leading black-box models (e.g., TQNet, CycleNet) while providing faithful explanations at no additional inference cost. Under matched sparsity budgets, its native explanations consistently outperform gradient-based, occlusion-based, and optimization-based baselines.

Experimental Results

The framework was evaluated on standard multivariate benchmarks (ETTh1/2, ETTm1/2, Weather, Electricity) with look-back windows of 96 steps and horizons up to 720.

Forecasting Accuracy: IB-Forecast achieved the best MSE among interpretable models on all datasets and matched the state-of-the-art black-box TQNet in overall MSE (statistically tied) and MAE. It significantly outperformed other interpretable baselines like PatchDecomp, TFT, and DLinear.
Faithfulness: Using a matched-budget fidelity metric (comprehensiveness minus sufficiency), IB-Forecast achieved the highest scores across all datasets.
- It outperformed post-hoc explainers (Occlusion, Integrated Gradients, Saliency, TIMING) and test-time optimizers (DynaMask, ExtremalMask).
- Native explanations from other interpretable models (TFT, PatchDecomp) often exhibited poor faithfulness, with negative scores indicating that keeping their top-credited inputs failed to reproduce the forecast.
Sparsity vs. Accuracy Trade-off: The model maintains competitive accuracy while consulting only 14–20% of the historical input. The accuracy loss remains minimal (e.g., <3.4% MSE gap on ETTh1) even with tight budgets.
Ablation Studies: Removing the gated base or hard gates destroyed the explanation's faithfulness (scores dropped to -1.0 or 0.62) while maintaining similar accuracy, confirming that the gating mechanism is essential for faithfulness. Removing the seasonal profile $Q$ caused a significant 10% drop in MSE, highlighting its importance.
Synthetic Ground Truth: On a synthetic benchmark with planted causal drivers, IB-Forecast achieved the highest AUROC and AUR (Area Under Recall), successfully recovering the true support while rejecting decoy signals, whereas other methods struggled.

Significance and Claims

The paper claims that IB-Forecast provides a rigorous foundation for interpretable forecasting by ensuring high explanation fidelity without sacrificing predictive performance.

No Inference Cost: Unlike post-hoc methods that require multiple forward/backward passes or optimization steps, IB-Forecast generates explanations natively during the forward pass.
Architectural Guarantee: Faithfulness is guaranteed by design; the forecast depends only on the selected tokens and the disclosed context.
Efficiency: The method scales to high-dimensional datasets (e.g., Traffic with 862 channels) with minimal memory overhead.
Insight: The results suggest that accurate forecasts often require only a small fraction of historical input, and that existing interpretable models often fail to utilize their internal structures faithfully.

The authors conclude that certified explanations and matched-budget evaluation are essential for moving beyond "interpretable-by-design" models that lack verified faithfulness, offering a practical solution for high-stakes decision-making in time-series domains.