🤖 machine learning

TiRex-2: Generalizing TiRex to Multivariate Data and Streaming

TiRex-2 is a novel recurrent xLSTM-based time series foundation model that extends univariate forecasting to multivariate scenarios with future covariates by employing a memory-centric design for constant-cost streaming and a synthetic coupling pipeline for scalable pretraining, achieving state-of-the-art zero-shot performance.

Original authors: Patrick Podest, Marco Pichler, Elias Bürger, Levente Zólyomi, Bernhard Voggenberger, Wilhelm Berghammer, Daniel Klotz, Sebastian Böck, Günter Klambauer, Sepp Hochreiter

Published 2026-07-02

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Patrick Podest, Marco Pichler, Elias Bürger, Levente Zólyomi, Bernhard Voggenberger, Wilhelm Berghammer, Daniel Klotz, Sebastian Böck, Günter Klambauer, Sepp Hochreiter

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the weather. In the old days, you might have looked at just one thermometer to guess tomorrow's temperature. But real life is messier: the temperature, humidity, wind speed, and air pressure all change together, and sometimes you already know things about the future (like a scheduled storm front or a holiday that usually brings crowds).

The paper introduces TiRex-2, a new "super-forecaster" AI designed to handle this messy, multi-variable reality better than previous models. Here is how it works, broken down into simple concepts:

1. The Problem: The "Library" vs. The "Notebook"

Previous AI models for time series (like Chronos-2) are like a giant library. Every time a new piece of information arrives (like a new temperature reading), the AI has to walk back through its entire library of past data to find connections. As the history gets longer, this "walking" gets slower and slower, eventually becoming too heavy to handle for real-time updates.

TiRex-2 is different. It's like a smart notebook that keeps a running summary. Instead of re-reading the whole book every time, it just updates its current page. This allows it to keep working at the same speed, no matter how much history you give it. This is called "constant cost streaming."

2. The Superpower: Knowing the Future (Without Cheating)

In many real-world scenarios, you know some things about the future in advance. For example, you know a factory will shut down next Tuesday for maintenance, or you know a holiday is coming up.

The Challenge: If an AI looks at "future" data to predict the "present," it's cheating. It needs to know the future only for the things you already know about (like the holiday), but it must never peek at the future of the thing it's trying to predict (like the actual sales numbers).
The Solution: TiRex-2 uses a special "traffic light" system (technically called an asymmetric grouped attention).
- It looks at the "future-known" data (the holiday) from both directions (past and future) to understand the full context.
- But it puts a strict wall between that data and the "target" data (the sales). The target can see the holiday info, but the holiday info cannot "peek back" at the target's future. This ensures the prediction is honest and causal.

3. The Training: The "Mixologist"

To teach this AI how variables interact (e.g., how wind affects temperature), the researchers needed a lot of practice data. Real-world data with many variables is rare and hard to find.

The Trick: They built a "mixologist" pipeline. They took thousands of single, lonely time series (like just temperature, just sales, just traffic) and mixed them together on the fly to create fake, complex multivariate scenarios.
They used different "recipes" to mix them: sometimes they just added them together, sometimes they made one cause the other, and sometimes they created complex, non-linear relationships. This taught the AI to recognize patterns in how different variables dance together.

4. The Results: The Marathon Runner

The paper tested TiRex-2 in three main ways:

Zero-Shot Performance: They threw the AI at problems it had never seen before (different industries, different time scales). TiRex-2 beat the current state-of-the-art models on two major benchmarks (fev-bench and GIFT-Eval).
The Marathon (Streaming): They fed the AI a massive stream of data (32 million steps!). While other models slowed down or got confused as the history grew, TiRex-2 kept its speed and accuracy steady. It didn't get tired.
The Long Haul: When asked to predict very far into the future (long horizons), TiRex-2 used the "future-known" data (like the storm front) to stay accurate, while other models fell back to guessing blindly.

Summary

TiRex-2 is a new type of AI that:

Runs fast even with huge amounts of history (unlike the "library" models).
Knows the difference between what it can peek at (future holidays) and what it must predict (future sales), ensuring it doesn't cheat.
Learns from synthetic mixing, allowing it to understand how different variables interact without needing a perfect real-world dataset for every scenario.

The authors claim this is the first model to combine all these features: multivariate support, future-known covariates, strict causality, and constant-speed streaming.

Technical Summary: TiRex-2

Problem Statement

Real-world time series forecasting is inherently sequential and multivariate. Observations arrive continuously, variables evolve jointly, and a subset of covariates (e.g., calendar features, scheduled interventions) is often known in advance. While existing Time Series Foundation Models (TSFMs) like Chronos-2 and MOIRAI have demonstrated strong multivariate capabilities, they predominantly rely on Transformer architectures. These models suffer from two critical limitations in streaming scenarios:

Quadratic Complexity: Inference costs grow quadratically with context length due to attention mechanisms, and full-history recomputation is required as new observations arrive.
Covariate Handling: Many models struggle to integrate future-known covariates without violating strict causality over target variables, or they lack support for both past and future covariates simultaneously.

Conversely, strong univariate models (e.g., TiRex, Reverso) often rely on channel-independence assumptions, neglecting cross-variate dependencies essential for multivariate systems. There is a gap in scalable, multivariate foundation models that support streaming inference with constant cost while maintaining strict target causality and leveraging future-known covariates.

Methodology

The authors introduce TiRex-2, a recurrent, xLSTM-based foundation model designed to generalize univariate forecasting to multivariate settings with both past and future covariates.

Architecture

TiRex-2 employs a memory-centric design based on xLSTM (specifically alternating mLSTM and sLSTM blocks), enabling constant per-patch cost updates. The architecture alternates between two mixing blocks:

Time Mixer: Acts along the patch axis ( $L$ $L$ ) independently for each variate.
- Target and past-covariate tokens are processed by a strictly forward xLSTM.
- Future-known covariate tokens are processed by the same weight-tied xLSTM in reverse, and the forward and reverse passes are fused linearly. This allows future-known covariates to encode information from both before and after their position without violating causality for targets.
Variate Mixer: Acts along the variate axis ( $V$ $V$ ) using asymmetric grouped attention.
- It employs block-diagonal grouped attention to handle arbitrary numbers of variates without padding.
- Crucially, it uses an asymmetric mask where target queries can read covariate keys, but covariate queries cannot read target keys. This ensures that future-known covariates can influence targets, but targets cannot influence covariates, preserving strict causality.

Training and Data Generation

To support scalable multivariate pretraining without relying on scarce curated multivariate corpora, the authors propose a synthetic coupling pipeline. This pipeline:

Draws univariate series from large corpora.
Augments them (amplitude trends, censoring, spikes).
Couples them on-the-fly using diverse mechanisms: identity, functional coupling, linear mixing, cointegration, and both linear and nonlinear structural causal models (SCMs).
Applies post-processing (covariate enrichment, time warping, discretization) to create realistic multivariate training instances with controllable cross-variate dependencies.

Inference Efficiency

The recurrent nature of the xLSTM backbone allows for streaming inference. As new data patches arrive, the model updates its hidden state in constant time ( $O(1)$ per patch), contrasting with the linear ( $O(L)$ ) or quadratic ( $O(L^2)$ ) costs of attention-based models with KV caching or full recomputation.

Key Contributions

Recurrent Multivariate Foundation Model: TiRex-2 extends the univariate TiRex to jointly model multiple target variates with observed (past) and future covariates. It activates 38.4M parameters in univariate mode and an additional 44.1M for multivariate forecasting.
Strict Causality with Future Covariates: Through the combination of bidirectional xLSTM for covariates and asymmetric grouped attention, the model integrates future-known covariates bidirectionally while ensuring target streams remain strictly causal in a single forward pass.
Constant-Cost Streaming: The recurrent state enables incremental updates, allowing forecasts to be refreshed with constant-time computation per time step, independent of the context length.
Synthetic Coupling Pipeline: A data generation framework that constructs diverse multivariate samples from univariate corpora, explicitly modeling indirect, causal, and direct cross-variate dependencies to expand the effective training distribution.

Results

The authors evaluate TiRex-2 on zero-shot, streaming, and long-horizon forecasting tasks:

Zero-Shot Performance: TiRex-2 achieves state-of-the-art performance on fev-bench (probing covariate utilization) and GIFT-Eval (probing generalization across domains and frequencies), outperforming models like Chronos-2, MOIRAI-2, and TimesFM-2.5.
Streaming Stability: In streaming experiments up to 32M steps, TiRex-2 maintains stable forecast quality (MASE) far beyond its training context boundaries (8k post-training), showing no signs of saturation or drift.
Long-Horizon Forecasting: On the dysts benchmark (chaotic systems), TiRex-2 leads at short-to-moderate horizons, effectively extracting signal from state variables to maintain accuracy.
Covariate Shift Sensitivity: TiRex-2 retains useful covariate signals over a wider lag range compared to Chronos-2, demonstrating robustness to misaligned covariates.
Ablation Studies: Experiments confirm that future-known covariates provide consistent gains, and the asymmetric group attention and bidirectional mixing are critical components for performance.

Significance

The paper claims that TiRex-2 is the first time series foundation model to achieve the specific combination of:

Native multivariate support with both past and future covariates.
Strict target causality (preventing future targets from leaking into past predictions).
Constant-memory and constant-cost streaming inference.
State-of-the-art zero-shot performance.

By addressing the misalignment between Transformer-based scaling behaviors and the requirements of streaming forecasting, TiRex-2 offers a practical solution for high-stakes, continuous decision-making systems (e.g., flood mitigation, predictive maintenance) where efficiency and real-time updates are paramount. The authors note that while future-known covariates are typically static within a forecast window, the model handles dynamic updates effectively by shifting them into the causal region. Future work is suggested to explore dynamic covariate selection to further optimize inference costs.