💻 computer science

TS-ICL: A Flexible Time-Indexed Foundation Model for Time Series via In-Context Learning

TS-ICL is a novel time-indexed foundation model that leverages In-Context Learning to unify forecasting and imputation, achieving state-of-the-art performance on irregular and partially observed time series by formulating tasks as timestamp-aligned regression with synthetic causal dependencies.

Original authors: Etienne Le Naour, Tahar Nabil, Adrien Petralia

Published 2026-06-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Etienne Le Naour, Tahar Nabil, Adrien Petralia

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the weather, but your weather station is broken. Sometimes it stops recording for hours; sometimes it only gives you temperature but no wind speed; and sometimes the data comes in at weird, irregular intervals. Most current "super-smart" computer models (called Foundation Models) are great at guessing the future if the data is perfect and regular, but they stumble when the data is messy or missing.

This paper introduces TS-ICL, a new kind of AI model designed specifically to handle this messy reality. Think of it as a universal time-traveling detective that can solve two different mysteries at once: filling in the missing pages of a diary (imputation) and predicting what happens next (forecasting).

Here is how it works, using simple analogies:

1. The "Context" Detective (In-Context Learning)

Most AI models are like students who memorize a specific textbook. If you ask them a question about a topic they didn't study, they fail. TS-ICL is different. It uses a technique called In-Context Learning.

Imagine you are a detective trying to solve a crime. Instead of having a pre-written rulebook, you look at the clues right in front of you (the "context").

The Analogy: If you see a pattern of footprints leading to a door, you don't need to have seen that specific door before to know someone went inside. You just look at the footprints you have right now and figure out the pattern.
How TS-ICL does it: It takes the data you give it (the "look-back" window) and treats it like a mini-lesson. It learns the pattern from those specific clues and immediately applies that lesson to predict the future or fill in the gaps, without needing to be retrained for every new job.

2. The "Time-Indexed" Map (Handling Irregular Data)

Traditional models often force time series data into a rigid grid, like a spreadsheet where every row must be exactly one hour apart. If a sensor misses a reading, the whole row breaks, or the model has to guess where the missing row goes.

TS-ICL is like a GPS map rather than a spreadsheet.

The Analogy: A spreadsheet says, "Row 1 is 1:00 PM, Row 2 is 2:00 PM." If you miss 2:00 PM, the spreadsheet is confused. A GPS says, "You are at 1:00 PM, and the next point is at 2:15 PM." It understands that time is a continuous flow, not a set of fixed boxes.
The Benefit: Because TS-ICL understands specific timestamps, it can handle data that is missing, irregular, or arrives late without breaking a sweat. It doesn't need to "fix" the data first; it just reads the time stamps and moves on.

3. The "Sidekick" System (Covariates)

Sometimes, to predict the future, you need more than just the main story. For example, to predict electricity usage, you might need to know the temperature (a "covariate").

The Analogy: Imagine you are predicting how fast a car will go. The main data is the car's speed. But if you also know the wind speed and road slope (the sidekicks), your prediction becomes much better.
How TS-ICL does it: It has a special mechanism to listen to these "sidekick" signals. It can figure out if the wind speed actually matters for the car's speed, or if it's just noise. It learns to ignore irrelevant sidekicks and focus on the helpful ones, all while looking at the main story.

4. The Two-in-One Superpower

Usually, you need one tool to fill in missing data (imputation) and a different tool to predict the future (forecasting).

The Analogy: It's like having a Swiss Army knife that is both a screwdriver and a bottle opener, but works better than buying two separate tools.
The Result: TS-ICL unifies these tasks. It can look at a messy, incomplete history, fill in the missing pieces, and then immediately predict the future, all in one single step.

What the Paper Claims (The Results)

The authors tested this "detective" against the best existing models:

Filling in the blanks: TS-ICL is the new champion at filling in missing data. It beats the previous best models (called Tabular Foundation Models) significantly and does it 50 times faster.
Predicting the future: When it comes to forecasting, it is just as good as the top-tier models (like Chronos-2), even though it uses a different approach.
The "Broken Data" Test: The most impressive claim is its performance when the past data is broken. If you give a standard model a history with 50% missing data, its prediction gets terrible. TS-ICL, however, stays strong and accurate because it is built to handle irregular time stamps natively.

Summary

TS-ICL is a flexible, smart model that doesn't care if your data is messy, missing, or irregular. It treats time as a continuous flow, learns from the clues right in front of it, and uses extra information (like temperature or wind) only when it helps. It's currently the best at fixing broken data records and is a top-tier competitor for predicting the future, all while being much faster than its rivals when dealing with missing information.

Technical Summary: TS-ICL

Problem Statement

Time Series Foundation Models (TSFMs) have shifted the paradigm from task-specific models to general-purpose, zero-shot inference systems. However, existing approaches face three critical limitations in real-world applications:

Task Fragmentation: Most TSFMs focus exclusively on forecasting, failing to natively support imputation for missing values, which is common in irregularly sampled data.
Covariate Handling: Many models lack robust mechanisms for covariate-aware inference, despite the necessity of external information for many practical tasks.
Sampling Rigidity: Standard patch-based architectures (e.g., Chronos, TimesFM) assume regularly sampled inputs, hindering their ability to handle asynchronous or partially observed time series without preprocessing.

While Tabular Foundation Models (TFMs) adapted for time series (e.g., TabPFN-TS) support covariates and imputation, they often rely on handcrafted temporal features and suffer from high inference costs, lagging behind pure TSFMs in forecasting benchmarks. There is a need for a unified framework that jointly handles forecasting and imputation, supports covariates, and operates efficiently on irregularly sampled data.

Methodology

The authors introduce TS-ICL (Time-Series In-Context Learning), a probabilistic Transformer foundation model that reformulates time series modeling as a time-indexed in-context regression problem. The architecture consists of four sequential modules:

Time Series Encoder ( $E$ ): Based on a Perceiver encoder design, this module processes context timestamps and values (and optional covariates) channel-independently. It uses a fixed set of learnable tokens that cross-attend to timestamp-value pairs, compressing arbitrary-length inputs into a compact latent sequence ( $Z_{val}$ ).
Channel Mixer ( $M$ ): This module aggregates information across channels. It uses cross-attention to allow the target time series to query covariate tokens, collapsing the channel dimension into a unified, covariate-aware representation ( $Z_{final}$ ). Subsequent self-attention layers model global dependencies.
Temporal Context Query Module ( $C$ ): To bridge discrete latent tokens with the continuous time domain, this module embeds any query timestamp $t$ using frequency-based positional encoding. It cross-attends to $Z_{final}$ to produce a context-aware representation $H(t)$ for arbitrary timestamps, enabling flexible handling of irregular sampling.
In-Context Learning Regressor ( $R$ ): The final module treats prediction as an in-context regression task. It constructs a training set $D_{train}$ from observed context pairs $(H(t), x_t)$ and uses causal self-attention to infer target values at unseen timestamps. It outputs a dense set of quantile estimates trained via a smoothed pinball loss.

Data Prior and Training:
TS-ICL is trained on a structured data prior combining real-world datasets and synthetic data.

Univariate: A mixture of 31 real-world datasets and synthetic signals from TempoPFN, covering diverse domains and frequencies.
Multivariate (Covariates): Synthetic target-covariate structures are generated using Directed Acyclic Graphs (DAGs) and a registry of Structural Causal Models (SCMs). This ensures the model learns robust zero-shot generalization to unseen dependency structures, including informative, redundant, and independent covariates.
Tasks: Training dynamically switches between imputation (masking points or blocks) and forecasting (masking future horizons).

Key Contributions

Unified Architecture: TS-ICL is a novel probabilistic TSFM that unifies forecasting and imputation within a single continuous-time framework, supporting both tasks natively without architectural changes.
Structured Synthetic Prior: The authors design a DAG-based causal prior for target-covariate relationships, enabling the model to learn robust zero-shot generalization to complex dependency structures.
State-of-the-Art Imputation: TS-ICL achieves new state-of-the-art performance on zero-shot imputation benchmarks, outperforming both task-specific models and adapted TFMs.
Efficient Covariate-Aware Inference: The model supports efficient inference with covariates, handling arbitrary availability patterns (e.g., covariates observed only in the look-back window or both look-back and horizon).
Robustness to Missing Data: Unlike patch-based models, TS-ICL's time-indexed formulation allows it to remain robust when historical observations are partially missing.

Experimental Results

The model was evaluated on fm-impute-bench (imputation) and fev-bench (forecasting), with additional validation on the TIME benchmark.

Imputation: TS-ICL sets a new state-of-the-art on zero-shot imputation benchmarks. It improves pointwise (NMAE) and probabilistic (CRPS) scores over leading TFMs (TabPFNv2.5-TS, TabICLv2-TS) by significant margins (e.g., 17% and 15% improvement over TabICLv2-TS). Crucially, it is up to 50× faster at inference than these TFMs.
Forecasting: On forecasting benchmarks, TS-ICL remains competitive with leading TSFMs like Chronos-2 and TiRex, ranking among the top performers. It effectively leverages covariates when available, improving its relative ranking among TSFMs.
Robustness to Missing History: In experiments with increasing levels of missingness in the look-back window (30%–90%), TS-ICL consistently outperforms Chronos-2. While Chronos-2's performance degrades significantly as context becomes sparser, TS-ICL maintains stability, highlighting the advantage of its time-indexed formulation over patch-based approaches.
Efficiency: While TS-ICL is approximately 4× slower than the highly optimized Chronos-2, it is roughly 40× faster than TFMs, offering a strong accuracy-efficiency trade-off.

Significance and Claims

The paper claims that TS-ICL addresses the fragmentation in current time series modeling by providing a single architecture capable of unified forecasting and imputation with native covariate support.

Its primary significance lies in its robustness to incomplete historical observations. The authors demonstrate that patch-based models are sensitive to missing context, whereas TS-ICL's time-indexed formulation allows it to tackle realistic forecasting settings where data is irregular or partially observed. By combining the flexibility of in-context learning with a structured causal prior, TS-ICL offers a scalable alternative to specialized models, achieving state-of-the-art imputation performance while maintaining competitive forecasting capabilities.

The authors note limitations, specifically that the pointwise regression formulation incurs higher computational costs than highly optimized patch-based models (like Chronos-2), though this is mitigated by its superior speed compared to TFMs. They suggest future work could involve architectural optimizations (e.g., caching) and further diversification of the training data prior to enhance zero-shot generalization.