🤖 AI

In-Context Inpainting for Time Series Forecasting

The paper introduces ICI-Time, a novel framework that reframes time series forecasting as a visual inpainting task by converting data into area charts and leveraging pre-trained large vision models to achieve competitive performance without specialized architectures or fine-tuning.

Original authors: Thang Nguyen, Dung Nguyen, Romero Morais, Truyen Tran

Published 2026-08-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Thang Nguyen, Dung Nguyen, Romero Morais, Truyen Tran

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Time series forecasting is the art of predicting the future based on the past. It is the engine behind weather reports, electricity grid management, and disease tracking, where understanding how a value changes over time allows us to prepare for what comes next. For decades, scientists have built complex computer programs specifically designed to recognize these patterns in numbers. These programs are powerful, but they are also rigid; they usually require massive amounts of data and extensive training to learn a new task, and they often struggle when data is scarce or when the patterns shift unexpectedly. Meanwhile, a different kind of artificial intelligence has emerged, one trained on millions of images to understand the visual world. These systems can look at a picture and fill in missing parts, or recognize a cat in a photo they have never seen before, simply by using examples provided at the moment of the task. This ability to learn from context without needing to be retrained is a powerful tool, but it has remained largely unused for predicting numbers because numbers do not naturally look like pictures.

A team of researchers at Deakin University in Australia has now bridged this gap by teaching a computer to see time as an image. They introduced a new method called ICI-Time, which reframes the problem of predicting the future not as a mathematical calculation, but as a visual puzzle. Instead of feeding raw numbers into a specialized forecasting model, the researchers first convert the history of a data stream—such as daily flu cases or hourly temperature readings—into a simple line chart. They then arrange these charts into a grid, placing a complete example of a past pattern next to the current situation, leaving a blank space where the future should be. They ask a pre-trained vision model, a type of artificial intelligence that has already learned to understand images from the internet, to fill in that blank space. The model treats the missing future data as a missing part of an image and uses the visual example provided to "inpaint," or paint over, the empty area with a plausible continuation of the line.

Once the vision model generates the completed image, the researchers translate the drawing back into numbers. They carefully extract the line from the new image and convert its height and position back into the original data values. The entire process happens without the computer ever being taught specifically how to forecast time series. It relies entirely on the model's existing ability to recognize visual patterns and its capacity to learn from the examples shown to it in that single moment. The researchers tested this approach on diverse real-world data, including influenza cases collected over two decades, weather indicators like humidity and air pressure, and electricity transformer temperatures. They compared their method against the most advanced forecasting models currently available, which are built with complex architectures designed specifically for time-based data.

The results were surprising. In almost every test, the visual approach matched or outperformed the specialized models, particularly when measuring how close the predictions were to the actual values. The method proved to be exceptionally robust when data was limited. While the specialized models struggled and their errors grew significantly when trained on only a tiny fraction of the available data, the visual method remained stable and accurate. This suggests that the ability to recognize shapes and trends in an image is a more universal skill than the specific mathematical rules usually applied to time series. The researchers found that the key to success was not just turning numbers into pictures, but doing so in a way that preserved the scale and shape of the data perfectly, ensuring the vision model could read the chart as accurately as a human would.

This work challenges the long-held assumption that predicting the future requires specialized, data-hungry algorithms. It demonstrates that a general-purpose vision system, when given the right visual representation, can understand temporal dynamics just as well as, and sometimes better than, models built from the ground up for that specific purpose. The study suggests that the boundary between visual reasoning and time-based prediction is not as fixed as previously thought. By simply showing a computer an image of the past and asking it to complete the picture, the system can infer the future without needing to be retrained or fine-tuned. This opens a new path for forecasting in situations where data is scarce or where rapid adaptation to new types of data is required, proving that sometimes, the best way to understand time is to look at it as a picture.

Technical Summary: ICI-Time

Problem Statement

Time series forecasting is a fundamental challenge in machine learning, critical for applications in finance, epidemiology, and climate modeling. Despite advances in deep learning, existing methods typically rely on specialized temporal architectures (e.g., Transformers with specific attention mechanisms) and require extensive domain-specific training or fine-tuning. This reliance limits scalability and generalization, particularly in low-data regimes or when adapting to new domains. Furthermore, while foundation models in language and vision have demonstrated powerful in-context learning (ICL) capabilities, leveraging Large Vision Models (LVMs) for time series forecasting remains an open challenge due to the "modality gap"—temporal signals are not natively visual.

Methodology: ICI-Time Framework

The authors propose ICI-Time (In-Context Inpainting for Time series), a framework that reframes time series forecasting as a visual inpainting task. The core hypothesis is that pre-trained generalist vision models can solve temporal prediction tasks without architectural modification or task-specific training if the data is presented in a suitable visual format.

The framework operates through four distinct stages:

Visual Case-Based Selection:
The system constructs a searchable database from historical data by partitioning time series into overlapping windows, each containing a lookback component ( $L_I$ ) and a prediction component ( $L_P$ ). For a given query window, the system retrieves the most similar historical example pair based on Euclidean distance between normalized inputs. This mimics case-based reasoning, where the LVM acts as a nonlinear interpolator in image space.
Time Series to Visual Representation:
Numerical time series are transformed into structured area charts (images).
- Normalization: The system employs a bidirectional mapping to ensure invertibility. Crucially, it uses a transfer scaling strategy ( $h_{transfer}$ ): the vertical axis range of the query image is determined by the local statistics of the query input plus the height scale derived from the retrieved example. This ensures visual consistency between the example and the query, allowing the model to recognize patterns across different magnitudes.
- Resolution: The input and target components are mapped to fixed 224×224 pixel images. The horizontal axis preserves the native temporal resolution of the input and target separately ( $w_{diff}$ ), rather than forcing a uniform resolution, to maintain temporal fidelity.
In-Context Visual Prompting:
The retrieved example pair (input-output) and the query (input with missing output) are arranged into a grid-like visual prompt. A pre-trained Large Vision Model (LVM), used off-the-shelf without fine-tuning, performs visual inpainting on this grid. The model infers the forecasting task from the example pair and "paints" the missing future region of the query image, effectively completing the pattern.
Time Series Recovery:
The generated image is converted back to numerical time series through a two-step process:
- Denoising: The image is binarized and processed with morphological operations to remove noise and artifacts.
- Curve Extraction: The boundary of the filled region is extracted column-by-column. To address the "disconnection problem" (errors at the boundary between input and prediction), a Gaussian smoothing decay is applied to blend the last input point with the predicted values.

Key Contributions

The paper outlines four primary contributions:

Cross-Modal Framework: A novel approach bridging temporal and visual domains, enabling rapid adaptation using off-the-shelf vision transformers without training.
Visual In-Context Learning Formulation: Extending ICL beyond NLP to a paradigm where temporal prediction is solved via visual prompts and inpainting.
Invertible Mapping: A carefully designed bidirectional mapping between time series and visual spaces that represents temporal dependencies via spatial layout while ensuring no loss of forecasting fidelity during transformation.
Empirical Validation: Strong performance across epidemiology (ILI), meteorology (Weather), and power systems (ETT) datasets, demonstrating competitiveness with deep learning baselines and superior robustness in low-data settings.

Experimental Results

The authors evaluated ICI-Time against Transformer-based baselines (Informer, Pyraformer, LogTrans) across multiple datasets and prediction horizons.

Full-Data Performance: ICI-Time outperformed baselines in the majority of cases, particularly on the Mean Absolute Error (MAE) metric (23 out of 24 cases). This confirms that off-the-shelf visual models can perform competitive forecasting without training.
Few-Shot Adaptation: In experiments restricting training data to 1%, 5%, and 10%, ICI-Time demonstrated remarkable stability. While baselines like Informer and Pyraformer saw significant error increases (e.g., +32% to +35% MAE increase as data dropped from 10% to 1%), ICI-Time's performance remained stable (e.g., +3.4% MAE increase). In extreme low-data scenarios (1%), ICI-Time often achieved errors multiples lower than the baselines, which struggled to converge.
Ablation Studies: The authors validated the importance of their design choices. The transfer scaling strategy ( $h_{transfer}$ ) and native resolution preservation ( $w_{diff}$ ) consistently reduced error compared to fixed scaling and uniform resolution. Post-processing smoothing also improved performance, particularly in reducing boundary errors.

Significance and Claims

The paper claims that ICI-Time introduces a new paradigm for time series analysis by demonstrating that visual reasoning models can generalize to temporal tasks when equipped with suitable representations.

Elimination of Training Overhead: The framework removes the need for domain-specific architectures, costly pre-training, or fine-tuning. It leverages the "free lunch" of pre-trained visual priors.
Data Efficiency: The results suggest that visual prompting is highly effective in low-resource settings, offering a robust alternative when labeled data is scarce.
Modality Transfer: The work challenges the conventional boundaries between temporal and visual modeling, suggesting that the rich pattern recognition capabilities of LVMs can be harnessed for time series analysis through careful representation design.

The authors conclude that this approach opens a new research direction in harnessing cross-modal transfer, with potential future extensions to anomaly detection and classification, though the current work focuses strictly on forecasting.