Technical Summary: InA-Probe

Problem Statement

Accurate long-term time series forecasting (LTSF) is critical for domains ranging from power grid management to meteorology. While specialized deep learning architectures (e.g., iTransformer, PatchTST) have established strong benchmarks, they often rely on passive numerical mapping. Concurrently, the "Time-as-Language" paradigm has emerged, leveraging pre-trained Large Language Models (LLMs) to recognize temporal patterns. However, existing LLM-based forecasters suffer from a passive processing bottleneck. They typically map temporal patches unidirectionally into the linguistic latent space via coarse-grained alignment or static task reprogramming. Consequently, the frozen LLM backbone acts as a passive recipient rather than an active reasoner, failing to:

1. Capture fine-grained, non-stationary temporal patterns.
2. Adapt to nuanced task intents or local signal regimes.
3. Ground numerical fluctuations in the physical concepts they represent.

This limitation hinders the model's ability to track non-stationary dynamics and adapt to differing task intents, precisely where pre-trained reasoning priors should be most effective.

Methodology: InA-Probe

The authors propose InA-Probe (Instruction-aware Active Probing), a framework that shifts the paradigm from passive alignment to an active, instruction-driven probing mechanism. The architecture transforms the frozen LLM into a context-aware forecaster through three synergistic components:

1. Multi-Level Instruction Injection

To bridge the gap between numerical signals and semantic reasoning, the model injects linguistic instructions at two granularities:

• Global Task Instruction: A prompt conveying dataset-specific context, forecasting horizon, and high-level objectives. This serves as a semantic anchor for the model's understanding of the data distribution.
• Patch-level Hybrid Prompts: For each temporal patch, a structured prompt encodes four dimensions:
  • Temporal Position: Location within the lookback window (e.g., Start, Mid, End).
  • Pattern Semantics: Qualitative descriptors of shape (trend direction, volatility).
  • State Level: Categorical indicators of relative magnitude (e.g., Extreme High).
  • Value Statistics: Quantitative descriptors (local slope, value range) grounded in Z-score normalized data.
• Note: These patch-level prompts are used exclusively during training to supervise the encoder via contrastive alignment and are discarded during inference.

2. Adaptive Query Generation (AQG)

Instead of using static, shared query vectors, InA-Probe generates sample-specific probes dynamically modulated by the temporal context. The AQG module operates in three stages:

• Time Context Extraction: An attention pooling mechanism summarizes the encoded patches into a global context vector representing the window's statistical properties and evolutionary dynamics.
• Dynamic Modulation: Learnable base query prototypes are modulated via a FiLM-based mechanism (scaling and shifting) derived from the context vector, aligning them with the current data distribution.
• Selection Gating: An instance-wise gating mechanism adaptively weights the modulated queries, allowing the model to prioritize specific features (e.g., sudden spikes vs. stable trends) relevant to the specific sample.

3. Instruction-Aware Active Probing (TL-Connector)

The generated adaptive queries enter the Time-Language (TL-) Connector, which employs a dual-stage attention process to interrogate temporal representations:

• Instruction-Aware Self-Attention: The adaptive queries are concatenated with global instruction embeddings. Self-attention allows the queries to internalize the overarching task intent and semantic anchors.
• Temporal Cross-Attention: The instruction-aware queries then selectively interrogate the projected temporal representations (keys and values) to extract salient patterns relevant to the internalized intent.
• Generative Decoding: The refined queries are fed into the frozen LLM for autoregressive next-token reasoning. A de-tokenization head maps the hidden states back to the numerical space to produce the final forecast.

Optimization

The framework utilizes a joint optimization objective during training:

• Forecasting Loss (MSE): Minimizes the discrepancy between predicted and ground-truth values.
• Patch-wise Contrastive Alignment: An InfoNCE loss aligns each temporal patch representation with its paired linguistic descriptor (from the patch-level prompts). This forces the encoder to embed temporal patterns close to their semantic descriptions, bridging the modality gap.
• Inference: During inference, the model relies solely on the global task instruction and the learned probing mechanism, operating autoregressively to generate forecasts.

Key Contributions

1. Paradigm Shift: Proposes InA-Probe, moving LLM-based forecasting from passive sequence processing to instruction-aware active probing, effectively bridging the structural and semantic gap between numerical time series and linguistic reasoning.
2. Specialized Architecture: Designs a TL-Connector that internalizes task instructions to drive targeted feature interrogation, coupled with an AQG module for sample-aware probing and a patch-wise contrastive alignment objective for dense semantic supervision.
3. Performance: Demonstrates superior performance across seven diverse real-world benchmarks, achieving significant gains in both one-for-all generalization (single model for all horizons) and zero-shot transfer scenarios, while maintaining an optimized computational footprint.

Experimental Results

The paper evaluates InA-Probe on seven benchmarks (ETTh1/2, ETTm1/2, Electricity, Weather, Traffic) against state-of-the-art deep learning (e.g., PatchTST, iTransformer) and LLM-based baselines (e.g., TimeLLM, AutoTimes, TALON).

• One-for-All Protocol: A single model trained on a 96-step horizon is recursively applied to all other lengths. InA-Probe achieved the best score on 11 out of 14 averaged metrics across the seven benchmarks, significantly outperforming baselines that require horizon-specific training.
• Zero-Shot Transfer: In cross-domain scenarios (e.g., training on ETT-h1 and testing on ETT-m1), InA-Probe achieved the best performance in 25 out of 32 individual metric comparisons. In challenging transfers, it reduced forecasting error (MSE) by 30% to 37% compared to the strongest baselines.
• Efficiency: The model maintains a single architecture for all prediction lengths, avoiding the computational cost of training separate models for each horizon.

Significance and Claims

The paper claims that InA-Probe successfully unlocks the reasoning power of LLMs for complex time series by enabling active interrogation rather than passive alignment. By synthesizing sample-specific queries that internalize task intents, the framework allows the frozen LLM to ground its pre-trained reasoning in the physical dynamics of the signal. The authors assert that the synergy between adaptive querying and fine-grained instructions is the key factor in achieving robust generalization and reducing errors in non-stationary, cross-domain scenarios, offering a promising route toward more unified and general-purpose time series forecasters.