CoGenCast: A Coupled Autoregressive-Flow Generative Framework for Time Series Forecasting

Technical Summary: CoGenCast

Problem Definition
Time series forecasting is fundamentally a generative problem requiring two distinct capabilities: deep semantic understanding of contextual conditions (e.g., historical patterns, domain knowledge, task instructions) and stochastic modeling of continuous temporal dynamics to capture uncertainty. Existing approaches typically fall into two categories, each with limitations. Large Language Model (LLM)-based methods excel at semantic context but often rely on simple regression mappings or lack robust probabilistic modeling of continuous dynamics. Conversely, diffusion-like models effectively capture stochastic evolution in continuous spaces but often struggle with the complex semantic conditioning required for diverse forecasting scenarios. Neither approach alone adequately models both aspects simultaneously.

Methodology: CoGenCast
The authors propose CoGenCast, a hybrid generative framework that couples pre-trained LLMs with a flow-matching mechanism to address these dual requirements. The framework operates through three core components:

1. Reconfigured Encoder-Decoder Backbone:
   Instead of training LLMs from scratch, CoGenCast reconfigures pre-trained decoder-only LLMs (specifically the Qwen family) into a native forecasting encoder-decoder backbone by modifying only the attention topology.

   • Encoder: Replaces causal self-attention with bidirectional self-attention. This allows the model to comprehensively understand contextual conditions by fusing look-back windows with text-based context features (domain knowledge, task instructions, statistics).
   • Decoder: Retains causal self-attention to ensure temporal causality during autoregressive generation. It introduces a cross-attention mechanism that queries the encoder's hidden states, allowing the generation of future representations to be conditioned on the rich semantic context.

2. Coupled Flow-Matching Mechanism:
   To model the continuous stochastic dynamics conditioned on the autoregressively generated representations, CoGenCast integrates a continuous flow-matching mechanism.

   • Interval-Conditioned Average Velocity: Unlike standard flow-matching or diffusion models that estimate instantaneous velocity at a single time point, CoGenCast explicitly models the average velocity over a finite time interval $[t, r]$. This is achieved by defining a linear probability path between Gaussian noise and the clean future patches.
   • Optimization: The model employs a Jacobian-Vector Product (JVP) corrected objective to guide the learning of this average velocity, reducing errors associated with local instantaneous estimation and coarse numerical integration.

3. One-Step Inference:
   A key architectural advantage is the ability to perform one-step generation. By learning an interval-conditioned average velocity, the transport process from noise to the clean prediction can be approximated within a single function evaluation (Euler step). This eliminates the need for iterative denoising, significantly reducing inference latency while maintaining high precision. During inference, future patches are generated autoregressively, with each step utilizing the one-step flow-matching mechanism.

Key Contributions

• Architectural Reconfiguration: The paper demonstrates that pre-trained decoder-only LLMs can be effectively repurposed as forecasting encoder-decoders solely through attention topology modifications, enabling both bidirectional context encoding and causal representation generation.
• Hybrid Framework: CoGenCast successfully couples the semantic reasoning capabilities of LLMs with the continuous probabilistic modeling of flow-matching, addressing the gap between semantic understanding and stochastic dynamics.
• Efficient Generation: The framework introduces a novel objective for interval-conditioned average velocity, enabling high-quality, low-latency one-step generation without iterative sampling.
• Multimodal and Cross-Domain Support: The framework naturally supports multimodal forecasting (numerical time series + textual context) and demonstrates effectiveness in cross-domain unified training settings.

Experimental Results
Extensive experiments were conducted on ten diverse benchmark datasets covering energy, environment, finance, health, and electricity domains.

• Performance: CoGenCast achieved competitive performance, consistently outperforming representative baselines across LLM-based (e.g., TokenCast, Time-LLM), generative-based (e.g., FlowTS, CDPM), and deep-based (e.g., PatchTST, iTransformer) methods. On average, it showed an approximate 10% reduction in MSE compared to LLM-based approaches and nearly 19% compared to generative-based approaches.
• Ablation Studies: Results confirmed that the full encoder-decoder architecture is superior to encoder-only or decoder-only variants. Furthermore, removing either the autoregressive (AR) mechanism or the flow-matching component led to significant performance degradation, validating the necessity of coupling both semantic understanding and continuous stochastic modeling.
• Efficiency: Despite using an LLM backbone, CoGenCast demonstrated lower inference latency than other LLM-based methods due to its one-step generation mechanism. It achieved performance comparable to multi-step sampling methods with a single function evaluation.
• Scalability: The model showed improved performance with extended look-back windows and maintained stability in long-term forecasting scenarios, outperforming baselines in extended horizons.

Significance and Claims
The paper claims that CoGenCast bridges the critical gap between semantic understanding and continuous stochastic modeling in time series forecasting. By linearizing generative dynamics through interval-conditioned average velocity, the framework establishes a direct and stable temporal trajectory solvable with a single function evaluation. The authors position CoGenCast not merely as an incremental improvement but as a structural shift that leverages the synergy between pre-trained LLMs and flow-matching to achieve high-precision, low-latency forecasting. The work suggests that effective forecasting benefits from simultaneously capturing structural dependencies (via LLMs) and temporal dynamics (via flow-matching), offering a valuable perspective for future research in this domain. The authors emphasize that while the method shows strong potential, it should be used with appropriate domain expert validation and uncertainty assessment in real-world deployments.