Investigating simple target-covariate relationships for Chronos-2 and TabPFN-TS

Technical Summary: Investigating Simple Target-Covariate Relationships for Chronos-2 and TabPFN-TS

Problem Statement
Time Series Foundation Models (TSFMs) have recently demonstrated state-of-the-art performance in zero-shot univariate forecasting, often surpassing supervised deep learning baselines. However, real-world applications frequently require leveraging dynamic covariates (e.g., in energy or retail domains). While recent architectures like Chronos-2 and TabPFN-TS aim to integrate covariates, existing large-scale benchmarks (e.g., fev-bench) offer limited insight into how effectively these models capture specific target–covariate dependencies. This paper addresses the gap in understanding whether the strong benchmark performance of these models translates to the accurate modeling of simple, controlled relationships between targets and covariates.

Methodology
The authors design a series of controlled experiments using minimal tasks specifically engineered to isolate target–covariate dependencies. The study compares two leading TSFMs:

1. Chronos-2: A model utilizing time and group attention layers, pretrained on synthetic multivariate data.
2. TabPFN-TS: An adaptation of the TabPFN tabular foundation model, pretrained on millions of synthetic regression tasks and adapted for time series via carefully designed features.

The experimental setup involves four distinct scenarios across real-world datasets (Electricity, Solar, Traffic) and synthetic datasets (Random Walk, KernelSynth):

• Identity: The covariate is identical to the target.
• Sum: The target is the sum of two synthetic time series ($Z = X + Y$).
• Aggregate: The target is a complex aggregate of real-world series ($Z = \sqrt{X} + Y - Z^2$).
• Quadratic: The target follows a quadratic relationship with synthetic inputs ($Z = a + bX + cY^2$).
• Auto-regressive Extension: An additional experiment introduces a temporal dependency term ($\phi Z_{t-1}$) to the quadratic task to observe the shift in model performance as time-dependence increases relative to covariate dependence.

Performance is evaluated using Normalized Mean Squared Error (NMSE) across varying look-back ($L$) and horizon ($H$) windows.

Key Results
The experiments yield several critical findings regarding the models' ability to utilize covariate information:

• Superiority of TabPFN-TS on Simple Relationships: In the Identity, Sum, and Quadratic experiments, TabPFN-TS consistently outperforms Chronos-2, particularly for short forecasting horizons. In the Identity experiment, where the target is provided as a covariate, TabPFN-TS achieves significantly lower errors than Chronos-2, even though Chronos-2 is a stronger univariate forecaster in isolation.
• Short-Horizon Advantage: TabPFN-TS demonstrates a marked advantage in short-horizon settings. The authors hypothesize this is because the input and output features in short horizons are more similarly distributed, resembling the independent and identically distributed (i.i.d.) regression problems encountered during TabPFN's pretraining.
• Robustness of Covariate Features: In the Aggregate experiment, removing time embeddings from TabPFN-TS and relying solely on covariate features (TabPFN(cov)) did not degrade performance; in fact, it became the most efficient model for short-range settings.
• Shift in Performance with Temporal Dependence: In the auto-regressive extension, as the coefficient $\phi$ (representing temporal dependence) increases, the performance gap narrows and eventually reverses. Chronos-2 begins to outperform TabPFN-TS as the task shifts from a purely nonlinear regression problem to one dominated by temporal structure. This suggests that model efficacy is contingent on the relative importance of time-to-target versus covariate-to-target relationships.

Significance and Claims
The paper claims that while Chronos-2 dominates large-scale covariate-based forecasting benchmarks, this dominance does not automatically imply optimal modeling of simple covariate–target dependencies. The results indicate that TabPFN-TS, leveraging its pretraining on expressive priors and extensive synthetic regression tasks, captures these specific relationships more reliably.

The authors conclude that there is significant potential to design future TSFMs that combine the explicit temporal dependency modeling found in Chronos-2 with the in-context regression capabilities inspired by TabPFN-TS. This hybrid approach is presented as a promising direction for developing more effective covariate-aware time series foundation models.