🤖 machine learning

A Causal DAG Prior for Synthetic Time-Series Classification Datasets

This paper introduces a causal DAG-based prior that synthesizes multivariate, multi-class time-series classification datasets with cross-modal temporal structures, demonstrating that finetuning TabPFN v2.5 on these synthetic datasets significantly improves performance on real-world benchmarks compared to existing models.

Original authors: Franco Martino O'Rourke, Ana Trisovic, Dimitris Bertsimas

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Franco Martino O'Rourke, Ana Trisovic, Dimitris Bertsimas

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to recognize different types of weather patterns. You have two kinds of data: static facts (like "it's summer" or "the soil is sandy") and moving stories (like a video of wind speed changing over time or a graph of temperature rising and falling).

Most current AI systems treat these two things separately. They might look at the static facts first, then look at the moving stories later, trying to glue the answers together at the end. This is like trying to solve a puzzle by looking at the edge pieces first, then the middle pieces, and hoping they fit.

This paper introduces a new way to teach the AI: a "Causal DAG Prior."

Here is the simple breakdown of what they did and why it matters:

1. The Problem: The "Two-Stage" Mess

Right now, AI models for time-series data (like heart rate monitors or stock prices) usually work in two steps:

Step 1: A giant brain reads millions of unlabeled videos to learn what "movement" looks like.
Step 2: A smaller, separate teacher tries to teach that brain how to guess the label (e.g., "Storm" vs. "Sunny") using a tiny amount of labeled data.

The problem is that the big brain never sees the label while it's learning the movement. It's like learning to drive a car by watching a movie, and then only being told "you crashed" after you've finished the movie. The two steps don't talk to each other.

2. The Solution: The "Synthetic Playground"

The authors wanted to build a single, smart system that learns everything at once. To do this, they needed a massive amount of practice data (labeled examples) to train on. But real-world data is scarce and messy.

So, they built a Synthetic Generator. Think of this generator as a "Video Game Engine" for data.

The Blueprint (The DAG): Instead of just making random numbers, the engine draws a map of cause-and-effect. It decides, for example: "The soil type (static) causes the humidity to rise (moving), and the humidity causes the plant to grow (moving), and the final plant size determines the harvest type (label)."
The Magic: This map connects the static facts and the moving stories naturally. It simulates how real-world variables influence each other over time.
The Output: The engine spits out thousands of complete, labeled datasets where the "story" (time series) and the "facts" (tabular data) are perfectly linked by cause and effect.

3. The Experiment: Testing the "Teacher"

To see if this new "playground" actually helps, they took a very smart, pre-existing AI model called TabPFN (which is great at solving puzzles with static facts) and gave it a special training course using their new synthetic data.

They made a few small adjustments so the model could read the "moving stories" (time series) without losing the flow of time, but they didn't change the model's brain. They just changed what it was learning from.

4. The Results: The "Cross-Modal" Advantage

They tested this new setup on 75 real-world datasets (like heartbeats, gestures, and power consumption).

The Control Group: They tested the original AI model (which had never seen their synthetic data).
The "Bare Bones" Group: They tested the model trained only on synthetic data that looked like static facts (ignoring the time-series part).
The Winner: The model trained on their full causal generator (where static facts and moving stories were linked by cause-and-effect) significantly outperformed everyone else.

The Key Takeaway:
The paper proves that the AI didn't just get smarter because it saw more data. It got smarter because it learned how static facts and moving stories influence each other.

When the model saw a dataset where "soil type" caused "humidity changes," it learned to look for that specific pattern. When they removed the time-series part of the training (the "Bare Bones" group), the model got no better than the original. This proves that the connection between the two types of data is the secret sauce.

Summary Analogy

Imagine you are learning to bake a cake.

Old Way: You watch a video of someone mixing ingredients (Step 1), then someone hands you a list of ingredients and asks, "Is this a cake?" (Step 2). You have to guess how the mixing relates to the list.
New Way (This Paper): You are given a magical cookbook that generates thousands of practice cakes. For every cake, the book explains: "Because we used flour (static fact), the dough rose (moving story), which means it is a sponge cake (label)."
Result: By practicing with this magical cookbook, you learn the relationship between the ingredients and the rising dough. When you face a real cake later, you don't just guess; you understand the cause-and-effect, and you get the answer right much more often.

The paper concludes that by teaching AI models to understand these cross-modal cause-and-effect relationships using synthetic data, we can build much better classifiers for time-series problems.

Technical Summary: A Causal DAG Prior for Synthetic Time-Series Classification Datasets

Problem Statement

Current foundation-model pipelines for Time-Series Classification (TSC) typically follow a two-stage paradigm: a large transformer is pretrained on unlabelled time-series corpora to act as an encoder, followed by a dataset-specific classifier (e.g., linear head, k-NN, or TabPFN) fitted on frozen embeddings. This approach suffers from a disconnect where the encoder cannot utilize training labels or other samples as context during representation extraction, and the two stages optimize different objectives.

While in-context learning (ICL) offers a single-stage alternative for tabular data (exemplified by TabPFN), applying this to TSC requires a synthetic generator capable of producing complete, labelled datasets with inherent temporal structure. Existing synthetic priors for time series, such as KernelSynth and CauKer, generate time series but fail to produce class labels, resulting in incomplete datasets. Furthermore, concurrent work like TiCT utilizes synthetic priors that are not natively multivariate, multi-class, or cross-modal. There is a critical absence of a prior that samples entire labelled datasets $(X, y)$ by explicitly modelling causal relationships between two modalities: tabular attributes and time series.

Methodology

The Causal DAG Prior

The authors introduce a synthetic causal prior that generates complete, labelled multivariate TSC datasets using a single Directed Acyclic Graph (DAG) based on Structural Causal Models (SCMs). Unlike approaches that unroll dynamic SCMs recurrently (which can suffer from instability), this design injects temporal structure non-recurrently within an acyclic graph.

Graph Structure and Node Types:

Typed Nodes: The DAG contains nodes of two types: tabular (scalars, $\in \mathbb{R}$ ) and series-valued (trajectories, $\in \mathbb{R}^T$ ).
Root Initialization:
- Tabular roots: Sampled from random normal or uniform distributions.
- Series roots: A random kernel (a composition of Linear, RBF, and Periodic base kernels) is sampled once per dataset. Per-sample trajectories are then drawn from the zero-mean Gaussian Process defined by this kernel.
Causal Edges and Mechanisms: Edges represent computational mappings chosen based on the child node type:
- Tabular children: Mapped via small neural networks with pointwise nonlinearities or discretisation functions (converting numerical outputs to categorical).
- Series children: Mapped via single random 1-D convolutions (Conv1D) with causal padding, random dilation, and nonlinear activation. This operation handles cross-channel mixing (concatenating parents) and temporal dependence.
- Cross-modal interactions: The graph naturally supports tabular $\to$ series, series $\to$ tabular, and series $\to$ series causal effects.
Dataset Generation: Fresh initialization data is propagated through the DAG. Selected series-valued nodes form the input features $X \in \mathbb{R}^{n \times m \times T}$ , while a discretised tabular node defines the class label $y \in \{1, \dots, K\}^n$ .
Augmentation: To simulate real-world challenges, the generator applies predictive truncation (variable-length observations), value-level missingness, random temporal granularity (strided pooling/step-repeat), and per-channel value warps.

Probe: Adapting TabPFN v2.5

To validate the prior, the authors finetune TabPFN v2.5, a transformer-based model pretrained on tabular causal priors. The goal is not to redesign the architecture but to isolate the contribution of the synthetic prior. Minimal adaptations are made to preserve temporal structure:

Tokenisation: Input data is flattened to $n \times (m \cdot T)$ . To respect channel boundaries, the time axis of each channel is padded to a multiple of the token group size $G$ . Tokens are formed by grouping $G$ consecutive timesteps of the same channel, preventing the mixing of channels within a single token.
Normalization: The default per-column z-scoring (which would destroy temporal patterns within a patch) is disabled. Instead, z-scoring is applied per channel before tokenisation.
Inference: The model performs in-context learning by conditioning on the entire labelled training set in a single forward pass.

Key Contributions

A Cross-Modal Causal Prior: The introduction of a synthetic prior that samples entire labelled datasets from a DAG containing both tabular and series-valued nodes. This allows for the modelling of complex causal regimes (e.g., soil type $\to$ humidity trajectory $\to$ growth trajectory $\to$ yield) that existing priors cannot generate.
Isolation of Temporal Structure: By finetuning a tabular foundation model (TabPFN) with minimal architectural changes, the authors demonstrate that performance gains are driven specifically by the cross-modal temporal structure of the prior, rather than architectural improvements or generic finetuning recipes.
Scalable Non-Recurrent Design: The use of an acyclic graph with non-recurrent temporal injection avoids the training instability and scaling difficulties associated with recurrent dynamics in untrained priors.

Experimental Results

The authors evaluated the approach on 75 UCR/UEA datasets that fit within TabPFN's operating regime ( $K \le 10$ classes, $m \cdot T \le 500$ ).

Comparative Configurations:

(A) Standard TabPFN: Pretrained weights, default tabular preprocessing.
(B) Pretrained + Custom Inference: Same weights as (A) but with the temporal-preserving preprocessing (patch-aligned tokenisation, per-channel z-scoring).
(C) Finetuned (Ours): Configuration (B) finetuned on the full causal DAG prior.
(D) Finetuned, Tabular-Only Ablation: Configuration (B) finetuned on a prior where all series-specific components (GP roots, Conv1D, time-series augmentations) are disabled ( $T=1$ ).

Findings:

Performance: Configuration (C) achieved the best mean accuracy and ROC–AUC across both ensemble sizes ( $e=1$ $e = 1$ and $e=8$ $e = 8$ ).
- At $e=8$ , (C) achieved 0.8575 accuracy and 0.9444 ROC–AUC, outperforming the standard upstream model (A) (0.8525 / 0.9405) and the tabular-only ablation (D) (0.8493 / 0.9393).
Statistical Significance:
- Finetuning on the full generator (C) significantly outperformed the tabular-only ablation (D) with a Wilcoxon signed-rank p-value of $3.0 \times 10^{-8}$ on ROC–AUC.
- The tabular-only ablation (D) showed no statistically significant improvement over the unmodified upstream model (A) ( $p=0.10$ for AUC), and was even outperformed by the custom inference baseline (B) ( $p=1.7 \times 10^{-4}$ ).
Conclusion on Mechanism: The results confirm that the series-specific mechanisms (GP roots and Conv1D) are necessary for the performance gain; simply finetuning on a DAG prior with the correct input shape but no temporal structure is insufficient and can be detrimental.

Significance and Claims

The paper claims to address a gap in the literature regarding synthetic priors for TSC. By introducing a prior that natively models causal relationships between tabular attributes and time series, the authors enable the training of in-context classifiers that can leverage the full space of cross-modal structures.

The primary significance lies in the isolation of the learning signal: the authors demonstrate that the improvement in TSC performance is directly attributable to the cross-modal temporal structure encoded in the synthetic prior, rather than the architecture itself. This validates the hypothesis that "prior-fitted networks" can learn useful inductive biases for time series if the prior correctly captures the causal mechanisms of the domain.

The authors remain modest regarding the scope, noting that:

The resulting model is a "probe" and not a state-of-the-art TSC foundation model (as the backbone was not redesigned).
Evaluation is restricted to the specific operating regime of TabPFN ( $m \cdot T \le 500$ ), excluding long or high-dimensional series.
The work proves the incremental value of series mechanisms over a tabular prior but does not claim absolute superiority over all existing TSC methods.
The framework is extensible to other structured data (e.g., images, video) by changing node types and mechanisms, suggesting a broader potential for multimodal causal priors.