Interventional Time Series Priors for Causal Foundation Models

This paper introduces CausalTimePrior, a novel framework for generating synthetic temporal structural causal models with paired observational and interventional data to enable the training of Prior-data fitted networks (PFNs) for in-context causal effect estimation in time series.

The Gist

Imagine you are trying to teach a super-smart robot how to understand cause and effect in the real world.

Right now, we have robots that are great at looking at a spreadsheet of numbers and guessing what will happen next (like predicting tomorrow's stock price based on yesterday's). But these robots are terrible at answering the question: "What would happen if I changed something?"

For example, if you ask a normal robot, "What happens to the ice cream sales if I turn off the air conditioning?" it might just say, "Well, usually when it's hot, people buy more ice cream." It doesn't understand that you turning off the AC is a new action that breaks the normal pattern. It hasn't learned the difference between watching the world and changing the world.

This paper introduces a new tool called CausalTimePrior to fix this problem, specifically for things that change over time (like weather, stock markets, or heart rates).

Here is the breakdown in simple terms:

1. The Problem: The Robot Has Never Seen a "What-If" Scenario

To teach a robot to understand cause and effect, you need to show it examples of interventions.

• Observation: Watching a ball roll down a hill.
• Intervention: Kicking the ball while it's rolling.

Most existing time-series datasets are like a security camera feed. They show you what happened, but they never show you what would have happened if you had kicked the ball. Without these "kicking" examples, the robot can't learn to predict the future after a change.

2. The Solution: A "Simulation Factory"

The authors built a CausalTimePrior, which is essentially a factory that manufactures fake time-travel scenarios.

Instead of waiting for real-world experiments (which are expensive and dangerous), this factory generates millions of synthetic time-series datasets. It creates two versions of every story:

1. The "Natural" Version: How the world behaves on its own.
2. The "Intervention" Version: What happens when we force a specific change (like "What if the temperature suddenly dropped?").

The factory is special because it can create very complex stories:

• Non-linear: Things don't just go up and down in straight lines; they can curve, spike, or behave wildly.
• Regime-Switching: Imagine a car driving on a road that suddenly turns into a swamp. The rules of driving change. This factory can simulate those sudden changes in rules (regime switches).
• Different Types of "Kicks": You can tell the factory to "hard stop" a variable (force it to zero), "softly nudge" it, or change it gradually over time.

3. The "Foundation Model" (The Student)

Once the factory has generated these millions of "Natural vs. Intervention" stories, they train a Foundation Model (a type of AI, specifically a Prior-Data Fitted Network or PFN) on them.

Think of this AI as a medical student who has read millions of medical textbooks (the synthetic data) but has never seen a real patient.

• The Test: You give the AI a new real-world dataset it has never seen before.
• The Question: "Here is a patient's heart rate history. Now, imagine we give them a specific drug at 2:00 PM. What will their heart rate be at 2:30 PM?"
• The Result: Because the AI studied the "What-If" scenarios in the factory, it can answer this question without needing to retrain on the specific patient. It uses "in-context learning," meaning it figures out the rules on the fly, just like a human expert.

4. Why This Matters (The Analogy)

Imagine you are learning to drive.

• Old Way: You sit in a car and watch a video of someone else driving for 10 years. You know how the car usually behaves. But if you get in the car and someone yells "Turn left!" (an intervention), you might panic because you've never practiced that specific scenario.
• New Way (CausalTimePrior): You spend 10 years in a driving simulator that randomly throws obstacles at you, changes the road conditions, and forces you to make sudden turns.
• The Outcome: When you finally get into a real car, you don't need to practice on that specific car. You already know how to handle the "What-If" situations because your simulator training covered every possible twist and turn.

Summary of the Paper's Achievements

1. First of its Kind: It's the first tool that generates time-series data with both "watching" and "changing" scenarios, including complex rule-changes (regime switching).
2. Proven to Work: They trained a simple AI on this data. When tested on new, unseen data, the AI could successfully predict the outcome of interventions, distinguishing between things that are truly connected (causal) and things that just happen to move together by coincidence (correlation).
3. The Future: This paves the way for "Foundation Models for Causality." In the future, we might have one giant AI that understands cause and effect for any time-based system (finance, climate, biology) without needing to be retrained for every single new problem.

In a nutshell: The authors built a time-travel simulator that teaches AI the difference between watching the world and changing it, so the AI can make better predictions about the future when we intervene.

Technical Summary

Here is a detailed technical summary of the paper "Interventional Time Series Priors for Causal Foundation Models" presented at ICLR 2026.

1. Problem Statement

The paper addresses a critical bottleneck in applying Prior-Data Fitted Networks (PFNs)—a class of foundation models capable of in-context learning—to time series causal inference.

• The Gap: While PFNs have shown success in tabular causal inference (e.g., Do-PFN, CausalFM) by training on synthetic Structural Causal Models (SCMs) with interventional data, this approach has not been extended to time series.
• The Obstacle: Existing time series causal benchmarks (e.g., CausalTime, TimeGraph, CauseMe) provide observational data with ground-truth causal graphs but lack interventional data. Without paired observational and interventional datasets, models cannot learn to predict counterfactual outcomes (i.e., "what would happen if we intervened?"), which is the core task of causal inference.
• Limitations of Existing Generators: Current generators with interventional support (e.g., CAnDOIT, TECDI, CaTSG) are limited by static interventions, linear mechanisms, or reliance on training separate generative models (like diffusion models) rather than analytical sampling. None support regime-switching dynamics (changing causal structures over time) combined with diverse interventional targets.

2. Methodology: CausalTimePrior

The authors propose CausalTimePrior, a principled framework for sampling Temporal Structural Causal Models (TSCMs) to generate paired observational and interventional time series suitable for training foundation models.

A. Temporal SCM Definition

The framework models discrete-time acyclic SCMs defined by:

• Graph Structure ($G$): A time-lagged Directed Acyclic Graph (DAG) comprising instantaneous edges ($G_0$) and lagged edges ($G_k$) up to lag $K$.
• Structural Equations ($F$): Nonlinear autoregressive mechanisms where $X_t^{(i)} = f_i(Pa(X_t^{(i)})) + \epsilon_t^{(i)}$.
• Noise ($\epsilon$): Sampled from diverse distributions (Gaussian, Uniform, Laplace).

B. The Prior Distribution ($\Pi$)

The framework samples TSCMs from a prior distribution covering:

1. Graph Prior: Samples variable count ($N$), max lag ($K$), and edge probabilities. Instantaneous edges are sampled via Erdős-Rényi models with topological ordering to ensure acyclicity; lagged edges decay in probability over time.
2. Mechanism Prior: Supports diverse nonlinear functions (identity, sin, cos, tanh, absolute value, square, exponential) with random weights and biases.
3. Regime-Switching Extension: A key innovation where the causal structure ($G$) and mechanisms ($F$) switch between $R$ regimes based on a Markov chain ($d_t$). This allows the model to learn causal inference under structural breaks.

C. Intervention Types

The framework generates data for three distinct intervention types, modifying the structural equations accordingly:

1. Hard Interventions ($do$-operator): Sets $X_t^{(i)} = c$, severing incoming edges.
2. Soft Interventions: Perturbs the mechanism by adding a shift $\delta$ to the structural equation.
3. Time-Varying Interventions: Sets $X_t^{(i)} = c(t)$, where $c(t)$ follows a dynamic profile (step, ramp, sinusoidal).

D. Data Generation Pipeline

For each training instance:

1. Sample a TSCM ($S$) from the prior.
2. Sample intervention specifications (targets, times, type, values).
3. Generate an observational series ($X_{1:T}^{obs}$) via forward simulation.
4. Generate an interventional series ($X_{1:T}^{int}$) by simulating the system under the $do$-operator.
5. Form a training tuple: $(X_{1:T}^{obs}, \text{intervention spec}, Y_{\tau}^{int})$.

3. Key Contributions

1. CausalTimePrior Framework: The first synthetic data generator for time series that produces paired observational and interventional data with regime-switching dynamics.
2. Comprehensive Intervention Support: Supports hard, soft, and time-varying interventions within nonlinear, lagged causal structures, overcoming the limitations of previous generators.
3. Foundation Model Pathway: Demonstrates that PFNs trained on this prior can perform in-context causal effect estimation on held-out TSCMs without task-specific fine-tuning.
4. Open Source: The framework is released as an open-source library to facilitate future research in time series causal foundation models.

4. Experimental Results

The authors trained a proof-of-concept 2-layer GRU-based PFN on 100,000 TSCMs generated by CausalTimePrior and evaluated it on 1,000 held-out TSCMs.

• Causal Understanding: The model successfully distinguished between causal and non-causal queries.
  • Intervened Queries: Pred/GT ratio of 0.95 (high accuracy).
  • Downstream Queries: Pred/GT ratio of 0.85.
  • Non-Causal Queries: Pred/GT ratio of 0.46, indicating the model correctly predicted near-zero effects despite spurious correlations.
• Comparison to Baselines:
  • The PFN achieved comparable RMSE to a Vector Autoregression (VAR-OLS) baseline (176.4 vs. 176.5) but required no per-dataset fitting, whereas VAR requires training on every new dataset.
  • While PCMCI+ (which discovers graphs per sample) had lower RMSE, it is computationally expensive and not an in-context learner.
• Ablation Studies:
  • Intervention Diversity: Training on mixed intervention types (hard, soft, time-varying) improved effect direction accuracy (70.4%) and size correlation compared to training only on hard interventions.
  • Out-of-Distribution (OOD): The model showed degradation on OOD data (larger/denser graphs, complex mechanisms) but retained the ability to distinguish causal from non-causal paths.
• Spurious Correlation Handling: In specific test cases with high negative correlation but no causal path, the PFN correctly predicted near-zero causal effects, whereas VAR-OLS failed significantly (177x larger error), proving the model learned causal structure rather than mere correlation.

5. Significance and Future Work

• Significance: This work bridges the gap between causal foundation models and time series analysis. By providing a scalable, principled way to generate interventional time series data, it enables the training of models that can reason about "what-if" scenarios in dynamic environments without retraining.
• Future Directions:
  • Extending to continuous-time dynamics (e.g., via Neural SDEs) to handle irregularly sampled data.
  • Incorporating non-Markovian confounding.
  • Explicitly stratifying over canonical causal motifs (confounders, mediators, colliders) to improve coverage.
  • Validating against real-world semi-synthetic datasets derived from observational data.

In conclusion, CausalTimePrior establishes a foundational step toward Time Series Causal Foundation Models, enabling in-context learning for causal inference in complex, dynamic, and regime-switching environments.