JointFM-0.1: A Foundation Model for Multi-Target Joint Distributional Prediction

JointFM-0.1 is a novel foundation model that bypasses traditional Stochastic Differential Equation (SDE) fitting by training on an infinite stream of synthetic SDEs to achieve zero-shot, calibration-free prediction of future joint probability distributions for coupled time series, outperforming baselines by reducing energy loss by 14.2%.

The Gist

Imagine you are trying to predict the weather for a whole city next week.

The Old Way (Traditional Modeling):
Right now, if a meteorologist wants to do this, they have to go through a tedious, three-step process every single time:

1. Pick a Theory: "Is it a hurricane? A heatwave? A cold front?" They have to guess the right mathematical formula.
2. Tune the Dials: They look at the last few days of data and manually adjust the numbers in that formula to make it fit. This is slow and fragile; if one new data point is weird, the whole formula breaks, and they have to start over.
3. Run the Simulation: They run a massive computer simulation thousands of times to see what might happen. This takes a lot of computing power and time.

If you need an answer right now (like "Should I cancel the outdoor concert in 10 minutes?"), this old way is too slow and too brittle.

The New Way (JointFM):
The paper introduces JointFM, which is like a "Super-Weather Oracle" that skips all those steps.

The Core Idea: Learning from a "Universe of Possibilities"

Instead of trying to fit a formula to your specific data, the researchers built a model that was trained on a massive, infinite library of fake universes.

Think of it like this:

• Imagine a video game developer who creates millions of different physics engines. Some have gravity like Earth, some have gravity like the Moon, some have wind that blows sideways, and some have rain that falls upward.
• They let an AI play in all these different worlds, over and over again, learning the patterns of how things move, crash, and interact.
• JointFM is that AI. It has "seen" every possible way a system of moving parts (like stock prices, energy grids, or supply chains) could behave because it was trained on synthetic data generated by random math equations.

How It Works in Real Life

Now, when you ask JointFM a question about a real-world problem (e.g., "What will happen to these 10 stocks tomorrow?"), it doesn't need to:

• Pick a formula.
• Calibrate dials.
• Run slow simulations.

It just looks at your current data (the "context") and instantly says, "I've seen this pattern before in my training library. Here is the full picture of what could happen next, including how all the stocks will move together."

Why "Joint" Matters

Most AI models predict one thing at a time. They might tell you Stock A will go up, and Stock B will go down. But they don't tell you if they will move together.

In the real world, things are connected. If the price of oil spikes, it might hurt airline stocks and help energy stocks simultaneously.

• Old AI: "Here is a prediction for Oil. Here is a prediction for Airlines." (It misses the connection).
• JointFM: "Here is a prediction of the entire ecosystem. It knows that if Oil jumps, Airlines will likely crash, and it shows you the probability of that exact scenario happening."

The "Magic" Results

The researchers tested this on a computer with made-up data (synthetic SDEs) that the AI had never seen before.

• Speed: It generated 10,000 possible future scenarios in about 10 milliseconds (faster than a human can blink).
• Accuracy: It was 14% better at predicting the future than the best traditional methods, even though it didn't know the "rules" of the game beforehand. It just knew the "feel" of the game.

The Analogy: The "Digital Quant"

Think of a traditional quantitative analyst (a "Quant") as a master chef who has to:

1. Buy fresh ingredients (data).
2. Taste-test and adjust the spices (calibration).
3. Cook the meal for hours (simulation).

JointFM is like a molecular food printer that has tasted every dish in every restaurant in the world. You hand it a few ingredients, and it instantly prints a perfect, complex meal that accounts for how every flavor interacts with every other flavor. It doesn't need to cook; it just knows.

Why This Changes Everything

This technology moves us from "slow, manual calculation" to "instant, intuitive prediction."

• Finance: Instantly assess the risk of a whole portfolio without waiting for a human to run numbers.
• Energy: Balance the power grid in real-time as wind and solar output fluctuate.
• Supply Chain: Predict how a delay in one port will ripple through the entire global shipping network.

In short, JointFM is the first AI that doesn't just guess a single number; it understands the entire story of how a complex system might evolve, instantly and without needing to be taught the rules of the specific game it's playing.

Technical Summary

1. Problem Statement

Traditional quantitative modeling for systems under uncertainty relies on a fragmented, three-stage pipeline: Select-Calibrate-Simulate.

1. Select: Choose a specific stochastic process (e.g., Geometric Brownian Motion, GARCH).
2. Calibrate: Fit parameters to historical data.
3. Simulate: Generate future paths via Monte Carlo simulation.

Limitations of the Traditional Approach:

• Brittleness & Overfitting: Simple models miss complex dynamics (volatility, jumps, regime shifts), while complex models are hard to calibrate and prone to overfitting. A single new data point can invalidate the entire model, requiring re-calibration.
• Computational Cost: High-fidelity simulations are expensive, making real-time risk assessment infeasible.
• Lack of Joint Modeling: Existing Time-Series Foundation Models (TSFMs) like Chronos or TimesFM focus on univariate forecasting. They predict individual series independently, failing to capture the joint probability distribution and cross-variable dependencies (correlations, tail dependencies) crucial for portfolio optimization, supply chain hedging, and grid balancing.
• SDE Recovery Issues: Neural approaches attempting to recover Stochastic Differential Equations (SDEs) from data often fail when the true dynamics are unknown, non-stationary, or too complex for a tractable SDE family.

2. Methodology: The JointFM Approach

JointFM inverts the traditional paradigm. Instead of fitting an SDE to data, it trains a foundation model on an infinite stream of synthetic SDEs to learn the "laws of physics" governing stochastic processes.

A. Synthetic Physics Pretraining (The Curriculum)

The model is trained on a procedurally generated, curriculum-controlled universe of multivariate SDE systems.

• Data Generation: For every training batch, a random SDE system is sampled from a hierarchy of complexity (Levels 0–7 in the study).
• SDE Components: The curriculum includes:
  • Drift: Deterministic trends, mean reversion.
  • Diffusion: Correlated Brownian motion, state-dependent volatility.
  • Jumps: Compound Poisson processes.
  • Regime Switching: Telegraph, logistic, and hazard-based switching.
  • (Note: Memory/Volterra terms are defined but not yet fully utilized in the current experiments.)
• Training Objective: Given a history window ($X_{1:T}$) and a sampled system specification ($\phi$), the model predicts the conditional density of future paths ($p(X_{T+1:T+H} | X_{1:T}, \phi)$).
• Zero-Shot Nature: The model never sees the same sample twice. It learns to generalize across a vast space of dynamics without task-specific finetuning.

B. Architecture & Output Heads

• Backbone: A Transformer-based architecture with Variable-Factored Attention to handle dynamic numbers of assets.
• Distributional Heads: The model outputs parameters for flexible joint distributions, comparing two specific heads:
  1. Multivariate Gaussian Mixture (MV-GMM): Predicts weights, means, and covariance matrices.
  2. Skewed Student-t Mixture: Predicts weights, degrees of freedom, location, scale, and skewness. This is critical for capturing heavy tails and asymmetry in financial risk.

C. Inference ("The Simulation Shortcut")

JointFM bypasses the select-calibrate-simulate loop. By implicitly performing model selection and calibration within its internal activations, it generates high-quality distributional predictions in a single forward pass.

• Speed: Generates 10,000 samples for 10 targets across 63 horizons in ~10ms on an NVIDIA H100 GPU.
• Cost Independence: Inference cost is independent of the complexity of the underlying dynamics.

3. Key Contributions

1. First Foundation Model for Joint Distributions: Unlike prior work that predicts univariate paths or independent marginals, JointFM is the first to train on and predict the full joint probability distribution of coupled time series, explicitly capturing dependency structures.
2. Synthetic Physics Pretraining: Scales the concept of synthetic pretraining (previously limited to univariate regimes) to a diverse, infinite stream of multivariate SDEs, enabling zero-shot generalization to unseen dynamics.
3. Unified Inference: Eliminates the speed-quality tradeoff. It provides instant, zero-shot distributional forecasts without requiring historical data fitting or retraining for new tasks.
4. Industry-Agnostic Generalization: The training curriculum maps to dynamics found in Finance (regime-switching jump-diffusion), Energy (mean-reverting demand), Commodities (Schwartz-Smith), and Logistics, proving the model's universality.

4. Experimental Results

Setup:

• Test Data: 200 unseen SDE systems (Level 7 complexity: jumps + logistic regime switching) with 10 coupled target series.
• Baselines: DCC-GARCH (parametric fit) and Historical Simulation (non-parametric bootstrap).
• Mode: Pure zero-shot (JointFM is not finetuned on test data).

Metrics:

• Energy Loss: Primary metric measuring statistical distance between generated and ground-truth joint samples (captures all dependencies).
• Marginal Energy: Measures calibration of individual variables (blind to correlation).
• CRPS-sum: Assesses the distribution of the aggregated sum of targets.

Findings:

• Superior Performance: JointFM (specifically the Skewed Student-t head) consistently outperformed all baselines across all metrics and forecast horizons.
• Quantitative Gains:
  • 14.2% reduction in Energy Loss relative to the strongest baseline (averaged across horizons).
  • 12.4% reduction in Marginal Energy Loss.
  • 7.2% reduction in CRPS-sum.
• Horizon Sensitivity: The performance gap widened as the forecast horizon increased. While Historical Simulation worked for short horizons, it degraded significantly for long horizons. DCC-GARCH struggled due to the limited context window ($T=504$) required for reliable estimation.

5. Significance and Future Outlook

Significance:

• Paradigm Shift: JointFM demonstrates that a pretrained neural model can generalize across the space of multivariate stochastic processes without access to the underlying equations or retraining.
• Real-Time Risk Management: It enables "agentic AI" workflows where autonomous systems can make risk-aware decisions instantly (e.g., portfolio rebalancing, grid dispatch) without waiting for human calibration cycles.
• Universal Estimator: It serves as a universal estimator for joint distributions, applicable to finance, energy, logistics, and retail.

Limitations & Future Work:

• Dynamics Coverage: Current experiments cover Markovian dynamics (Levels 0–7). Future work will include non-Markovian memory (Volterra), rough volatility (Heston, SABR), and more complex regime structures.
• Scale: The current proof-of-concept uses 10 targets and 504 steps. Scaling to higher dimensions and longer contexts is planned but faces hardware bottlenecks.
• Observed Dynamics Fallacy: If a specific risk (e.g., a jump) hasn't occurred in the context window, the model may miss it. Future iterations plan to incorporate "hybrid inference" using categorical tokens (e.g., "Jump-Diffusion") to guide priors with domain knowledge.
• Data Types: Currently limited to continuous float time series; future versions aim to handle categorical and mixed-type variables.

In conclusion, JointFM represents a foundational step toward real-time, universal quantitative modeling, replacing brittle, manual calibration pipelines with a robust, zero-shot foundation model capable of predicting complex joint distributions instantly.