💻 computer science

Orca: Neural Operators for Causal Reasoning in Continuous Time

The paper introduces Orca, a framework leveraging neural operators to extend structural causal models to continuous-time systems with feedback loops and irregular observations, enabling the learning of causal mechanisms as mappings between function spaces for robust counterfactual reasoning.

Original authors: Gerrit Großmann, David A. Selby, Sebastian J. Vollmer

Published 2026-07-31

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Gerrit Großmann, David A. Selby, Sebastian J. Vollmer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand a story that is constantly being written, not in chapters, but in a flowing river of time. In the world of science, we often try to figure out cause and effect: "If I eat more vegetables, will I live longer?" or "If I exercise, will my heart get stronger?" For decades, scientists have used a tool called a "Structural Causal Model" to answer these questions. Think of this tool like a snapshot camera. It takes a picture of a person at one specific moment, measures their exercise and their health, and draws a line between them. But real life isn't a snapshot; it's a movie. People don't just exercise once; they build up fitness over years. Health isn't a single number; it's a journey that changes every day. Furthermore, real life has loops: being sick might make you exercise less, which makes you sicker, creating a cycle. The old "snapshot" tools struggle with these moving, looping stories because they are designed for static pictures, not flowing movies.

This is where a new idea called "Neural Operators" comes in. If a normal computer program is like a calculator that adds two numbers, a Neural Operator is like a machine that learns to translate entire movies into other movies. It doesn't just look at one frame; it understands the whole flow of time, even if the frames are recorded at weird, irregular intervals. Scientists care about this because so many of the things we worry about—our health, the climate, the economy—are these very same flowing, looping systems. If we want to know what would happen if we changed a policy or a treatment, we need a way to reason about these continuous, messy stories, not just frozen moments.

Enter Orca, a new framework proposed by researchers at the German Research Center for Artificial Intelligence. Orca is like a super-smart director for these causal movies. Instead of treating variables like "exercise" or "health" as single numbers, Orca treats them as entire functions of time—continuous lines that stretch from the past into the future. The paper suggests that by using Neural Operators, we can teach a computer to learn the "rules of the game" (the mechanisms) that drive these systems, even when the data is messy and the systems have feedback loops.

The researchers tested Orca on synthetic examples, which are like highly detailed video game simulations where they know the exact rules of the world. In one example, they looked at how age, exercise, and health interact over 20 years. They found that older methods, which tried to freeze time or ignore the loops, got the answers wrong. They overestimated how much exercise helped because they didn't account for the fact that older people tend to exercise less. Orca, however, successfully learned the underlying "kernel"—a fancy word for the specific recipe that says how past exercise turns into future health. It figured out that the effect of exercise isn't instant; it's like a ripple in a pond that fades over time.

Crucially, Orca doesn't just predict the future; it can imagine "what if" scenarios, known as counterfactuals. It can ask, "If this specific person had exercised twice as much, what would their health trajectory look like?" To do this, the model has to figure out the hidden "noise" or unique luck of that person. In some tests, the researchers assumed this noise was simple (like a steady background hum), and Orca nailed the counterfactuals. In more complex tests, where the noise was tricky and changed over time (like a variable drug sensitivity in a tumor growth simulation), they used a clever matching technique to pair real people with simulated "luck" profiles. The results showed that Orca could reproduce the population's behavior and generate plausible individual counterfactuals.

The paper is careful to note that these are simulations, not real-world medical advice yet. The "proof" is in the math and the synthetic data, where the ground truth is known. The authors suggest that Orca is a promising foundation because it respects the "arrow of time" (causes must happen before effects) and can handle data collected at irregular times, unlike older models that get confused by uneven schedules. While the current version assumes the map of causes (the graph) is already known, the authors see this as a major step toward understanding the continuous, looping, and messy causal stories that define our real world. They conclude that by treating time as a continuous flow rather than a series of steps, Orca offers a more natural and accurate way to reason about interventions and counterfactuals in complex systems.

Technical Summary: Orca: Neural Operators for Causal Reasoning in Continuous Time

Problem Statement

Structural Causal Models (SCMs) are the standard framework for reasoning about interventions and counterfactuals. However, classical SCMs are designed for static variables measured at single points in time and typically forbid cyclic dependencies. Many real-world systems of interest—such as patient health trajectories, climate dynamics, and economic cycles—evolve continuously, are observed at irregular time points, and contain feedback loops. Existing approaches to extend causality to time-series data often rely on discrete time unfolding, which ties conclusions to a specific sampling grid, or combine causality with dynamical systems in ways that lack a unified learning recipe for irregular, continuous data. There is a need for a framework that supports causal reasoning on continuous-time data with cyclic dependencies while remaining invariant to the resolution of observation.

Methodology: The Orca Framework

The paper proposes Orca (Operator-based Reasoning for Causal Analysis), a framework that treats each node in a causal graph as a continuous function of time and each causal mechanism as a learned map between function spaces.

1. Generative Model and Semantics

Orca defines a cyclic continuous-time causal model over a time interval $[0, T]$ .

Nodes and Graph: The model uses a directed graph $G$ with nodes $V$ and edges that may include cycles. Each node $v$ represents a real-valued function $x_v: [0, T] \to \mathbb{R}$ .
Temporal Priority: To handle cycles without instantaneous circularity, the model enforces the "arrow of time." The value of a node $x_v(t)$ at time $t$ is determined by a forward operator $F_v$ acting on the history of its parents $H_v(t)$ (values strictly before $t$ ) and the current exogenous noise $N_v(t)$ .
$x_v(t) = F_v(H_v(t), N_v(t), t)$
where $H_v(t) = \{(u, t', x_u(t')) \mid u \in \text{pa}(v), t' < t\}$ .
Noise: Exogenous noise is modeled as a stochastic process (e.g., Brownian motion). In the simplest case, the model assumes additive noise, where the mechanism splits into a deterministic function of the past plus the current noise: $x_v(t) = F_v(H_v(t), t) + N_v(t)$ .

2. Neural Operator Architecture

Orca learns the mechanisms $F_v$ using neural operators, which are designed to learn discretization-invariant mappings between function spaces.

Deep-Set Representation: To handle irregular, sparse, and asynchronous measurements, the architecture uses an encode-sum-decode structure (inspired by DeepSets and MIONet).
- Embedding: A shared network $\phi_{\theta_1}$ embeds every observed measurement of the parent nodes.
- Integration: The embeddings are summed with time-based weights $w_{t'}$ to approximate an integral over the parent trajectories. This ensures the output is independent of the sampling density.
- Decoding: A decoding network $\rho_{\theta_2}$ maps the aggregated embedding and the query time $t$ to the predicted value $\hat{x}_v(t)$ .
Training: The model is trained by minimizing a subject-normalized mean squared error over irregular observations. The weighting scheme ensures that densely sampled subjects do not dominate the loss, allowing the model to converge to the underlying continuous operator as the sampling grid is refined.

3. Causal Inference and Counterfactuals

Once the mechanisms are learned, Orca supports standard causal queries:

Interventional Queries: To estimate dose-response curves or average treatment effects, the model simulates the system under a modified mechanism (e.g., $do(E=e)$) while keeping other mechanisms and the noise distribution unchanged.
Counterfactuals: To compute individual counterfactuals (e.g., "What would this specific patient's trajectory have been?"), the framework must infer the subject's latent noise.
- Additive Noise: The noise is recovered directly as the residual between the observed trajectory and the model prediction.
- Non-Additive Noise: For cases where noise is not additive (e.g., multiplicative effects), the paper proposes a coupling procedure. It uses an optimal transport objective to match observed residual trajectories to sampled noise trajectories from a known noise model, based on a monotonicity assumption. This allows the inference of subject-specific latent noise for counterfactual simulation.

Key Contributions

Framework Definition: Introduction of Orca, a framework that unifies neural operator learning with causal semantics for continuous-time, cyclic systems.
Resolution Invariance: Demonstration that neural operators can learn causal mechanisms from irregular, sparse data such that predictions converge to the true continuous-time operator as sampling density increases, avoiding the grid-dependency of discrete-time methods.
Causal Semantics for Operators: Extension of neural operator architectures to respect causal graph structure and the temporal priority principle, enabling the learning of mechanisms that compute node values from parent functions.
Counterfactual Reasoning: A method for inferring latent noise variables (both additive and non-additive) to enable individual counterfactual reasoning in continuous time.

Experimental Results

The paper validates Orca on synthetic continuous-time examples:

Glucose-Insulin Model (Resolution Convergence): In a model with confounding, Orca learned the dose-response curve. As the sampling density increased, the estimated curves converged to the ground truth. Crucially, a model trained on sparse data could accurately predict outcomes on a finer grid than the training data, whereas a pooled baseline remained biased.
Tumor Growth (Non-Additive Noise): In a model where drug sensitivity varied non-additively with tumor size, Orca successfully coupled observed subjects to latent noise trajectories using optimal transport. The re-simulated population matched the observed population's statistics, and individual counterfactuals (e.g., halving a dose) produced plausible trajectories.
Comparison with Baselines: In all experiments, baselines that ignored the causal graph (causally blind neural operators) or ignored time (static SCMs) produced biased estimates and failed to recover ground-truth mechanisms.

Significance and Claims

The paper claims that neural operators provide a "natural foundation" for causal reasoning in continuous time. By treating nodes as functions and mechanisms as maps between function spaces, Orca addresses the limitations of static SCMs and discrete-time approximations.

Interpretability: The learned operators (particularly kernel-based variants) offer interpretable insights into how past inputs influence current states (e.g., decaying exercise effects).
Limitations: The authors are modest about the scope. They note that Orca assumes the causal graph is known and causally sufficient; unobserved confounders would bias results. Furthermore, counterfactual reasoning relies on assumptions about the noise structure (additivity or monotonicity) which cannot be verified from data alone. The current validation is limited to synthetic data, and the framework has not yet been tested on real-world longitudinal data like electronic health records.

The work positions itself as a step toward a unified theory of causal learning for continuous, irregular, and cyclic systems, bridging the gap between structural causal models and modern deep learning architectures for function spaces.