⚛️ quantum physics

Agent policies from higher-order causal functions

This paper establishes a mathematical correspondence between agent policies in POMDPs and higher-order process functions, using category theory to demonstrate that agents utilizing indefinite causal structures can outperform those restricted to fixed causal orders in decentralized environments.

Original authors: Matt Wilson

Published 2026-02-10

📖 4 min read🧠 Deep dive

CC BY 4.0

Original authors: Matt Wilson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are playing a video game. In most games, there is a clear "order of operations": you press a button (Action), the game calculates what happens (Environment), and then you see what happened on the screen (Observation). This is the standard "cause and effect" we live by.

This paper, written by Matt Wilson, explores a mind-bending idea: What if the "order" of cause and effect wasn't fixed? What if the game could react to your move before you even made it, or if your move and the game's reaction happened in a tangled loop where neither truly came first?

To explain this, the paper builds a bridge between two very different worlds: Artificial Intelligence (AI) and Quantum Physics.

1. The Two Worlds

The AI World (The "Agent"): Think of an AI agent as a player in a game. The player has a "memory" (what happened in the last round) and uses it to decide their next move. The goal is to get the highest score possible.
The Physics World (The "Process"): In the foundations of physics, scientists study "processes"—the rules of how information flows through space and time. Usually, these rules follow a strict timeline (A causes B). But in the quantum world, things can get "indefinite," meaning the causal order is fuzzy or overlapping.

2. The "Bridge" (The Core Discovery)

The author’s big breakthrough is proving that these two worlds are actually speaking the same mathematical language.

He shows that an AI's strategy (how it uses memory to pick actions) is mathematically identical to a physical process (how information flows through a system).

The Analogy: The Recipe vs. The Chef
Imagine a recipe (the Environment/POMDP) and a chef (the Agent).

In standard AI, the chef reads the recipe, cooks, tastes, and then decides what to do next.
The paper shows that the "Chef's decision-making style" can be treated as a mathematical object that can be "plugged into" the recipe. By treating the chef as a "higher-order function," we can study how the chef and the recipe interact as a single, complex system.

3. The "Superpower": Indefinite Causal Order

This is where it gets wild. Because the math of AI and the math of Physics are the same, the author asks: "Can an AI use 'Quantum-style' logic to win games better than a normal AI?"

In a normal game (Definite Causal Order), Agent A acts, then Agent B acts. It’s a line.
In a "Quantum-style" game (Indefinite Causal Order), the agents act in a way that their actions are "tangled." It’s more like a circle or a knot.

The Analogy: The Synchronized Dance
Imagine two dancers, Alice and Bob, trying to perform a complex move together.

Standard AI (Definite Order): Alice moves, then Bob reacts to her. If they aren't perfectly synced, they might miss the beat.
The "Quantum" AI (Indefinite Order): Alice and Bob move in a way that their actions are so deeply intertwined that you can't say who led and who followed. They are essentially "looping" their information.

The Result: The author proved this using a specific "game" (called the GYNI game). He showed that a "normal" AI is stuck with a maximum score (like a speed limit), but an AI that uses this "indefinite causal" logic can actually break that limit and achieve a perfect score.

Why does this matter?

Better AI: It suggests that if we want to build truly advanced, multi-agent AI systems (like a swarm of robots working together), we shouldn't just teach them to react to each other; we should teach them to use these "tangled" causal strategies.
Quantum Computing: It provides a roadmap for "Quantum AI." It gives us a mathematical way to design AI that is built specifically to run on quantum computers, potentially giving them a massive advantage in solving complex problems.
Understanding Reality: It suggests that the way "agents" (intelligent beings) interact with their world might be deeply connected to the very fabric of how time and causality work in the universe.

Technical Summary: Agent Policies from Higher-Order Causal Functions

Author: Matt Wilson
Affiliation: Université Paris-Saclay, CNRS, ENS Paris-Saclay, Inria, CentraleSupélec, Laboratoire Méthodes Formelles

1. Problem Statement

The paper addresses a fundamental conceptual gap between two established fields:

Artificial Intelligence (AI): Focuses on agents interacting with partially observable environments (POMDPs) to maximize cumulative rewards.
Foundations of Physics: Focuses on "agents in laboratories" modeled via higher-order quantum and classical processes to study causal structures (including indefinite causality).

While both fields study the interaction between an agent and an environment, they lack a formal mathematical correspondence. There is no established bridge to determine if the "indefinite causal structures" used in physics could provide a framework for new, optimal decision-making strategies in multi-agent AI systems, or if higher-order quantum operations could provide a principled way to generalize reinforcement learning into the quantum domain.

2. Methodology

The author employs Category Theory and Type Theory to establish a formal bridge. The methodology follows these steps:

Formalization of Agent-State Policies: The author uses deterministic agent-state policies (comprising a policy function $\pi$ and a memory update function $U$ ) as the AI-side representation.
Process Function Correspondence: On the physics-side, the author uses process functions—the classical-deterministic limit of higher-order quantum operations. These are functions that satisfy a unique fixed-point criterion, allowing them to model closed time-like curves and non-causal structures.
Categorical Construction: The author constructs a $*$ -autonomous category $\mathcal{PF}$ of types. This category supports the interpretation of policy evaluation and multi-agent constraints through categorical operations like cuts and monoidal products.
Mapping Dec-POMDPs: The author identifies observation-independent decentralized POMDPs (dec-POMDPs) as the natural domain for multi-input process functions. This allows the study of agents that do not necessarily follow a fixed, definite causal order.

3. Key Contributions

The Agent-Process Correspondence Theorem: The paper proves a one-to-one correspondence (bijection) between equivalence classes of deterministic agent-state policies and one-input process functions. This means an agent's behavior is mathematically identical to a specific higher-order process function.
Categorical Framework ( $\mathcal{PF}$ ): The development of a $*$ $*$ -autonomous category that provides a purely type-theoretic language for:
- Decentralization.
- Observation-independence (no-signaling).
- Indefinite causal structure.
Generalization to Multi-Agent Systems: The author extends the correspondence to decentralized settings, showing how multi-input process functions can model agents acting in environments where the causal order of their actions is not fixed.
Proof of Causal Advantage: The paper provides a formal proof that "indefinite causal order" is a computational resource that can be exploited to achieve higher rewards in specific multi-agent tasks.

4. Results

Mathematical Equivalence: The author successfully demonstrates that the evaluation of a policy on a POMDP is equivalent to the contraction of a higher-order process function on a lower-order function representing the environment.
The GYNI Game Separation: To prove the practical utility of this bridge, the author introduces a "majority-GYNI" causal game.
- Definite Causal Order: For agents restricted to a fixed causal order, the maximum expected reward is bounded by $3/4$ per round.
- Indefinite Causal Order: Using the Lugano process (a known non-causal process function), agents can achieve a perfect reward of $1$ per round.
Strict Separation: This establishes a strict performance gap, proving that policies utilizing indefinite causal structures can strictly outperform those restricted to a fixed background causal structure.

5. Significance

For AI: It opens a new frontier for Quantum Reinforcement Learning. It suggests that "indefinite causality" is not just a physical curiosity but a potential strategy for multi-agent coordination, potentially leading to more efficient decentralized learning algorithms.
For Physics: It provides a new way to interpret "agents" in the foundations of physics, potentially allowing tools from AI (like planning and learning) to be applied to the study of spacetime and causal structures.
For Computer Science: The use of $*$ -autonomous categories and type theory provides a rigorous logical framework for reasoning about composite, multi-agent systems and the communication constraints between them.