🤖 AI

Prediction and Empowerment: A Theory of Agency through Bridge Interfaces

This paper proposes a theory of agency in deterministic partially observable worlds that distinguishes between prediction, compression, and empowerment, demonstrating that perfect prediction requires either identifying relevant latent structures or exerting overwrite control, thereby offering a design framework for aligning AI objectives with human intent through refined bridge interfaces.

Original authors: Richard Csaky

Published 2026-05-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Richard Csaky

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a mystery in a giant, dark warehouse. You have a flashlight (your sensors) and a set of tools (your actions). The paper by Richard Csaky is a guide on how to be a truly effective detective in this warehouse, rather than just a lucky one.

Here is the breakdown of the paper's ideas using simple analogies.

1. The "Bridge" Problem

The paper argues that we often misunderstand how we interact with the world. It says there is a Bridge between you and the world.

Your Side of the Bridge: This is what you control. Your flashlight beam, the questions you ask, the buttons you press.
The World's Side of the Bridge: This is what the environment controls. The dust in the air blocking your light, the locked doors, the permission to enter a room, or the weather outside.

The paper claims that to be smart, you can't just look at your side of the bridge. You have to understand that the whole bridge (both your settings and the world's conditions) determines what you see and what you can change.

2. The Three Big Goals: Guessing, Squeezing, and Pushing

The paper studies three things agents try to do:

Prediction (Guessing): Trying to figure out what the hidden state of the warehouse is (e.g., "Is there a cat behind that box?").
Compression (Squeezing): Trying to remember only the important details and forget the noise.
Empowerment (Pushing): Trying to have as much control as possible over what happens next.

The Big Discovery: The paper proves that these three goals are not the same thing. You can be great at one and terrible at the others.

The "Overwrite" Trap: Imagine you want to predict the weather. If you just turn off the thermostat and set it to "Freeze," you have perfectly "controlled" the temperature. But you learned nothing about the actual weather outside. You "overwrote" the reality instead of understanding it. The paper says high "Empowerment" (control) does not automatically mean you have "Prediction" (understanding).
The "Distractor" Trap: Imagine you are in a room with a hidden safe (the real problem) and a flashing red light (a distractor). If you just turn the red light on and off, you feel powerful (high empowerment), but you haven't found the safe.

3. The "Bridge Gap"

The paper introduces a concept called the Bridge Gap. Think of this as a "missing link" in your understanding.

If your "Bridge Gap" is zero, it means your internal map of the world is perfect, and your ability to control the world is perfect. You know exactly what you need to know, and you have the power to do exactly what you need to do.
If the gap is large, you might be guessing blindly, or you might be controlling things that don't matter.

The paper proves that to be a perfect agent, you don't just need to maximize control; you need to minimize this gap. You need to ensure that the things you can see are the same things you can control.

4. The Solution: "Bridge-Gap Pursuit" (BGP)

The author proposes a new way for AI agents to learn, called Bridge-Gap Pursuit.

Old Way: "Try to get the highest score" or "Try to learn the most new things right now."
New Way (BGP): "Try to close the gap."

The agent asks itself:

"Is there a hidden piece of information I need to solve the task?" (If yes, go find it).
"Is there a part of the world I need to unlock to get that information?" (If yes, go unlock it).
"Am I just playing with a toy that doesn't matter?" (If yes, stop).

The Analogy:
Imagine you are trying to open a locked door.

Standard AI: Might try to hit the door with a hammer (high effort, low info) or just stand there guessing the code.
Bridge-Gap AI: Realizes, "I can't see the keyhole because the light is off." It first uses its power to turn on the light (refining the bridge), then it looks for the keyhole, and then it unlocks the door. It values the act of "turning on the light" because that action reduces the "gap" between not knowing and knowing.

5. The "Budget" of the Bridge

The paper also mentions a "budget." Your bridge (your connection to the world) has a limited capacity.

You can't send infinite information out (control) and receive infinite information in (sensing) at the same time if the bridge is narrow.
If you spend all your bandwidth just shaking a toy (distractor control), you have no bandwidth left to listen to the clues.
The paper suggests that smart agents must manage this budget carefully, ensuring they don't waste their "control power" on things that don't help them understand the hidden truth.

Summary

The paper is a warning and a guide for building smarter AI (and understanding human intelligence).

Warning: Just because an agent is powerful or can predict the future doesn't mean it understands the world. It might just be "overwriting" reality or playing with distractions.
Guide: To be truly intelligent, an agent must design its "bridge" to the world so that it can both see the hidden truths and reach the things that matter. It must actively seek out the conditions (like turning on a light or asking a clarifying question) that allow it to close the gap between what it knows and what it needs to know.

The paper concludes that for AI to align with human goals, we need to design these "bridges" carefully, distinguishing between what the agent controls and what the agent understands.

Technical Summary: Prediction and Empowerment: A Theory of Agency through Bridge Interfaces

1. Problem Statement

The paper addresses the challenge of agency under partial observability in deterministic physical or simulated worlds. In such environments, apparent randomness arises from uncertainty regarding initial conditions, fixed "law bits," and unrolled exogenous noise. Standard Partially Observable Markov Decision Process (POMDP) formulations often leave the interface (the bridge between agent and environment) implicit, making it difficult to diagnose failures in prediction, control, and empowerment.

The core problem is distinguishing between different modes of agency:

Prediction: Identifying hidden states or quotients relevant to a task.
Control/Empowerment: The capacity to influence future states.
Interface Refinement: The ability to modify sensing or actuation settings to access better information or control channels.

The paper argues that high empowerment alone is insufficient for agency, as an agent might control irrelevant "distractors" while failing to identify task-relevant hidden states. Conversely, perfect prediction can be achieved by "overwrite control" (forcing a target state regardless of the hidden initial condition) without actually identifying the hidden state. The paper seeks to formalize the conditions under which these objectives align or diverge.

2. Methodology and Framework

The Bridge-Interface POMDP

The authors model sensing and actuation as bridge interfaces split into two components:

Agent-owned side ( $\phi^S, \kappa^S$ ): Settings controlled by the agent (e.g., query format, camera pose, actuator mode, prompt).
Environment-owned side ( $\phi^X, \kappa^X$ ): Channel conditions determined by the world state and laws (e.g., occlusion, lighting, authorization, contact geometry).

The environment is modeled as a deterministic latent bridge-POMDP. Uncertainty is encapsulated in a latent variable $Z$ (including initial state $X_0$ , law bits $\Lambda$ , and exogenous noise $\Xi$ ). A prior $p_0$ over $Z$ drives the system. The agent's policy maps history to actions and bridge settings. The closed-loop trajectory is a deterministic function of $(Z, \Sigma)$ , where $\Sigma$ is a private policy seed.

Key Definitions

Target Quotient ( $Q$ ): A coarse-grained variable representing the distinctions relevant to a family of tasks.
Losslessness: A transcript is lossless for a quotient if the posterior entropy $H(Q | \text{Transcript}) = 0$ .
Bridge Gap: A set of metrics quantifying the "distance" between prediction, control, and observation capabilities. These include:
- $\Delta_{sense}$ : Hidden-state compression error.
- $\Delta_{act}$ : Control capacity deficit (difference between log-reachability and mutual information).
- $\Delta_{V \tilde{V}}$ : Observation control vs. state control mismatch.

Theoretical Tools

The paper utilizes information-theoretic bounds, specifically directed information and mutual information, to derive inequalities relating the agent's history, actions, and the latent environment. It employs the concept of Blackwell refinement to compare experiments and defines bridge potential as a scalar function to be minimized.

3. Key Contributions

A. The Bridge-Gap Theorems

The paper proves universal bridge-gap inequalities (Theorem 1) that bound the difference between information quantities (e.g., $I(Q; T)$ vs. $I(W; T)$ ) using conditional entropies.

Result: If bridge-gap components vanish, prediction, compression, and empowerment objectives align to their lossless or full-capacity values.
Regret Transfer (Theorem 2): The paper establishes that optimizing one objective (e.g., empowerment) as a surrogate for another (e.g., prediction) is only valid if the objective-difference oscillation over the policy class is small. A small gap at a single policy is insufficient; the gap must be uniformly small across the optimization landscape.

B. Separation of Objectives

The paper proves that prediction, identification, and empowerment are arbitrarily separable (Proposition 1):

Zero Empowerment, Perfect Identification: An agent can perfectly identify a latent state without having any control over it.
High Empowerment, Zero Identification: An agent can control a distractor variable while knowing nothing about the relevant latent state.
Perfect Prediction via Overwrite: An agent can perfectly predict a terminal target by overwriting the state (collapsing distinctions) without identifying the initial hidden state.

C. Bridge-Gap Pursuit (BGP) Algorithm

The authors propose Bridge-Gap Pursuit (BGP), a model-based intrinsic-control algorithm.

Mechanism: BGP defines a bridge potential $\Phi(h)$ that aggregates deficits in sensing (posterior entropy), channel reachability, controllability, and counterfactual relevance.
Reward Shaping: The agent receives a reward based on the decrease of this potential: $r_t = r_{task} + \beta(\Phi(H_t) - \Phi(H_{t+1}))$ .
Relevance Gates: The algorithm includes "counterfactual relevance gates" that suppress empowerment over variables independent of the target quotient or future task value, preventing the agent from optimizing irrelevant distractors.
Exactness: In finite domains, BGP minimizes the expected terminal bridge gap via dynamic programming.

D. Finite Shared-Bridge Budget

Theorem 7 establishes a capacity budget for a finite interface transcript $B$ . The sum of outward influence (action-to-observation directed information) and inward evidence (observation-to-action plasticity) is bounded by the entropy of the bridge transcript $H(B)$ . This implies a trade-off: an interface cannot simultaneously maximize control over the world and information gain from the world unless the bridge is widened or the state is internalized.

E. Prediction-Absorption Bound (H2)

The paper formalizes Hypothesis H2: Joint maximization of H1 objectives drives the agent toward a "lossless interface" or absorption of the relevant environment.

Theorem 8: If a terminal quotient $V$ is perfectly predictable from the agent's internal state $M_T$ , then the mutual information between the initial latent quotient $Q$ and $M_T$ is at least the mutual information between $Q$ and $V$ .
Implication: Perfect prediction forces the agent to either identify the hidden state (absorption) or overwrite it (collapse). If the target family is task-separating, perfect prediction implies the agent has absorbed the relevant environmental distinctions.

4. Results and Benchmarks

The paper presents exact finite benchmarks (Section 10, Table 2) comparing BGP against standard objectives (ungated empowerment, one-step information gain/EFE, terminal prediction loss, and coarse training return).

Settable Distractor: Ungated empowerment fails to identify a hidden 4-bit latent ( $H(Q|H_T)=4$ ) by controlling an 8-bit distractor. BGP achieves 100% success with 0 ambiguity.
Delayed Sensor: One-step information gain fails to take a preliminary action that enables future sensing (zero immediate gain). BGP succeeds by valuing channel reachability.
Inspect-Overwrite: Terminal prediction loss may choose to overwrite the target (collapsing the state) rather than inspecting it. BGP selects inspection when the target quotient requires identification.
Quotient Transfer: Coarse training ignores refined bits. BGP, evaluated on refined quotients, resolves the ambiguity.

In all cases, BGP selects the zero-gap option when reachable, whereas baseline objectives suffer from specific failure modes (distractor control, delayed sensing, overwrite collapse, or coarse resolution).

5. Significance and Claims

The paper claims to provide a design principle rather than a theorem of inevitability for modern AI agents.

Design Principle: Objectives should explicitly distinguish between hidden-state identification, interface refinement, task-relevant controllability, and overwrite/distractor control.
Human-AI Alignment: The paper frames alignment partly as an interface-design problem. The "bridge" connects human intent, agent internal state, external tools, and world-side channel conditions. Misalignment can arise if the agent optimizes the wrong side of the bridge (e.g., controlling permissions without understanding intent).
Operationalizing Intelligence: Human intelligence is described as "boundary management"—learning to reach channel states (line of sight, contact, shared attention) that allow for better experiments and compressing task quotients into memory.
Limitations: The exact theorems are finite-horizon and finite-state. Scalable application requires learned estimators for entropy, capacity, and relevance. The paper does not claim that BGP is the only solution, but that it explicitly addresses the "bridge gap" that other intrinsic rewards (like curiosity or empowerment) may miss.

The work suggests that for agents to generalize, they must reduce the "missing sensing bits," "missing authority bits," and "quotient mismatch" that prevent prediction and control from referring to the same part of the world.