Artificial Agency Program: Curiosity, compression, and communication in agents

This paper proposes the Artificial Agency Program (AAP), a research agenda and falsifiable framework for developing resource-bounded AI agents driven by curiosity-as-learning-progress, which unifies concepts like predictive compression, intrinsic motivation, and interface quality to enhance human-tool systems through staged experiments and a concrete multimodal testbed.

The Gist

Imagine you are trying to teach a robot to be smart. Most current AI research is like building a super-fast race car engine and then asking, "How fast can this go?" The Artificial Agency Program (AAP) paper by Richard Csaky argues that we are asking the wrong question.

Instead of just building a faster engine, we should be asking: "How does this car drive in real traffic, with a limited gas tank, a driver who gets tired, and a map that is only half-finished?"

Here is the paper explained through simple analogies and metaphors.

1. The Core Idea: The "Smart Assistant" vs. The "Oracle"

Current AI is often trained like an Oracle: it sits in a library with infinite memory, reads every book ever written, and answers questions perfectly. But in the real world, we don't have infinite time, energy, or sensors.

The AAP proposes building AI as an Explorer.

• The Explorer has a backpack with limited space (memory).
• The Explorer has a flashlight with a dying battery (energy).
• The Explorer can only see what's in front of them, not the whole world (partial observation).

The paper argues that true intelligence isn't about knowing everything; it's about knowing how to manage your limited resources to learn, act, and survive in a messy, real world.

2. The Engine of Curiosity: "The Compression Game"

How does this Explorer get curious? It doesn't just look for "new" things (like a child staring at a flashing light). Instead, it plays a game of Compression.

• The Analogy: Imagine you are trying to describe a movie to a friend.
  • If the movie is random static (noise), you can't compress it; you have to describe every single frame. That's boring and useless.
  • If the movie is a simple cartoon where a ball bounces up and down, you can compress it into one sentence: "Ball bounces." That's easy.
  • True Curiosity happens in the middle. The Explorer looks for patterns that are just hard enough to be interesting but just simple enough that it can figure out a better way to describe them.

The AI gets a "reward" (like a dopamine hit) not just for seeing something new, but for getting better at predicting the future with less effort. It wants to turn a complex, confusing world into a simple, predictable story.

3. The Budget: The "Three Buckets"

The paper suggests that an intelligent agent has a limited budget of "tokens" (units of effort) to spend every second. It has to decide how to split this budget between three buckets:

1. Observation (Looking): "Should I spend energy to look at this new object?"
2. Deliberation (Thinking): "Should I spend energy to think about what I saw?"
3. Action (Doing): "Should I spend energy to move or talk?"

The Metaphor: Think of this like a video game character with a stamina bar.

• If you run around wildly (high action) without looking, you get lost.
• If you stare at a wall (high observation) without moving, you go nowhere.
• If you think too much without acting, you freeze.
• The Goal: The AI learns to dynamically shift its stamina. If the situation is confusing, it spends more on thinking. If it's safe, it spends more on moving.

4. The "Interface" Problem: The Glass Wall

A major point of the paper is about the Interface—the glass wall between the AI and the real world.

• Is the glass foggy? (Bad sensors)
• Is the glass thick? (Slow reaction time)
• Is the glass cracked? (Noisy data)

The paper introduces a concept called Unification. Imagine the AI is a person wearing a pair of heavy, foggy goggles and thick gloves.

• Bad Interface: The AI tries to solve a puzzle but can't see the pieces clearly or can't pick them up.
• Good Interface (Unification): The AI realizes, "Hey, if I spend some of my energy budget to upgrade my goggles or get thinner gloves, I can solve the puzzle much faster."

The AI should be willing to "spend" energy to improve its own senses and tools if it helps it learn better in the long run.

5. The "Inner Monologue": Talking to Yourself

We often think AI needs to talk out loud (like a chatbot) to be smart. This paper suggests that talking is just one tool, and sometimes a wasteful one.

• The Analogy: Imagine you are solving a math problem.
  • Option A: You write every step down on a piece of paper (Public Language).
  • Option B: You just think the steps in your head (Latent Deliberation).
  • Option C: You whisper a quick reminder to yourself (Private Tokens).

The paper argues that AI should have a "Private Channel." It should be able to "think" in a secret language or hidden symbols that are faster and more efficient than writing out full sentences. It should only "speak" (output text) when it needs to talk to a human or another machine. This is like a pilot having a private radio channel for their co-pilot, rather than shouting instructions to the whole airport.

6. The Ultimate Goal: The "Human-Tool" Team

The paper concludes that we shouldn't judge AI by how smart it is in isolation. We should judge it by how well it fits into a Human-Tool System.

• The Metaphor: A hammer is not "smart" on its own. But a hammer in the hands of a carpenter is a powerful tool.
• The AI is the hammer. The human is the carpenter.
• If the hammer is too heavy, too slippery, or requires too much energy to swing, it's a bad tool, even if it's made of the strongest steel.
• The goal of AAP is to build AI that feels like a perfect extension of the human hand—easy to use, efficient, and perfectly tuned to our limitations.

Summary: What does this mean for the future?

This paper is a call to stop building "God-like" AI that knows everything but costs a fortune to run and is hard to control. Instead, it wants us to build "Survivor" AI:

• Agents that know their limits.
• Agents that get curious about things they can actually understand.
• Agents that know when to think, when to act, and when to save energy.
• Agents that treat "talking" as just one option in a toolbox, not the only way to think.

It's about moving from Raw Power to Smart Efficiency.

Technical Summary

1. Problem Statement

Contemporary frontier AI systems (e.g., Large Language Models) are often trained and evaluated under conditions that diverge significantly from biological agency. These systems typically possess superhuman memory for textual regularities but lack the developmental constraints of reality-embedded interaction, finite compute/memory, constrained sensing/actuation, and the necessity to act under uncertainty.

The paper argues that simply increasing raw capability is insufficient if the resulting system is hard to steer, poorly coupled to human intent, or inefficient under real-world time, energy, and communication budgets. Current AI often fails to treat agency as a property of the coupled human–tool–environment system. The core problem is the lack of a unified framework that integrates intrinsic motivation, information theory, thermodynamics, and bounded rationality to build agents that are efficient, interpretable, and capable of adaptive resource allocation.

2. Methodology and Formal Setup

The author proposes the Artificial Agency Program (AAP), a research agenda and experimental framework grounded in the following formalisms:

A. The Embedded Agent Model

The agent is modeled as a partially observed controlled process interacting with a hidden environment state ($X_t$). The system is constrained by dynamic capacities:

• Observation ($c^O_t$): Sensor bandwidth, noise, latency.
• Actuation ($c^A_t$): Action alphabet and determinism.
• Compute/Interface ($c^C_t$): Context length, update budget, deliberation tokens.

These capacities are treated as agent-controllable variables that can be modified (e.g., widening a sensor or increasing memory) at a cost.

B. Objective Function: Curiosity as Learning Progress

Instead of raw novelty or surprise, the intrinsic reward is defined as learning progress (improvement in predictive compression).

• Intrinsic Reward ($r_t$): The reduction in predictive loss ($L_{pred}$) over a horizon $H$. It rewards the agent for compressing spatiotemporal patterns better than it could previously.
• Total Objective ($J$): A weighted sum of intrinsic reward minus explicit costs:
  $$J = \mathbb{E} \left[ \sum \gamma^{t-1} (r_t - \lambda_O C_O(t) - \lambda_E C_E(t) - \lambda_C C_C(t) - \lambda_M C_M(t)) \right]$$
  Where costs include observation processing, actuation energy, compute/deliberation, and memory maintenance.

C. Key Metrics and Concepts

1. Empowerment: The channel capacity from action sequences to future observations (control over the environment).
2. Plasticity: The directed information from observations to actions (reactivity/adaptability).
3. Unification (Interface Quality): A score ($U_t$) measuring the reduction of bottlenecks (lossiness, noise, latency) between the agent and the environment. The hypothesis is that agents will invest resources to maximize $U_t$ only when it yields long-horizon returns.
4. Language as a Bottleneck: Language is treated not as a privileged mode of thought, but as a selective, lossy communication channel. The framework distinguishes between:
   • Private tokens ($S_s$): Internal deliberation (hidden state updates).
   • Public tokens ($S_o$): External communication.
   • The agent must decide when to use "thinking" (deliberation) vs. "speaking" (communication) based on cost-benefit analysis.

3. Key Hypotheses

The paper formulates five falsifiable hypotheses:

• H1 (Pragmatic Alignment): Interventions that improve learning progress (compression) will generally increase useful control (empowerment) over task-relevant variables.
• H2 (Boundary Pressure toward Unification): Agents will allocate resources to improve sensing/acting interfaces only when the long-horizon return justifies the cost, leading to monotonic gains in interface quality until costs dominate.
• H3 (Constraint-Induced Pressure): Under strict viability and cost constraints, agents are forced to optimize for better prediction and selective control to reduce wasted computation and costly reactive plasticity.
• H4 (Adaptive Compute Optimality): A meta-controller that dynamically allocates budget among observation, action, and deliberation will outperform fixed schedules under the same total budget.
• H5 (Self-Communication Bottleneck): Explicit private self-communication tokens (reasoning traces) improve performance on long-horizon tasks compared to latent-only recurrence, provided the channel is bandwidth-regularized.

4. Experimental Agenda

The paper outlines a staged approach to validate these hypotheses:

1. Stage 1: Synthetic POMDPs: Toy gridworlds with controllable sensor noise, latency, and action cardinality to calibrate metrics and test H1–H3.
2. Stage 2: ARC-AGI-style Inference: Interactive tasks emphasizing compositional priors and generalization to test the trade-offs between observation, deliberation, and action costs.
3. Stage 3: Multimodal VLA Meta-Control: Using a frozen multimodal backbone with a lightweight meta-controller to decide when to observe, act, or deliberate in real-world-like settings.

Evaluation Metric: The primary success metric is not raw task score, but Pareto optimality on an energy/compute–performance frontier. Systems should minimize the distance to the optimal frontier, demonstrating adaptive resource allocation.

5. Results and Expected Outcomes

Note: As this is a position paper and research agenda, it does not present final empirical results but rather a roadmap for expected outcomes.

• Theoretical Contribution: The paper unifies predictive compression (Schmidhuber), empowerment (Klyubin), and bounded rationality (Lieder/Griffiths) into a single objective function.
• Proposed Architecture: A modality-agnostic token taxonomy allowing agents to interleave input ($V_i, S_i$), private deliberation ($S_s$), and output ($V_o, S_o$) dynamically.
• Training Strategy: A proposed method for training "hidden state deliberation" involves inserting "blank" tokens into reasoning traces when model confidence is low, forcing the model to learn when to pause and think without emitting language tokens.

6. Significance and Impact

The Artificial Agency Program offers a paradigm shift in AI development:

• From Capability to Agency: It moves the focus from "how smart is the model?" to "how effectively does the coupled human-tool system operate under constraints?"
• Efficiency as a First-Class Citizen: By explicitly modeling energy, memory, and communication costs, it addresses the sustainability and scalability of AI.
• Reframing Language: It challenges the dominance of language as the sole medium for reasoning, proposing that deliberation (internal state updates) and communication (external tokens) are distinct, budgeted resources.
• Scientific Rigor: By defining falsifiable hypotheses and specific experimental stages (from synthetic to multimodal), it provides a concrete path to test the intersection of thermodynamics, information theory, and AI.

In summary, the paper argues that true intelligence in AI must be embedded, resource-bounded, and driven by the pressure to compress and predict the world, with the ultimate goal of optimizing the human-tool-environment interface rather than just maximizing model scores.