🤖 AI

Artificial Empathy: Towards a Framework for Unsupervised Agency Detection and Policy Reconstruction

This paper proposes a framework that leverages a reinforcement learning agent trained on an independent task as a prior to enable unsupervised detection of other agents and reconstruction of their policies solely from environmental observations.

Original authors: Peter Kuhn, Chris Pang, Sonakshi Chauhan

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Peter Kuhn, Chris Pang, Sonakshi Chauhan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the quiet corners of artificial intelligence research, a fundamental question persists: how can a machine understand that something else in the world is acting with purpose? For a computer to cooperate with humans or other machines, it must first distinguish between a rock falling due to gravity and a person reaching for that rock with intent. This ability, known as agency detection, is the bedrock of social interaction. Without it, an artificial intelligence sees only a chaotic stream of events, unable to tell the difference between random noise and a goal-directed plan. The challenge is even greater when the machine must figure out what that other agent wants to do, a task called policy reconstruction. If a machine could learn to identify these hidden intentions and model them without being explicitly taught, it would take a massive step toward becoming a truly cooperative partner, capable of aligning its own goals with those of the beings around it.

A team of researchers at the Human Inductive Bias Project in Cambridge has taken a significant step toward solving this puzzle by proposing a system they call "artificial empathy." Rather than trying to build a complex theory of mind from scratch, they suggest that an artificial intelligence can understand others by using its own internal mechanics as a guide. The core idea is that an AI, which is itself an agent that makes decisions and acts on the world, already possesses a built-in understanding of what agency looks like. By training a system to predict the future based on its own learned patterns of behavior, the researchers created a framework that can scan an environment, spot other agents, and figure out what those agents are trying to achieve, all without human labels or instructions.

The researchers tested this concept in a controlled digital environment, a simple grid world where an agent moves around to collect rewards. They trained a standard AI agent to solve a specific task, such as reaching the bottom-left corner of the grid to earn points. This trained agent served as the "expert" or the "other" that the new system needed to understand. To make the task difficult and realistic, the researchers introduced a confounding element: a second, fake agent that moved randomly, like a leaf blowing in the wind. The goal for the new system was to ignore the random movement and correctly identify the true agent, then learn to mimic its specific strategy to reach the goal. The system was not told which one was the real agent; it had to figure it out on its own.

To achieve this, the new system was given a head start by loading it with the "brain" of a different AI that had been trained on a completely unrelated task, such as collecting food items. This pre-loaded knowledge acted as a prior, a set of expectations about how agents generally behave and how their actions change the world. The system then observed the stream of events in the grid world. It used its internal model to ask a simple question: if I assume an agent is acting here, can I predict what happens next? When the system looked at the random walker, its predictions failed because the random movements did not follow any logical pattern of goal-seeking. However, when it observed the expert agent, its predictions worked well because the expert's movements were consistent with the logic of an agent trying to reach a goal. By minimizing the error between its predictions and reality, the system effectively filtered out the noise and zeroed in on the true agent.

The results of this experiment were promising. After observing the environment for a short period, the system learned to ignore the random decoy and successfully adopted the strategy of the expert agent. In a series of trials, the system's performance quickly rose above that of a random guess, eventually matching the skill level of the original expert. This demonstrated that the system could indeed perform unsupervised agency detection, identifying the purposeful actor among the chaos and reconstructing its policy. The researchers noted that this success relied on the system using its own internal dynamics as a lens to view the world, effectively saying, "I know what it feels like to have a goal, so I can recognize it in others."

Despite these encouraging findings, the researchers are careful to frame this as an early proof of concept rather than a finished solution. The experiments were conducted in a small, fully visible grid world with only one real agent and one fake one. The system relied heavily on the fact that the prior agent and the target agent were structurally similar, and it is not yet known how well this approach would work if the agents were vastly different or if the environment was complex and partially hidden. The study suggests that self-referential priors can support the detection of agency, but it does not yet claim to have solved the broader problem of aligning artificial intelligence with human values in the real world. The work opens a door, showing that an AI can learn to care about others by first understanding itself, but the journey to a fully cooperative future remains long and untested.

Technical Summary: Artificial Empathy: Towards a Framework for Unsupervised Agency Detection and Policy Reconstruction

Problem Definition

The paper addresses two interconnected challenges in AI alignment and multi-agent systems:

Agency Detection: The task of distinguishing between agentic behavior (goal-directed activity) and non-agentic random dynamics within an environment.
Policy Reconstruction: The task of inferring an agent's underlying policy ( $\pi_O$ ) solely from observed behavior, without access to reward functions or labeled action trajectories.

While Inverse Reinforcement Learning (IRL) addresses policy reconstruction, it typically requires preprocessed observation-action pairs and assumes the agent's identity is already known. The authors argue that solving both tasks jointly in an unsupervised manner is a necessary step toward AI alignment, enabling an AI to develop a "joint policy" that synthesizes its own goals with those of other agents, effectively granting the system a propensity to "care" for others.

Methodology

The authors propose Artificial Empathy (AE), a framework where an AI system uses its own learned agential dynamics as a prior to detect and model other agents. The approach relies on a model-based Reinforcement Learning (RL) architecture.

1. Base Agent Architecture

The foundation is a deep Q-learning agent augmented with a trainable world model, inspired by Variational Autoencoders (VAEs). The architecture comprises:

Encoder ( $p_\theta$ ): A convolutional network inferring latent world states ( $s_t$ ) from observations ( $o_t$ ).
Transition Model ( $q_\phi$ ): A dense network predicting the next latent state ( $s_{t+1}$ ) given the current state and action ( $s_t, a_t$ ).
Decoder ( $d_\omega$ ): Reconstructs observations from latent states.
Q-Network ( $Q_\rho$ ): Predicts Q-values for world states.

The base agent is trained using a combined loss function ( $L_{base}$ ) consisting of:

Reconstruction Loss ( $L_{recon}$ ): MSE between true and reconstructed observations.
Transition Loss ( $L_{trans}$ ): KL-divergence between the modeled posterior and the approximate posterior.
Q-Loss ( $L_Q$ ): MSE between target and predicted Q-values.

2. Artificial Empathy (AE) System

The AE system utilizes a trained base agent (the "prior agent") as a structural prior for agentic dynamics. It attempts to reconstruct the policy of a "target" or "expert" agent acting in an environment.

Initialization: The AE system initializes its weights based on the prior agent's weights, enforcing a Gaussian prior in weight-space via penalty terms ( $R_p, R_q, R_Q$ ).
Prediction Mechanism: The AE system assumes the environment contains an agent. It encodes observations into latent states and predicts the next state ( $p^*(s_{t+1}|s_t, A)$ ) by marginalizing over all possible actions ( $N$ ), weighted by the probability of each action derived from the AE's Q-network:
$p^*(s_{t+1}|s_t, A) = \sum_{i=0}^{N} P(a_i)q'(s_{t+1}|s_t, a_i)$
Loss Function: The system minimizes the discrepancy between the encoded world states and the predicted world states under the assumption of agency. The total loss ( $L_{AE}$ $L_{A E}$ ) includes:
- KL-Divergence ( $L_{KL}$ ): Between the encoded distribution and the agent-predicted distribution.
- Entropy Penalty ( $R_H$ ): To force the system to commit to specific actions rather than smooth distributions.
- Weight Penalties ( $R_p, R_q, R_Q$ ): To maintain structural similarity to the prior agent.

Experimental Setup and Results

The authors validated the framework in a 15×15 gridworld with two distinct tasks:

Food-Game: Collecting three food items.
Edge-Game: Reaching the lower-left edge.

Setup:

An agent trained on the Food-Game served as the prior.
An agent trained on the Edge-Game served as the expert.
The AE system was initialized with the Food-Game prior and tasked with imitating the Edge-Game expert.
Confounding Factor: A "decoy" agent performing a random walk was added to the observation stream to test agency detection.

Results:

The AE system successfully learned to imitate the expert's policy (Edge-Game) despite being initialized with a structurally different prior (Food-Game).
Crucially, the system implicitly solved the agency detection task: it ignored the random dynamics of the decoy agent and focused on the agentic behavior of the expert.
Performance metrics showed that after 35 episodes, the AE agent's cumulative reward significantly outperformed a random walk and approached that of the base expert agent.

Key Contributions

Framework Proposal: Introduction of "Artificial Empathy" as a first-principles framework for unsupervised agency detection and policy reconstruction.
Self-Referential Prior: The novel use of an agent's own dynamics (via a trained world model) as a prior to identify and model other agents, inspired by simulation theory and mirror neuron mechanisms in cognitive science.
Joint Task Solving: Demonstration that a single architecture can simultaneously distinguish agentic from non-agentic activity and reconstruct the underlying policy without labeled data.

Significance and Claims

The paper positions this work as a foundational step toward solving the AI alignment problem. By enabling an AI to detect and model other agents' goals, the system can theoretically construct a joint policy ( $\pi_\Omega$ ) that synthesizes its own objectives with those of others, fostering cooperative behavior.

The authors explicitly state that their current results demonstrate feasibility rather than robustness or scale. They acknowledge that the experiments are limited to small, fully observable gridworlds with a single confounding agent. The paper does not claim to have solved AI alignment but suggests that establishing artificial empathy as a framework is a necessary prerequisite for developing systems capable of "caring" for other agents. Future work is identified as extending the framework to partial observability, multiple agents, and non-stationary policies.