💻 computer science

Latent Perspective-Taking via a Schrödinger Bridge in Influence-Augmented Local Models

This paper proposes a neuro-symbolic framework that combines Influence-Augmented Local Models with an amortized Schrödinger Bridge to enable robots to learn structured mental models and perform efficient, perspective-taking inference for socially aware decision-making in uncertain environments.

Original authors: Kevin Alcedo, Pedro U. Lima, Rachid Alami

Published 2026-02-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Kevin Alcedo, Pedro U. Lima, Rachid Alami

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a robot trying to navigate a busy room full of people. To move safely and politely, the robot can't just look at where people are; it has to guess what they are thinking and planning to do next. This is the core challenge the paper tackles: teaching robots to "put themselves in someone else's shoes" without actually being able to read minds.

Here is a breakdown of their solution using simple analogies:

1. The Problem: The "Mind-Reading" Nightmare

Usually, for a robot to guess what a human is thinking, it has to build a complex mental model of that human's brain, then a model of the human's model of the robot, and so on. The paper calls this "nested belief."

The Analogy: Imagine trying to solve a puzzle where every piece you pick up turns into a new, smaller puzzle, which then turns into another. It gets too big, too fast, and the robot gets stuck (this is what the authors call "intractable").
The Old Way: Previous methods either tried to be too perfect (and failed) or used rigid, pre-written rules that broke as soon as the real world got messy.

2. The Solution: The "Influence-Augmented Local Model" (The Smart Filter)

Instead of trying to understand the entire room and every single person in it, the robot focuses only on what matters to it right now.

The Analogy: Think of the robot wearing a pair of special glasses. These glasses blur out the background noise and only highlight the "influence points"—the specific things that actually affect the robot's next move (like a person walking toward it).
How it works: The robot breaks the world down into small, manageable chunks: its own local situation, the specific "social influence" coming from others, and random outside factors. This keeps the math simple and fast.

3. The Core Innovation: The "Schrödinger Bridge" (The Time-Traveling Simulator)

This is the most technical part, but here is the simple version. The robot needs to take its own current belief ("I think the person is going left") and transform it into a belief about what the other person is thinking ("They think I am going right").

The authors use a mathematical tool called a Schrödinger Bridge.

The Analogy: Imagine you have a map of a river (the robot's current path). You want to know what the river would look like if it were flowing in the opposite direction or if the wind were blowing differently.
The "Bridge": The Schrödinger Bridge is like a magical bridge that calculates the most likely path to get from "My Current Thought" to "Their Thought" without breaking the laws of physics (or in this case, the laws of how the world works).
The "Amortized" Part: Instead of calculating this bridge from scratch every single second (which is slow), the robot learns a "shortcut" beforehand. It's like memorizing the best route through a maze so you don't have to re-solve the puzzle every time you enter.

4. The "Neuro-Symbolic" Brain

The robot's brain is a hybrid of two types of thinking:

Neural (The Intuition): It uses deep learning to look at raw data (pixels, sensor readings) and turn them into simple, discrete concepts (like "Person is near," "Door is open").
Symbolic (The Logic): It uses structured rules (like a flowchart) to make sure these concepts make sense together.
The Analogy: It's like having a painter who can sketch a scene quickly (Neural) and an architect who checks if the sketch follows the rules of gravity and building codes (Symbolic).

5. How They Tested It

They tested this in a simple grid-world game (like a digital version of Pac-Man or a maze).

The Task: The robot had to follow a person to a target while following social rules (like not bumping into them).
The Result: The robot using this "Schrödinger Bridge" method learned faster and got better scores than robots that just guessed randomly or tried to use perfect information (which isn't realistic in the real world).
The Takeaway: By simulating "what if I were them?" at the moment of decision, the robot made smarter choices.

Summary

The paper proposes a new way for robots to understand human intentions. Instead of getting lost in complex, endless calculations about what humans are thinking, the robot uses a smart filter to focus on what matters and a mathematical bridge to quickly simulate the other person's perspective. This allows the robot to make socially aware decisions in real-time, learning faster and acting more naturally.

Technical Summary: Latent Perspective-Taking via a Schrödinger Bridge in Influence-Augmented Local Models

Problem Statement
Operating in environments alongside humans requires robots to make decisions under uncertainty, necessitating reasoning not only about exogenous dynamics but also about the hidden mental models and mental states of others. While frameworks like Interactive POMDPs (I-POMDPs) and Bayesian Theory of Mind (BToM) provide principled approaches, they face significant limitations: exact nested-belief inference is computationally intractable (PSPACE at level-0, EXPSPACE in worst-case scenarios), and hand-specified symbolic models are brittle in open-world settings with stochastic dynamics. Conversely, standard probabilistic sequential decision-making handles uncertainty but typically relies on prescribed models and still suffers from the "nested-belief explosion." The core challenge is to unify symbolic knowledge representation with probabilistic decision-making to learn structured mental models and a tractable estimator for others' mental states.

Methodology
The authors propose a data-driven architecture that decomposes socially-aware robot tasks using Influence-Augmented Local Models (IALMs) and performs perspective-taking via an amortized Schrödinger Bridge (SB).

Influence-Augmented Local Model (IALM):
Building on the Influence-Based Abstraction (IBA) framework, the authors instantiate an IALM to factorize the global state into local dynamics, social influences, and exogenous factors. The model defines a minimal $d$ -separating set ( $D_t$ ) from the action-local-state history, which serves as a sufficient statistic to predict the influence distribution. This allows the agent to compute an exact best response for fixed external policies without reasoning over the full global space.
Neuro-Symbolic World Model:
The IALM is implemented as a neuro-symbolic world model composed of a hierarchy of discrete variational autoencoders (dVAEs) with Gumbel-Softmax. This architecture learns a factored, discrete Dynamic Bayesian Network (2DBN) end-to-end.
- Encoding: Raw observations are encoded into a latent space of discrete factors.
- Transition: A local, influence-naïve transition model ( $\dot{T}$ ) maps current factored states and actions to the next latent state. It utilizes $\alpha$ -entmax cross-attention to induce exact-zero, data-driven sparsity, explicitly modeling latent parent-child structures.
Perspective-Shift Operator (Schrödinger Bridge):
To estimate others' beliefs, the authors introduce a perspective-shift operator, $\Phi_{SB}$ , modeled as an amortized Schrödinger Bridge. This operator transports factored egocentric beliefs ( $b_0$ ) into other-centric beliefs ( $\hat{b}_i$ ) over the learned local dynamics.
- Mechanism: The SB is formulated as a dynamic entropic optimal transport problem. It seeks a new transition law ( $\bar{T}$ ) that minimizes the pathwise KL divergence to the reference dynamics ( $\dot{T}$ ) while satisfying specific start and end marginals (beliefs).
- Implementation: The bridge is realized via the Doob $h$ -transform, steered by forward ( $\phi$ ) and backward ( $\psi$ ) potentials. These potentials are parametrized by context ( $c_t$ ) derived from the history and are trained offline using "epistemic counterfactuals" (sampled other-centric beliefs).
- Inference: At decision time, the agent "tilts" the reference dynamics $\dot{T}$ using the learned amortized potentials to simulate others' mental-state dynamics, effectively performing model predictive control in mental-state space.

Key Contributions
The paper outlines three primary contributions:

A practical influence-augmented decomposition for social tasks that separates local dynamics from social influences.
A factored, discrete neuro-symbolic world model that learns structured mental models from data.
A perspective-taking mechanism via an amortized Schrödinger Bridge, enabling model-based belief-space planning at decision-time without requiring exact nested inference.

Preliminary Results
The approach was evaluated in a multi-agent, partially observable Minigrid environment involving a "person-following" task where an agent must follow a fixed-policy agent to a target while adhering to social norms.

Baselines: The method was compared against (i) a Perfect-Information baseline (true current belief), (ii) a No-Information baseline (uniform distribution), and (iii) a Reference baseline (multi-step rollout of local dynamics).
Findings: The SB-based perspective-taking approach learned faster and achieved higher episodic returns than all baselines, including the No-Information and Reference rollouts. The results suggest that decision-time, context-aware mental-state planning yields superior performance over context-free estimates.

Significance and Claims
The authors claim that this architecture bridges the gap between symbolic representation and probabilistic decision-making. By casting perspective-taking as belief transport via a Schrödinger Bridge over learned local dynamics, the system produces other-centric beliefs that are compatible with local dynamics but account for the "what-if" scenarios of other agents' mental states. The paper positions this as a step toward unifying knowledge representation and sequential decision-making, offering a tractable alternative to the intractable nested-belief reasoning of traditional I-POMDPs. The work is presented as preliminary, with future directions identified as addressing heterogeneity in other agents' perception and action models.