🤖 AI

CARD: Diagnosing Belief to Action Routing Failures in Vision Language Models

This paper introduces CARD, a diagnostic method that reveals vision-language models fail to effectively route internal belief representations into their action predictions, a limitation identified through a new cooperative grid-world benchmark called Relay Chain.

Original authors: Souptik Kumar Majumdar, Fabian Kögel, Andreas Bulling

Published 2026-08-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Souptik Kumar Majumdar, Fabian Kögel, Andreas Bulling

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the growing field of artificial intelligence, researchers are increasingly interested in whether machines can understand the minds of others. This ability, known as a "theory of mind," allows a person to infer what someone else sees, knows, or wants, even when that information is hidden from their own view. It is the mental glue that makes cooperation possible, letting us predict a partner's next move based on their perspective rather than just our own. As artificial intelligence systems are deployed to work alongside humans—as robotic assistants, driving co-pilots, or team members—scientists need to know if these systems possess this same capacity. If an AI cannot truly grasp what its partner knows, it cannot cooperate effectively, no matter how intelligent it appears in other ways.

To investigate this, a team of researchers at the University of Stuttgart developed a new way to look inside the "brain" of modern vision-language models. These are advanced computer programs that can see images and understand language simultaneously. The researchers wanted to know if these models actually use their internal understanding of a partner's mental state when deciding what action to take next. They created a new testing environment called Relay Chain, a cooperative game played on a digital grid. In this game, two agents hold levers that open gates for a third agent to pass through. Success depends entirely on the agents knowing whether their partner can see the traveler crossing. If an agent cannot see the crossing, it must hold the lever; if it can see the crossing, it must release the lever. This setup forces the AI to make decisions based on what it believes its partner sees, rather than just what is happening in the world.

The researchers used a technique called activation steering to probe the models. Imagine the AI's internal thoughts as a vast landscape of signals. The team identified specific signals that represented "belief"—the AI's internal record of what its partner sees. They then artificially strengthened or weakened these belief signals while the AI was playing the game, essentially nudging the model's internal state to be more or less certain about its partner's view. If the AI were truly using its understanding of the partner's mind to make decisions, changing that internal belief should change the action the AI chose. For instance, if the model was nudged to believe the partner could see the traveler, it should release the lever. If nudged to believe the partner could not see, it should hold on.

The results revealed a critical failure in how these models work. The experiments showed that the models did indeed possess a clear, readable representation of the partner's belief. When asked directly, "Does your partner see the traveler?", the models answered correctly, and the researchers could even change the answer by steering the internal signals. However, when the models were asked to perform the cooperative action—deciding whether to release the lever or hold it—their internal belief signals were completely ignored. Even when the researchers forcefully altered the model's internal belief about what the partner saw, the model's decision on what to do remained exactly the same. It acted as if it had a perfect memory of the partner's view but simply chose not to use that memory when making a move.

This disconnect was consistent across four different types of open-source models tested, ranging from smaller to larger systems. The models were not failing because they lacked the information; they were failing because they were not routing that information to the part of the system that decides on actions. The researchers ruled out several other possibilities, such as the models being confused by the wording of the questions or the steering method being too weak to register. They tested dozens of different ways to phrase the instructions and found that no amount of prompting could force the models to use their internal beliefs to guide their actions. The models were effectively "blind" to their own knowledge when it came to taking a step.

The study concludes that while these artificial intelligence systems can be trained to describe a partner's mental state, they do not automatically connect that description to their own behavior. They can talk about what someone else knows, but they do not use that knowledge to decide what to do. This suggests that simply teaching a model to answer questions about beliefs is not enough to create a truly cooperative agent. For AI to work seamlessly with humans, the internal understanding of others must be wired directly into the decision-making process, a connection that current models appear to lack.

Technical Summary: CARD – Diagnosing Belief-to-Action Routing Failures in Vision–Language Models

Problem Statement

Vision-Language Models (VLMs) are increasingly deployed in cooperative human-AI scenarios (e.g., embodied robots, co-pilots) that require Theory of Mind (ToM)—the ability to infer a partner's beliefs, knowledge, and intentions. While existing diagnostic tools like linear probing and activation steering have confirmed that VLMs internally represent these mental states, a critical gap remains: it is unclear whether these internal representations are actually routed to downstream action predictions.

Current behavioral benchmarks evaluate model outputs along belief, knowledge, and intention axes but fail to distinguish between two failure modes: (1) the model lacks the representation entirely, or (2) the model possesses the representation but fails to utilize it when predicting actions. Similarly, prior activation steering work typically operates "within-axis" (steering a belief representation to influence a belief prediction), leaving the "cross-axis" routing (steering a belief representation to influence an action prediction) unexamined. This paper addresses the question: Does a VLM's internal representation of a partner's mental state causally drive its cooperative actions?

Methodology

1. Cross-Axis Routing Diagnostic (CARD)

The authors introduce CARD, a mechanistic diagnostic framework designed to test cross-axis routing. Unlike standard steering which steers and reads along the same axis, CARD operates as follows:

Probe Extraction: For a specific mental state axis (e.g., belief), a linear probe is trained on paired activations (True-Belief vs. False-Belief scenarios) to identify a unit-norm steering direction $v_a$ in the model's residual stream.
Cross-Axis Intervention: At inference time, the model's residual stream is hooked at a specific layer, and the steering vector $v_a$ is added (or subtracted) to the activations.
Cross-Axis Read-out: The model's prediction is measured on a different axis (e.g., action/intent).
Metric: The causal influence is quantified by the "gap-shift" ( $M_{A,B}$ ), measuring how the True-Belief vs. False-Belief accuracy gap on the target axis changes when the source axis is steered. A non-zero shift indicates successful routing; a null shift indicates a routing failure.

2. Relay Chain Benchmark

To evaluate CARD, the authors propose Relay Chain, a new cooperative grid-world benchmark.

Task: Three agents (two lever-holders, one traveller) must cooperatively open gates for the traveller to reach a goal. The lever-holders must release their levers based on whether they see the traveller crossing.
Design: Scenarios are constructed with True-Belief (TB) (the agent sees the event) and False-Belief (FB) (the event occurs but is occluded from the agent's view) variants. Crucially, "Counter" scenarios are included where the event never happens, preventing models from solving the task via a constant prior (e.g., always "wait") or world-state tracking alone.
Axes: The benchmark evaluates three axes:
- Belief: Does the protagonist perceive the event?
- Knowledge: Does the protagonist have sufficient perceptual access to know the event status?
- Intent (Action): Should the protagonist take the cooperative action (release lever) now?

3. Experimental Setup

The study evaluates four open-weight VLMs (Qwen2-VL-7B, Gemma-4-E4B-it, LLaVA-NeXT-Video-7B, InternVL2.5-8B) across 444 scenarios. The authors employ rigorous controls, including:

Positive Controls: Verifying that belief is linearly decodable and that steering affects belief/knowledge predictions.
Negative Controls: Projection-removal (subtracting the belief direction), prompt-resistance tests (12 prompt variants), and CLEVR benchmarks (to ensure steering hooks are functional).
Ablations: Testing different token positions and model scales (up to 32B parameters).

Key Results

1. Belief is Encoded but Not Routed to Action

The experiments reveal a consistent routing failure across all four VLM families:

Decodability: Belief representations are linearly decodable from the residual stream with high accuracy (78–94%).
Within-Axis Success: Steering the belief direction successfully shifts predictions on the belief and knowledge questions.
Cross-Axis Failure: Steering the belief direction has no causal effect on the intent/action prediction. The accuracy of the action prediction remains invariant to the steering intervention, regardless of the model or protagonist role.

2. Behavioral Dissociation

Behavioral analysis confirms a three-way dissociation:

Belief Questions: Models act as "world-state trackers," correctly identifying events regardless of the protagonist's view (failing to gate by observation).
Knowledge Questions: Models show partial belief-conditionality, tracking the protagonist's observation.
Intent Questions: Models collapse to a constant per-protagonist prior (e.g., "don't release" for lever-holders, "keep waiting" for the traveller). They fail to consult either the world state or the protagonist's observation when deciding on an action.

3. Ruling Out Alternatives

The authors systematically rule out alternative explanations for this failure:

Amplitude Limitation: Projection-removal (subtracting belief) also fails to shift the action prediction, proving the issue is not merely that additive steering is too weak.
Prompting Artifacts: Extensive prompt engineering (chain-of-thought, cost framing, in-context examples) failed to recover belief-conditional action.
Hook/Architecture Issues: Steering works on non-ToM tasks (CLEVR) and on other axes (belief/knowledge) within the same models, confirming the steering mechanism is functional and the failure is specific to the intent prediction.
Scale: The routing failure persists in the 32B parameter Qwen2.5-VL model, indicating it is not a limitation of smaller model scales.

Significance and Claims

The paper claims to identify a structural routing failure in current VLMs. The core finding is that while VLMs encode a partner's mental state well enough to answer questions about that state, they do not utilize these representations when predicting actions in a cooperative setting.

The authors emphasize that this is a routing fact, not an absence of representation. The information is present and readable by other parts of the model (belief/knowledge heads) but is effectively disconnected from the action head.

Implications:

Alignment Gap: The results suggest that "verbal-ToM alignment" (training models to answer correctly about beliefs) does not imply "action-ToM alignment." Explicit objectives on action prediction are necessary to close this routing gap.
Diagnostic Utility: CARD provides a new tool to distinguish between models that lack mental state representations and those that possess them but fail to use them for decision-making.

The authors remain modest regarding generalizability, noting the findings are specific to the Relay Chain task family and the tested open-weight models, and that nonlinear probes might reveal subspace routing missed by linear methods. However, the consistency across diverse architectures and scales suggests a fundamental limitation in current VLMs for cooperative embodied AI.