Technical Summary: Using Theory of Mind to Arbitrate between Social and Non-social Learning

Problem Statement

Social learning allows agents to acquire information from others, yet humans do not rely on it indiscriminately. They must decide when to observe, whom to observe, and how long to watch, balancing the potential information gain against the cognitive and time costs of observation versus independent exploration. Existing computational models of social learning generally fall into two categories:

1. Heuristic accounts: These rely on fixed rules (e.g., "copy the majority" or "copy the prestigious") and treat observed behavior as a signal to be weighted, ignoring the generative mental states (goals and beliefs) that produced the behavior.
2. Reinforcement Learning (RL) accounts: These weight social information based on estimated source reliability (volatility) but similarly treat behavior as a noisy signal rather than evidence of underlying mental states.

Both approaches struggle to explain efficient learning from sparse, indirect observations where the critical inference is not what an agent did, but why they did it (i.e., their goals and beliefs). Furthermore, while active information seeking literature addresses the cost-benefit trade-off of non-social queries, extending this to social sources is complex because the informativeness of an observation depends on the observed agent's latent goals and beliefs, which may differ from the observer's.

Methodology

Experimental Paradigm

The authors introduced a novel grid-world game (12x11) to create parametrically controllable trade-offs between social and non-social learning.

• Setup: A human-controlled "red agent" must retrieve a specific treasure chest. The path is blocked by colored barriers requiring matching amulets held by specific wizards (among decoys).
• Mechanics: At each timestep, the agent can either act (move one step, incurring a cost of 3 points) or observe a selected Non-Player Character (NPC) (incurring a cost of 1 point per step, during which only the NPC moves).
• Variables: The study varied:
  • Goal Uncertainty: Single vs. multiple treasure chests (Experiments 1 vs. 2).
  • Agent Availability: Single vs. multiple NPCs (Experiment 3).
  • Agent Expertise: Expert (knows amulet location) vs. Novice (does not know) (Experiment 4).
• Cost Structure: Movement costs 3 points, wizard interaction costs 5 points, and observation costs 1 point. Participants were incentivized to minimize total cost.

The Rational Mentalizing Model

The authors propose a Rational Mentalizing model that integrates Bayesian Theory of Mind (ToM) with rational cost-benefit analysis. The model operates by comparing the expected utility of acting immediately versus observing an agent.

1. Generative Model of the Observed Agent: The observer models the NPC as a near-optimal planner in a partially observable environment. The observer maintains a posterior distribution over the NPC's latent goals ($g$) and beliefs ($b$) using a particle filter (Sequential Inverse Plan Search, SIPS).
2. Simulation and Prediction: For each hypothesis $\theta = (g, b)$, the observer simulates the NPC's future trajectory using a Boltzmann policy. It predicts:
   • $T^i$: The number of observation steps required for the NPC's trajectory to narrow the observer's candidate set of wizards to one (or falsify the hypothesis).
   • $C^i_{plan}$: The residual exploration cost the observer would face after acquiring this information.
3. Utility Arbitration: The model computes the expected cost of observing agent $j$:
   $$Q^j_{obs}(t) = \sum_i p^i_t [c_{obs} T^i + C^i_{plan}]$$
   where $p^i_t$ is the posterior weight of hypothesis $i$.
   The observer compares this against the cost of acting immediately, $C_{self}(b_t)$ (estimated via a greedy planner).
   • If $Q^j_{obs}(t) < C_{self}(b_t)$, the model observes agent $j$.
   • Otherwise, it acts.
   • In multi-agent settings, the model selects the agent $j^*$ that minimizes $Q^j_{obs}(t)$.

Ablation Studies

To isolate the contributions of specific components, the authors compared the full model against three ablations:

• Rational Observer: Performs cost-benefit analysis but lacks ToM (does not infer goals/beliefs).
• Mentalizing Observer: Infers goals/beliefs but lacks utility estimation (does not weigh observation cost against exploration cost).
• Naive Observer: Observes indiscriminately based on fixed rules (e.g., watch until first wizard interaction) with no inference or cost reasoning.

Key Results

Quantitative Fit

Across four experiments with increasing complexity, the Rational Mentalizing model achieved high fidelity in capturing human observation behavior:

• Correlation: Per-experiment Pearson correlations ranged from $r = 0.90$ to $0.95$.
• Concordance: The pooled Lin's concordance correlation coefficient (CCC) was 0.89 (95% CI [0.84, 0.92]), approaching the upper bound of human inter-participant reliability ($r_{SB} = 0.83-0.92$).
• Comparison: All ablation models performed significantly worse (CCC $\leq 0.21$).

Qualitative Patterns and Ablation Insights

• Goal Uncertainty (Exp 2): When the NPC's goal was ambiguous, humans and the full model observed longer to disambiguate. The Mentalizing Observer over-observed because it could reason about goals but failed to weigh the cost of observation against the benefit. The Rational Observer under-observed because it could not anticipate that the NPC's trajectory would be informative.
• Agent Selection (Exp 3): When two experts were present, humans and the full model selectively observed the agent whose trajectory was most likely to be informative for the observer's specific goal. The Naive and Rational Observers failed to distinguish between agents, while the Mentalizing Observer identified the informative agent directionally but failed to allocate observation time efficiently.
• Expertise Calibration (Exp 4): When an expert and a novice were present, humans and the full model dynamically shifted observation based on the environment (e.g., observing the novice when their path was more diagnostic in sparse wizard maps). The Mentalizing Observer over-observed the novice in cases where the information gain did not justify the cost. The Rational and Naive Observers defaulted to the expert, failing to utilize the counterfactual signal from the novice.

Observation vs. Execution

The full model correctly reproduced the positive coupling between observation and execution steps found in human data (more observation correlates with more execution steps). The Mentalizing and Naive observers inverted this relationship (negative slopes), while the Rational Observer showed a weaker positive slope with a confidence interval including zero.

Significance and Claims

The paper claims that selective social learning is guided by Theory of Mind in the service of utility maximization.

1. Integration of ToM and Utility: The study demonstrates that neither ToM alone nor cost-benefit analysis alone is sufficient. Humans (and the model) require both: the ability to infer the generative mental states of others to predict informativeness, and the ability to arbitrate this potential gain against the cost of observation.
2. Beyond Signal Reliability: Unlike RL accounts that arbitrate based on source reliability (volatility), this model shows that arbitration depends on the inferred goals of the source. An agent's behavior is only informative if their goals align or interact meaningfully with the observer's objectives.
3. Generative View of Behavior: The results support the view that observed behavior should be treated as evidence about the goals and beliefs that produced it, rather than a fixed signal. This allows for efficient learning from sparse, indirect data where agents have different objectives.
4. Unified Mechanism for Selective Trust: The paper suggests that "selective trust" (preferring experts) and "selective observation" (choosing whom to watch) may be products of a unified Bayesian inference over agent models, rather than distinct cognitive systems.

The authors note limitations, including that the model treats observation as the only social channel (ignoring communication/pedagogy) and assumes independent trials without accumulating agent-specific priors. Additionally, the model systematically underpredicts the absolute amount of observation humans engage in, suggesting a role for intrinsic curiosity or conservative evidence gathering beyond strict utility maximization. However, the model successfully captures the relative patterns of human decision-making across complex social learning environments.