💬 NLP

Language and Experience: A Computational Model of Social Learning in Complex Tasks

This paper presents a computational framework that integrates linguistic guidance with direct experience through joint probabilistic inference over structured world models, demonstrating how this approach accelerates learning, reduces risk, and enables effective knowledge transfer between humans and AI agents across complex tasks.

Original authors: Cédric Colas, Tracey Mills, Ben Prystawski, Michael Henry Tessler, Noah Goodman, Jacob Andreas, Joshua Tenenbaum

Published 2026-02-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Cédric Colas, Tracey Mills, Ben Prystawski, Michael Henry Tessler, Noah Goodman, Jacob Andreas, Joshua Tenenbaum

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are dropped into a brand-new, mysterious video game. You don't know the rules, you don't know what the colorful blocks do, and you don't know how to win. You have to figure it out by trying things, making mistakes, and dying a lot. This is how most computer programs learn: through trial and error.

But humans are different. If you were dropped into that same game, you might ask a friend, "Hey, what's the deal with the red blocks?" If they say, "Don't touch them, they kill you," you instantly avoid a fatal mistake. If they say, "Push the green ones into the orange ones to win," you skip the guessing game and go straight to the solution.

This paper is about teaching computers to do exactly that: learn by combining their own experience with advice from others.

Here is a simple breakdown of how they did it, using some creative analogies.

1. The Problem: The "Silent" Learner vs. The "Talkative" Human

Current AI is like a silent explorer. It can eventually learn to play a complex game, but it has to walk through every single trap, fall off every cliff, and die thousands of times before it figures out the rules. It's slow and inefficient.

Humans, on the other hand, are social learners. We use language to pass down knowledge. We don't need to reinvent the wheel; we just need someone to tell us, "The wheel is round, and it rolls."

The researchers wanted to build an AI that acts like a human: one that can listen to advice, understand it, and use it to learn faster, while still learning from its own mistakes.

2. The Solution: The "Theory Detective"

The team created a computer model that acts like a detective trying to solve a mystery.

The Mystery: The rules of a video game (which objects kill you, which ones give points, how to win).
The Evidence:
1. Experience: The detective plays the game, sees a red block kill them, and notes, "Red is dangerous."
2. Language: The detective reads a note from a previous player that says, "Red blocks are deadly."

The detective doesn't just blindly believe the note. Instead, they run a mental simulation: "If the rule is 'Red kills,' does that explain why I died? Yes. Does it explain why the note-writer told me this? Yes."

They use a Language Model (like the AI behind this chat) as a "translator." The AI asks the Language Model: "If a person believed the rule was 'Red kills,' how likely are they to write the note 'Red is deadly'?" If the answer is "very likely," the AI updates its belief to trust that rule.

3. The Experiment: 10 Video Games as a Classroom

To test this, the researchers used 10 simple video games (like Pac-Man or Space Invaders but with random, unknown rules). They set up three groups:

The Solo Player: Learns only by playing and dying.
The Student: Learns by playing, but gets a note from a human who played before.
The Robot Student: Learns by playing, but gets a note from the AI model who played before.

The Results were surprising:

Speed: The players who got advice learned much faster. They died fewer times and solved the levels quicker.
Safety: Advice acted like a "warning sign." When told "Don't touch the yellow thing," players avoided it immediately, saving them from fatal errors.
The "Double-Edged Sword": Sometimes, the advice was wrong (e.g., "Green is safe" when it actually kills you). In those cases, the players followed the bad advice and got stuck. This mirrors real life: bad advice can be just as dangerous as no advice.

4. The "Generational" Effect: Passing the Torch

The researchers also ran a "telephone game" experiment.

Generation 1 plays the game, gets a little advice, and writes a note for the next person.
Generation 2 reads that note, plays, and writes a new note.
This continues for 10 generations.

The Result: The knowledge got better and better over time. Even though each player only had a few minutes to play, the "cultural memory" of the group accumulated. By the 10th generation, the players were experts, having inherited the refined wisdom of the previous nine.

5. Why This Matters: The Future of Human-AI Teams

This isn't just about video games. It's about how humans and AI can work together in the real world.

AI teaching Humans: The AI was able to write advice that helped humans learn faster.
Humans teaching AI: Humans wrote advice that helped the AI learn faster.

This creates a collaborative loop. Imagine a doctor (human) and a diagnostic AI. The AI learns from the doctor's notes ("This symptom usually means X"), and the doctor learns from the AI's analysis ("I noticed a pattern in 10,000 cases that suggests Y").

The Big Takeaway

This paper shows that language is a superpower for learning. By giving AI the ability to treat human words as "evidence" (just like physical data), we can create systems that learn faster, make fewer mistakes, and can pass their knowledge down to future generations. It's the difference between a lone explorer stumbling in the dark and a team of explorers sharing a map.

1. Problem Statement

Human learning is characterized by the ability to seamlessly integrate direct experience (trial-and-error) with linguistic guidance (advice from others). This dual-source learning enables rapid adaptation, safety, and the accumulation of knowledge across generations. However, current computational models fail to replicate this capability holistically:

Reinforcement Learning (RL): Agents (e.g., Deep Q-Networks) require millions of interactions to master complex tasks and lack the ability to learn from language without massive paired datasets.
Large Language Models (LLMs): While excellent at processing text, they struggle with interactive planning, embodied learning, and inferring causal world models from sparse data.
Existing Bayesian Models: Often operate in simple, non-interactive environments with predefined hypothesis spaces, failing to scale to complex, dynamic tasks where the rules must be inferred from scratch.

The core challenge is to develop a computational framework that treats linguistic guidance and direct experience as complementary evidence sources to infer executable, structured world models in complex, unknown environments.

2. Methodology

The authors propose a Bayesian framework where an agent maintains a distribution over structured, program-like world models (theories). The agent updates its beliefs by jointly inferring from gameplay experience ( $E$ ) and linguistic advice ( $L$ ).

A. Core Framework: Joint Probabilistic Inference

The agent models the environment as a distribution over theories $T$ (executable programs defining game rules). Belief updates follow Bayes' rule:
$P(T | E, L) \propto P(E | T) \times P(L | T) \times P(T)$

$P(T)$ : A simplicity-biased prior favoring theories with fewer rules.
$P(E | T)$ : Likelihood of experience. The agent simulates the candidate theory $T$ to see if it reproduces observed state transitions (e.g., object collisions, rewards).
$P(L | T)$ : Likelihood of language. This is the probability that a speaker holding belief $T$ would produce the observed message $L$ .

B. Key Technical Components

World Models as Executable Programs:
- Theories are represented in VGDL (Video Game Description Language), specifying object types, collision effects, rewards, and win/loss conditions.
- The agent maintains a population of $M=20$ candidate theories (particles).
- Inference Algorithm: A Particle Filter with Metropolis Rejuvenation. The agent resamples theories based on posterior probability and proposes local modifications (e.g., changing an interaction rule) guided by new data.
LLMs as Speaker Models:
- To compute $P(L | T)$ , the authors use a pre-trained LLM (LLaMA-3.1-70B) as a probabilistic speaker model.
- Given a theory description $T$ , the LLM is prompted to generate advice. The likelihood $P(L | T)$ is approximated by the LLM's probability of generating the actual received message $L$ .
- This allows the agent to interpret advice as evidence: if a message says "avoid yellow," theories where "yellow kills the avatar" receive a higher likelihood score.
Language-Guided Proposals:
- To accelerate inference, the LLM is used to bias the proposal distribution during the Metropolis step.
- When receiving advice (e.g., "yellow kills you"), the LLM is prompted to answer multiple-choice questions about specific rules (e.g., "Does yellow kill the avatar?"). The resulting probabilities guide the generation of new candidate theories, focusing the search on the most promising regions of the hypothesis space.
Goal-Directed Planning:
- The agent selects actions based on the Maximum A Posteriori (MAP) theory.
- It balances exploitation (pursuing goals based on current beliefs) and exploration (testing interactions where theories disagree) using a genetic algorithm to optimize action sequences.
Bidirectional Communication:
- The same LLM architecture is used to generate advice. The agent translates its MAP theory into natural language to teach future players, closing the loop for social learning.

3. Experimental Setup

Environment: 10 distinct video games defined in VGDL, ranging from simple navigation to complex relational reasoning (e.g., object transformations, resource management).
Conditions:
1. Experience Only: Agents learn via trial-and-error.
2. Experience + Human Advice: Agents receive advice written by previous human players.
3. Experience + Model Advice: Agents receive advice generated by the computational model.
Baselines:
- Oracle: Uses ground-truth rules.
- Deep RL: Double DQN (pure trial-and-error).
- Pure LLM: LLaMA-3.1-70B using ReAct/Chain-of-Thought without structured world modeling.
Metrics: Number of lives required to solve levels, normalized Area Under the Curve (nAUC) for learning efficiency, and success in iterated learning (generational transfer).

4. Key Results

A. Learning Efficiency

Structured Models vs. Baselines: The proposed model and humans achieved high sample efficiency, solving most games within 10 lives. In contrast, Deep RL failed to solve any game within 10 lives, and Pure LLMs solved fewer than 1 level per game on average. This highlights the necessity of structured, program-like world models for efficient exploration.
Impact of Language: Both humans and the model learned significantly faster with linguistic guidance.
- Humans reduced median attempts by 1.75 lives.
- Models reduced median attempts by 1.25 lives.
- Guidance was most effective in games with opaque mechanics (e.g., relational) or multiple hazards, where advice could directly communicate critical interactions.

B. Knowledge Transfer and Asymmetry

Human-to-Model: Models learned effectively from human advice, demonstrating that the framework can interpret natural, sometimes imperfect, human language.
Model-to-Human: Models generated pedagogical advice that significantly improved human learning ( $\Delta nAUC = 0.15$ ).
Asymmetry: Models learned better from model-generated advice than human-generated advice, while humans showed only a modest improvement from model advice. This suggests models struggle to interpret human metacognitive strategies (e.g., analogies, emotional context) that humans rely on.

C. Generational Learning

In iterated learning experiments (10 generations, 2 lives per agent), performance reliably increased across generations.
Knowledge accumulated incrementally, allowing later agents to master games that earlier agents could not solve alone.
Failure Modes: In complex games requiring precise coordination (relational) or fast-paced action (plaqueAttack), knowledge transmission was brittle, leading to occasional regressions. This mirrors the "double-edged sword" of pedagogy observed in human development.

5. Significance and Contributions

Unified Framework for Social Learning: The paper provides the first computational account that successfully integrates linguistic guidance and direct experience within a unified Bayesian inference loop for complex, dynamic tasks.
LLMs as Cognitive Tools: It demonstrates a novel use of LLMs not just as chatbots, but as probabilistic speaker models and proposal generators within a Bayesian cognitive architecture. This bridges the gap between the flexibility of LLMs and the rigor of structured probabilistic reasoning.
Bidirectional Human-AI Collaboration: The system enables bidirectional knowledge transfer. AI agents can learn from humans and, crucially, generate advice that helps humans learn, laying the groundwork for collaborative human-AI learning systems.
Insights into Human Cognition: The results validate the hypothesis that humans likely use structured, program-like representations to integrate language and experience. The model's success in mimicking human learning curves and error patterns supports this cognitive theory.
Scalability: By using LLMs to guide the search in the hypothesis space, the approach overcomes the intractability of exact Bayesian inference in vast rule spaces, offering a path toward scalable, sample-efficient AI agents in open-ended environments.

Conclusion

This work establishes that structured, language-compatible representations are essential for efficient social learning. By treating language as evidence for inferring executable world models, the proposed framework enables agents to learn rapidly, avoid dangerous mistakes, and accumulate knowledge across generations, offering a promising direction for developing AI systems that can truly collaborate with humans.