💬 NLP

Inner Speech as Behavior Guides: Steerable Imitation of Diverse Behaviors for Human-AI coordination

The paper proposes MIMIC, a framework that leverages vision-language models to generate "inner speech" as a linguistic representation of behavioral intent, enabling a diffusion-based policy to achieve diverse, high-fidelity human-like behaviors with fine-grained steerability at inference time without requiring additional training data.

Original authors: Rakshit Trivedi, Kartik Sharma, David C Parkes

Published 2026-02-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Rakshit Trivedi, Kartik Sharma, David C Parkes

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Giving AI a "Inner Monologue"

Imagine you are learning to cook a complex meal by watching a master chef.

The Old Way (Standard AI): You just copy the chef's hands. If the chef moves their hand left, you move yours left. If the chef moves right, you move right. You are a "mindless mimic." If the kitchen layout changes slightly, you get confused because you only memorized the movements, not the reasons.
The New Way (MIMIC): You watch the chef, but you also imagine what they are thinking. You hear their inner voice saying, "I need to grab the knife from the right because the left handle is slippery," or "I'm going to chop slowly because the onion is small."

This paper introduces a system called MIMIC (Modeling Inner Motivations for Imitation and Control). It teaches AI agents to generate their own "inner speech" (a stream of internal thoughts) before they act. This inner speech acts as a bridge between what they see and what they do.

The Problem: Robots are Too "Robot-Like"

Current AI robots are great at copying specific actions, but they struggle with two things:

Diversity: Humans are messy. Sometimes we walk around a table; sometimes we walk under it. Standard AI usually picks one "average" path and gets stuck.
Steering: If you want a robot to act specifically (e.g., "be very careful" or "be super fast"), you usually have to retrain the whole robot from scratch.

The authors realized that humans don't just react to the world; we have an internal conversation with ourselves that guides our choices.

The Solution: The "Inner Monologue" Framework

MIMIC treats language as the secret sauce. It breaks the robot's brain into three parts, working like a theater production:

1. The Director (The Vision-Language Model)

Before the robot learns, a "Director" (a powerful AI like GPT-4) watches videos of humans doing tasks. The Director writes a script describing why the human is doing what they are doing.

Example: Instead of just seeing "Robot moves arm," the Director writes: "The robot is approaching the box from the side to avoid hitting the wall."
Analogy: This is like a screenwriter writing the internal thoughts of a character for a movie.

2. The Actor's Inner Voice (The CVAE Generator)

The robot learns to turn those scripts into its own "inner voice." It uses a special neural network (a CVAE) to compress those long, detailed scripts into short, punchy mental notes.

Analogy: Think of this as an actor memorizing a script. They don't recite the whole play in their head every second; they condense it into a feeling or a quick thought like "Be careful" or "Go fast."
This inner voice is stochastic (random). Just like humans, the robot might think "Go left" today and "Go right" tomorrow, even in the same situation. This creates natural diversity.

3. The Performer (The Diffusion Policy)

Finally, the robot acts. But it doesn't just look at the room; it looks at the room AND listens to its inner voice.

Analogy: Imagine a dancer. They see the music (the environment), but they also have a rhythm in their head (the inner speech). The dance moves are a mix of the music and that internal rhythm.

Why This is a Game-Changer

1. It's "Steerable" (You can talk to the robot)

Because the robot's behavior is guided by language, you can change its personality by changing its "inner speech" prompt.

Scenario: You want the robot to be a clumsy, cautious helper.
Action: You tell the system, "Inner thought: I am nervous and moving slowly."
Result: The robot starts moving carefully without you needing to retrain it.
Analogy: It's like putting a different pair of glasses on the robot. If you tell it "I am a cautious cat," it moves like a cat. If you say "I am a speedy dog," it moves like a dog.

2. It Handles "Human Messiness"

Humans are inconsistent. Sometimes we are tired; sometimes we are energetic. Standard AI tries to find the "perfect" average move, which often looks robotic and stiff.

MIMIC's Approach: By generating random inner thoughts, the robot naturally produces a variety of moves, just like a human. It captures the "flavor" of human behavior, not just the mechanics.

3. It Works Better in Teams

The paper tested this in a game called Overcooked (a chaotic cooking game).

When MIMIC robots played with human proxies, they got higher scores than standard robots.
Why? Because the MIMIC robot could "think" about what the human was doing. It wasn't just reacting; it was simulating a human-like thought process, making it a better partner.

A Simple Summary of the Process

Watch: The AI watches humans doing tasks.
Translate: A smart AI (the Director) writes down what the humans are thinking based on what they are doing.
Internalize: The robot learns to turn those thoughts into a compressed "inner voice."
Act: When the robot plays, it generates a new inner thought, then acts based on that thought + what it sees.
Control: Humans can whisper new thoughts to the robot to change how it behaves on the fly.

The Takeaway

This paper suggests that to build truly human-like AI, we shouldn't just teach them what to do. We should teach them how to think about what they are doing. By giving AI an "inner speech," we make them more flexible, more diverse, and much easier to control, turning them from rigid machines into adaptable partners.

1. Problem Statement

Effective Human-AI coordination requires artificial agents that can not only mimic human behaviors but also adapt to changing contexts and exhibit the inherent diversity of human decision-making. Current Imitation Learning (IL) and Behavior Cloning (BC) approaches face several critical limitations:

Direct State-to-Action Mapping: Traditional BC models learn a direct mapping $\pi(a|s)$ , treating the human mind as a black box. This overlooks the cognitive processes (motivations, deliberation) that mediate between perception and action.
Lack of Diversity: Standard models often struggle to capture multimodal distributions of human behavior, leading to mode collapse or rigid, non-diverse trajectories.
Inflexibility: Most methods lack mechanisms for steerable behavior generation at inference time. They cannot easily adjust an agent's behavior based on designer-specified intent without retraining.
Non-Markovian Nature: Human behavior often depends on historical context and internal deliberation, which simple state-action mappings fail to capture.

2. Methodology: The MIMIC Framework

The authors propose MIMIC (Modeling Inner Motivations for Imitation and Control), a framework inspired by cognitive science theories (specifically Vygotsky's theory of inner speech). Instead of mapping state directly to action, MIMIC introduces an intermediate inner speech layer ( $m$ ) that acts as a mediational mechanism.

The core theoretical proposition is that human behavior is a marginalization over inner speech:
$P(a|s) = \int P(a|s, m) P(m|s) dm$

The framework consists of three key components:

A. Theoretical Formulation

MIMIC formalizes inner speech with three structural properties derived from cognitive science:

Predicativity: Inner speech focuses on relationships and actions (e.g., "move left to coordinate") rather than static entities.
Semantic Condensation: Complex strategies are compressed into compact linguistic representations.
Temporal Regulatory Dynamics: Inner speech operates over extended timescales, conditioned on behavioral history rather than instantaneous states.

B. Architectural Components

Inner Speech-Conditioned Behavior Cloner (Policy):
- Architecture: A Diffusion-based Policy with Transformer architecture (DDPM-T).
- Function: It learns the conditional probability $P(a|s, m)$ . The policy takes the current state $s$ and the inner speech embedding $m$ as input to denoise and predict actions.
- Training: Uses classifier-free guidance, allowing the model to operate with or without inner speech conditioning.
Behavior-Guided Inner Speech Generator:
- Architecture: A Conditional Variational Autoencoder (CVAE).
- Function: It learns to generate the inner speech embedding $m$ from the history of observations $H_t$ (past states and actions), modeling $P(m|s)$ .
- Mechanism: It captures the stochastic nature of inner speech, allowing for diverse behavioral modes even from the same state.
Vision-Language Model (VLM) Scaffolding:
- Role: Since human demonstrations do not include explicit "thoughts," MIMIC uses a pre-trained VLM (e.g., GPT-4o) to generate external linguistic descriptions of the demonstrated behaviors.
- Process: The VLM analyzes GIFs of demonstrations and generates descriptive text. These texts are encoded (via CLIP) to serve as the "teacher" targets for the CVAE.
- Internalization: The CVAE learns to compress these external descriptions into a latent inner speech representation, effectively internalizing the linguistic structure without requiring human annotation of thoughts during data collection.

C. Inference and Steering

Periodic Generation: The agent generates new inner speech periodically (every $W$ steps) based on its recent behavioral history, rather than at every time step. This mimics strategic deliberation.
Steerable Control: Designers can intervene by providing a specific text prompt (e.g., "approach from the right") as the initial inner speech. The agent then generates its own subsequent inner speech to maintain consistency with this intent, enabling steerable imitation without retraining.

3. Key Contributions

Cognitive-Inspired Architecture: MIMIC is the first IL framework to operationalize the theory of "inner speech" as a latent, stochastic mediator between perception and action, bridging cognitive science and deep reinforcement learning.
Steerable Imitation: It enables fine-grained control over agent behavior at inference time via natural language prompts, a capability absent in standard BC or goal-conditioned methods.
Scalable Training via VLMs: By using VLMs to generate training targets from visual data, MIMIC avoids the expensive requirement of human-annotated "thought" data for every step of a demonstration.
Enhanced Diversity and Fidelity: The framework successfully models the non-Markovian, diverse nature of human behavior, outperforming state-of-the-art baselines in both success rates and behavioral entropy.

4. Experimental Results

The authors evaluated MIMIC on two primary benchmarks:

A. Robotic Manipulation (D3IL Benchmark)

Tasks: Aligning, Sorting, and Stacking blocks.
Metrics: Success rate, behavioral entropy, and Wasserstein distance (fidelity to human distributions).
Results:
- MIMIC significantly outperformed the state-of-the-art DDPM-T baseline.
- Success Rate: Improved by ~15-20% in complex tasks (e.g., Aligning task success rose from 0.66 to 0.80).
- Diversity: Achieved higher behavioral entropy, indicating a broader coverage of human-like strategies.
- Fidelity: Showed lower Wasserstein distances between generated and human state/time distributions.
- Ablation: Removing inner speech or using random vectors significantly degraded performance, confirming the necessity of the linguistic mediator.

B. Human-AI Coordination (Overcooked)

Tasks: 2-player cooking games (Cramped Room, Coordination Ring, Asymmetric Advantages).
Setup: MIMIC agents played alongside human proxy models.
Results:
- MIMIC agents achieved significantly higher collective rewards (up to 30% improvement) compared to standard BC agents.
- This demonstrates that modeling inner speech allows AI to better anticipate and adapt to human partners, acting as more effective "human surrogates."

C. Steerability

Experiments showed that providing specific text descriptions (e.g., "rotate counter-clockwise") as initial inner speech successfully guided the agent to execute specific behavioral modes that matched the description, validating the framework's controllability.

5. Significance and Impact

AI Safety and Alignment: By creating agents that can mimic the diversity and deliberation of human behavior, MIMIC enables safer pre-deployment testing of AI systems in complex human-AI collaboration scenarios.
Beyond Direct Mapping: The paper challenges the dominant paradigm of direct state-to-action mapping, proposing that internal cognitive representations (inner speech) are essential for robust, adaptable, and controllable AI.
Generalizability: The approach is model-agnostic regarding the policy (can work with Transformers, Flow models, etc.) and relies on visual data, making it applicable to a wide range of robotics and multi-agent domains without needing expensive thought-annotation datasets.
Future Directions: The framework lays the groundwork for systems that can reason about collective dynamics in multi-agent settings and offers a new path for "cognitively grounded" AI that aligns with human values and decision-making patterns.