HAMLET: A Hierarchical and Adaptive Multi-Agent Framework for Live Embodied Theatrics

This paper presents HAMLET, a hierarchical adaptive multi-agent framework that leverages large language models to autonomously generate and perform immersive, interactive theatrical experiences by combining narrative planning, persona-driven improvisation, and real-time embodied interactions with physical props, all evaluated by a specialized critic model.

The Gist

Imagine you want to put on a live play, but instead of hiring human actors, you hire a team of super-smart AI computers. The goal is to create a show that feels real, where the actors can improvise, interact with props (like opening a letter or picking up a sword), and react to the audience in real-time.

The problem with previous attempts is that these AI actors were either too robotic (waiting for you to tell them exactly what to say), too confused (forgetting their character), or they couldn't actually "touch" anything in the digital world.

Enter HAMLET. Think of HAMLET not just as a scriptwriter, but as a high-tech theater production company run by a team of specialized AI agents. Here is how it works, broken down into simple parts:

1. The Two-Stage Process: Planning vs. Acting

HAMLET splits the work into two distinct phases, much like a real theater production.

• Phase 1: The "Blueprint" (Offline Planning)
  Imagine you give the team just a simple idea, like "Hamlet wants to test his uncle's guilt using a play."

  • The Designers: A team of AI agents (The Actor Designer, Plot Designer, and Reviewer) brainstorm. They create character profiles (who is Hamlet? What is his secret goal?), map out the story beats, and list the props needed (a dagger, a letter, a throne).
  • The Director: A "Director" agent takes all these scattered ideas and organizes them into a Narrative Blueprint. This isn't a rigid script; it's more like a map with checkpoints. It says, "The story must end with the uncle fleeing in fear," but it doesn't dictate exactly how the actors get there. This leaves room for improvisation.

• Phase 2: The "Live Show" (Online Performance)
  Now the show begins. The actors (AI agents) step onto the digital stage.

  • The Improv: Unlike old AI that just waits for your input, these actors have initiative. They can decide to whisper a secret to a friend, pick up a weapon, or storm out of the room on their own, as long as it fits the plot.
  • The "PAD" Brain: This is the secret sauce. Every actor has a special module called PAD (Perceive and Decide). Think of it as the actor's "gut feeling" vs. "logical brain."
    • System 1 (Fast): "He insulted me! I'm angry!" (Immediate reaction).
    • System 2 (Slow): "If I attack him now, I'll get caught. I should pretend to be nice first." (Strategic thinking).
    • This allows the AI to make complex, human-like decisions rather than just spitting out the next line of dialogue.

2. The "Stage Manager" and the "Reality Check"

One of the biggest challenges in AI theater is keeping the physics and logic straight. If an actor tries to fly or use a gun in a medieval castle, the show breaks.

• The Narrator (The Referee): HAMLET has a special agent called the Narrator. Its job is to watch every action. If an actor tries to "pick up a sword," the Narrator checks: Is there a sword? Is the actor strong enough? Is it physically possible?
  • If yes: The Narrator updates the world. "Hamlet draws his sword."
  • If no: The Narrator says, "Action failed," and explains why (e.g., "There is no sword on the table").
  • This keeps the world feeling real and embodied, not just a chat room.

3. The "Critic" (HAMLETJudge)

How do you know if the AI play is good? You can't just ask "Did the grammar look okay?" because a play is about emotion, plot twists, and character depth.

• The Judge: The team built a special AI critic called HAMLETJudge. It watches the whole play and compares it to other versions. It doesn't just look at one sentence; it looks at the whole story arc.
  • Did the characters stay in character?
  • Was the story exciting?
  • Did the audience feel immersed?
  • It gives the play a score, helping the system learn how to get better.

4. Why This Matters

Previous AI drama systems were like puppets: you pulled the strings (typed instructions), and they moved.
HAMLET is like a troupe of method actors:

• They have their own memories and goals.
• They can improvise if the story goes off the rails.
• They can interact with the environment (opening doors, hiding items).
• They can even handle a human player joining the cast, reacting to them naturally.

The Bottom Line

HAMLET is a framework that turns a simple topic into a living, breathing, interactive play. It uses a team of AI specialists to plan the story, a "gut-check" brain for the actors to make smart decisions, a referee to keep the physics real, and a critic to ensure the show is entertaining. It's a major step toward creating AI that doesn't just talk, but performs.

Technical Summary

Here is a detailed technical summary of the paper "HAMLET: A Hierarchical and Adaptive Multi-Agent Framework for Live Embodied Theatrics".

1. Problem Statement

The paper addresses critical limitations in current Large Language Model (LLM)-based interactive drama and role-playing systems:

• Lack of Initiative: Existing agents often react passively to user inputs rather than proactively driving the narrative, leading to stagnant or repetitive interactions.
• Absence of Embodiment: Current systems focus primarily on dialogue, failing to simulate physical interactions with the environment (e.g., picking up objects, changing scene states), which is essential for immersive "live" theatre.
• High Input Dependency: Many methods require detailed user-provided story outlines or scripts, limiting flexibility and increasing design costs.
• Evaluation Gaps: There is no standardized method to evaluate the quality of live embodied performance, as existing benchmarks focus on static text coherence or single-turn role fidelity rather than holistic dramatic enactment.

2. Methodology: The HAMLET Framework

HAMLET is a hierarchical, multi-agent framework designed to generate and perform interactive drama in two decoupled stages: Offline Planning and Online Performance.

A. Offline Planning (Narrative Blueprint Generation)

This stage transforms a simple topic (or a full literary work) into a structured narrative blueprint without requiring detailed user outlines.

• Multi-Agent Workflow:
  • Actor Designer: Generates character profiles (personas, goals, relationships) using external knowledge retrieval.
  • Plot Designer: Creates a preliminary narrative draft based on the topic and characters.
  • Reviewer: Validates character rationality and plot consistency.
  • Director: Integrates approved profiles and plots into a hierarchical Narrative Blueprint. This includes defining acts, scenes, interactive props, and "narrative points" (milestones) with specific flags to mark completion.
• Key Feature: The blueprint provides a structural guide (Acts $\to$ Scenes $\to$ Points) while allowing for improvisation within the constraints.

B. Online Performance (Live Execution)

This stage executes the blueprint in real-time, supporting autonomous AI actors and human players.

• Hierarchical Control System:
  • Planner: Pre-designs multiple potential trajectories (sequences of "beats") between narrative points to prevent plot stagnation.
  • Transfer: Monitors "flags" (conditions) to determine when a narrative point is complete and transitions the scene.
  • Advancer: Intervenes if the plot stalls (e.g., actors chit-chatting without action), issuing directives to force narrative progression.
• Embodied Interaction:
  • Narrator Agent: Acts as an adjudicator for physical actions. It validates whether an actor's attempt to interact with a prop (e.g., "open the letter") is feasible based on the current environment state and physical rules. It broadcasts the result (success/failure) to all participants.
• Perceive and Decide (PAD) Module:
  • Core Innovation: A specialized module for each actor agent that simulates human cognitive dual-process theory (System I: fast/intuitive; System II: slow/deliberate).
  • Function: Before generating a response, PAD analyzes the Internal State (Persona, Goals, Memory) and External Stimulus (Environment, Dialogue History, Props).
  • Output: It selects a response strategy via tool calling: Fast (impulsive), Slow (deliberate), Silence (strategic), or Action (physical interaction). This ensures decisions are context-aware and human-like.

C. Evaluation System

• HAMLETJudge: A specialized critic model (fine-tuned 8B model) trained to evaluate live drama performance.
• Dimensions: Evaluation is holistic, covering three dimensions:
  1. Character Performance (CP): Believability and Agency.
  2. Narrative Quality (NQ): Coherence, Resonance, and Integrity.
  3. Interaction Experience (IE): Responsiveness, Immersion, and Fluency.
• Method: Uses pairwise comparisons against baselines with a 5-point Likert scale.

3. Key Contributions

1. HAMLET Framework: A novel architecture separating offline structural planning from online improvisational performance, enabling autonomous drama generation from simple topics.
2. PAD Module: A cognitive-inspired decision-making module that balances intuitive and analytical reasoning, allowing agents to choose between speech, silence, and physical actions dynamically.
3. Embodied Interaction Mechanism: A robust system where a Narrator agent adjudicates physical actions against environmental constraints, bridging the gap between dialogue and physical theatre.
4. HAMLETJudge & Benchmark: The introduction of a comprehensive evaluation framework and a dedicated critic model that outperforms general LLMs in assessing dramatic performance.
5. Extensive Experiments: Validation across 100 diverse cases (classical literature and custom topics) in both English and Chinese.

4. Experimental Results

• Performance: HAMLET (specifically with the PAD module) achieved state-of-the-art results on the HAMLET Leaderboard. Reasoning-enhanced models (e.g., Claude-4-sonnet-Thinking, Qwen3-32B-Thinking) showed superior performance in subtext and strategic planning compared to non-reasoning models.
• PAD Efficiency: The PAD module (an 8B model) achieved the highest overall scores while maintaining negligible latency (0.36s average), whereas reasoning-heavy models incurred significant latency penalties (up to 49.5s for DeepSeek-R1).
• Ablation Studies:
  • Removing the PAD module significantly reduced performance, confirming its role in natural interaction.
  • Removing the Reviewer degraded both offline blueprint quality and online narrative coherence.
  • Removing the Advancer caused the task completion rate to drop from 100% to 68.1% due to narrative deadlocks.
• Reliability: HAMLETJudge showed a high Pearson correlation (0.791) with human evaluators, outperforming strong baselines like GPT-4.1 and Claude-4.

5. Significance

HAMLET represents a paradigm shift from passive, turn-based role-playing to active, embodied, and immersive theatrical experiences. By integrating hierarchical planning, cognitive decision-making (PAD), and physical environment simulation, it solves the "initiative" and "embodiment" problems in AI drama. The framework demonstrates that autonomous agents can collaboratively create coherent, emotionally resonant, and physically interactive stories, paving the way for future applications in AI-driven entertainment, interactive education, and immersive storytelling. The introduction of HAMLETJudge also provides a much-needed standard for evaluating the complex, multi-dimensional nature of live AI performance.