COMIC: Agentic Sketch Comedy Generation

The paper presents COMIC, a fully automated AI system that generates high-quality, diverse comedic sketch videos by employing a multi-agent framework with specialized roles and LLM-based critics trained on YouTube data to iteratively refine content toward professional standards.

The Gist

Imagine you want to make a funny sketch comedy show, like Saturday Night Live, but you don't have a team of writers, directors, or actors. You just have a computer.

Usually, if you ask a computer to write a joke, it gives you a dad joke that makes you groan. It's not actually funny. But a new system called COMIC (Content Optimization via Multi-agent Iterative Competition) has figured out how to make a computer laugh, and more importantly, how to make you laugh.

Here is how COMIC works, explained through a simple analogy: The "Comedy Factory" with a Twist.

1. The Problem: The "One-Shot" Mistake

Most AI systems work like a student taking a test. They get the prompt, think for a second, and write one answer. If that answer is bad, they stop.

• The Issue: Humor is tricky. What's funny to one person is boring to another. A single "best guess" usually misses the mark.

2. The Solution: The "Comedy Factory" with Many Teams

Instead of one writer, COMIC sets up a factory with many different teams of AI agents, each playing a specific role from a real TV studio:

• The Writers: They brainstorm ideas and write scripts.
• The Directors: They plan the scenes, camera angles, and character expressions.
• The Critics: This is the secret sauce. They are the "tough bosses" who decide what's actually funny.

3. The Secret Sauce: The "Island" Tournament

Here is where COMIC gets clever. It doesn't just ask one critic to grade the work. It creates Islands.

Imagine a reality TV show like Survivor, but for jokes.

• The Setup: The AI splits its writers into separate "Islands."
• The Judges: Each Island has its own unique panel of Critics. One panel might love "silly slapstick" (like slipping on a banana peel), while another loves "dry, sarcastic wit" (like a deadpan stare).
• The Tournament: The writers on each island pitch their jokes. The Critics vote. The "losers" (the unfunny scripts) get sent back to the writers with specific feedback: "This joke is too long," or "The punchline needs more surprise."
• The Evolution: The writers rewrite the jokes based on the feedback. They do this over and over again. The "losers" get better until they are good enough to beat the "winners" from the previous round.

The Result: The jokes don't just get "okay"; they evolve into something genuinely funny because they have been tested against many different styles of humor.

4. Making the Video: The "Director's Cut"

Once the script is funny, the system has to turn it into a video. This is hard because AI video generators often make weird glitches (like a character's face melting or a background changing).

COMIC uses the same "Tournament" idea here:

• The Director Agent breaks the script into shots.
• It generates a video clip for a scene.
• Video Critics look at the clip. Did the character blink naturally? Is the background consistent?
• If the clip is weird, the system generates a new version, critiques it, and tries again.
• Finally, it holds a "Final Cut" tournament where it picks the absolute best version of every scene to stitch together into a 1–2 minute movie.

5. Why is this a Big Deal?

• It learns from real people: The "Critics" aren't just random code. They were trained by analyzing thousands of real comedy sketches on YouTube to see what actually makes humans click "Watch More" and laugh.
• It scales: If you give the computer more time and power (more "Islands" and more "Tournaments"), the jokes get funnier. It's like hiring more writers and running more rehearsals.
• It beats the competition: When tested against other top AI video tools, COMIC produced videos that humans rated as significantly funnier and more engaging.

The Bottom Line

Think of COMIC not as a robot that "knows" how to be funny, but as a robot that knows how to practice.

It generates thousands of bad jokes, throws them in a ring, lets the "tough critics" beat them up, and forces the writers to try again. After enough rounds of this "creative sparring," the survivors are the truly funny sketches. It turns the messy, subjective art of comedy into a rigorous, competitive game that a computer can win.

Technical Summary

Here is a detailed technical summary of the paper "COMIC: Agentic Sketch Comedy Generation".

1. Problem Statement

The paper addresses the challenge of fully automated sketch comedy video generation. Unlike domains with objective ground truths (e.g., coding or mathematics), humor is subjective, context-dependent, and multidimensional. The system must solve three coupled sub-problems simultaneously:

1. Conceptualization: Generating a compelling comedic premise.
2. Scriptwriting: Producing a funny, coherent script with character interactions and a payoff.
3. Visual Realization: Generating a high-quality, consistent video (1–2 minutes long) that maintains character identity, scene continuity, and narrative flow across multiple shots.

Existing generative models typically produce short clips (10s) lacking narrative coherence, while current agentic video systems often rely on fixed, single-pass pipelines that fail to explore the vast creative space required for humor.

2. Methodology: The COMIC Framework

The authors propose COMIC (Content Optimization via Multi-agent Iterative Competition), a fully automated system modeled after human production studios. It utilizes a population of AI agents with distinct roles (Writer, Critic, Director, Editor) organized into iterative loops.

A. Multi-Agent Architecture

The system is structured into two main loops:

1. Script Writing Loop:

   • Island-Based Evolution: The global script population is partitioned into $K$ isolated "islands." Each island is governed by a specialized critic committee representing different comedic philosophies (e.g., slapstick, dry wit, surrealism).
   • Iterative Competition: Scripts undergo round-robin pairwise tournaments. A critic compares two scripts, selects a winner, and provides natural language feedback. The losing script is refined using this feedback (acting as a semantic crossover/mutation).
   • Fitness Landscape: A script's "fitness" is not absolute but relative to the current population. As the population improves, the baseline for winning rises, forcing continuous adaptation.

2. Video Rendering Loop:

   • Storyboarding: A Scene Director agent breaks the final script into shot-by-shot directions (characters, backgrounds, camera angles) using a memory bank for continuity.
   • Script-Conditioned Critics: For each script, a meta-critic generates a set of rendering critics with diverse personas to evaluate visual realization.
   • Iterative Refinement: Shots are generated and refined iteratively. Critics evaluate the shot against the script and previous shots, proposing refinements.
   • Tournament Selection: Instead of accepting the final iteration, the system runs a single-elimination tournament across the history of generated shots and full video variations to select the best realization.

B. Alignment with Real Viewers (The "Critic" Innovation)

A key contribution is the method for aligning AI critics with human humor preferences without fine-tuning models:

• Data Source: The system analyzes 4,940 videos from five major YouTube sketch comedy channels (e.g., SNL, Key & Peele).
• Engagement Scoring: View counts are normalized by video age using a logistic growth model to estimate a video's "carrying capacity" (potential popularity).
• Generate-and-Select Strategy: A meta-critic generates a diverse pool of candidate critics (defined by system prompts/personas). These critics are evaluated on their ability to predict the engagement scores of the YouTube data.
• Task-Specific Selection: The system selects the best critics for specific tasks (e.g., distinguishing "Top vs. Bottom" quality vs. "Top vs. Middle" subtle nuances) and channels, creating a specialized pool that captures diverse comedic traditions.

3. Key Contributions

1. First Fully Automated Comedy Generator: COMIC is the first system to automate the end-to-end generation of sketch comedy videos (script + video + audio) from minimal inputs (character images, voices, text).
2. Multi-Agent Iterative Competition: It moves beyond single-pass generation by implementing a "Red Queen" evolutionary process where scripts and videos continuously improve through competition against co-evolving peers.
3. Human-Aligned Critics via Zero-Shot Selection: Instead of training a reward model, the authors use a "generate-and-select" approach to create diverse, channel-specific critics that align with real-world viewer engagement patterns.
4. Test-Time Scaling: The framework allows for scaling inference compute (more islands, more critics, more iterations) to improve quality without retraining the underlying foundation models.

4. Results

The paper evaluates COMIC against state-of-the-art baselines, including frontier video models (Sora 2, Veo 3.1) and agentic video systems (VGoT, MovieAgent).

• Human Evaluation: In blind studies, COMIC significantly outperformed all baselines in Funniness, Script Quality, and Narrative Coherence.
  • On a 7-point Likert scale, COMIC scored 3.45 for funniness, compared to 2.73 for Sora 2 and 2.32 for Veo 3.1.
  • Crucially, COMIC approached the quality of human-produced sketches (scoring between "Slightly Less Funny" and "Comparable"), a level no other automated system reached.
• Automated Evaluation: Using the aligned critic framework, COMIC achieved a Win Rate of 0.44 against median human sketches, far surpassing baselines (which scored near 0.0).
• Diversity: COMIC demonstrated superior Inter-Diversity (variety across different generated scripts) and Intra-Diversity (specialization within specific styles), proving the multi-island topology successfully explores the creative space.
• Ablation Studies: Removing the critic refinement loop or using a single-island topology resulted in significant drops in win rate and diversity, confirming the necessity of the competitive, multi-agent architecture.

5. Significance

• Redefining Creative AI: The paper demonstrates that for open-ended creative tasks where "correctness" is undefined, relative competition and diverse evaluation are more effective than fixed reward functions.
• Bridging Short-Form and Long-Form: COMIC successfully extends short-form generative capabilities into coherent, multi-minute narratives with consistent characters and plotlines.
• Efficiency: The system achieves professional-level quality with a low API budget (~$5 per video) and no parameter updates, suggesting a scalable path for future creative AI applications.
• Future Directions: The work opens new avenues for applying evolutionary multi-agent systems to other subjective domains (e.g., art, music, design) and highlights the potential of "test-time compute" as a lever for quality improvement in generative AI.