💬 NLP

Re-defining Humor Data Objects for AI Humor Research

This paper redefines humor in AI research as a context-dependent social interaction by developing an improved prompting strategy for LLMs to generate high-quality humor explanations, thereby creating a scalable dataset for data synthesis and advancing the understanding of humor as a social behavior.

Original authors: Anna Arnett, Bang Nguyen, Meng Jiang

Published 2026-05-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Anna Arnett, Bang Nguyen, Meng Jiang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to understand a joke. For a long time, researchers treated humor like a simple light switch: either the joke was "on" (funny) or "off" (not funny). They just looked at the text and asked, "Is this a joke? Yes or No?"

But the authors of this paper argue that humor isn't a light switch; it's more like a live theater performance. You can't understand a play just by reading the script; you need to know who the actors are, what the audience is doing, and what happened right before the punchline.

Here is a simple breakdown of what they did, using some everyday analogies:

1. The Problem: The "Blind" Script

The researchers started with a pile of transcripts from funny moments (like TED Talks or sitcoms). They noticed a big problem: the text was often missing the most important parts.

The Analogy: Imagine reading a transcript of a magic show that just says, "The magician waves his hand. The audience laughs."
- If you are the robot, you might think, "Okay, waving hands is funny."
- But in reality, the audience laughed because the magician pulled a rabbit out of a hat, or because he made a silly face. The transcript didn't tell you that.
- The old way of doing things forced the AI to guess (or "hallucinate") what happened, often getting it wrong.

2. The Solution: A New "Joke Report Card"

Instead of just labeling a joke as "funny," the team created a new way to describe humor, which they call a Humor Reasoning Data Object. Think of this as a detailed Joke Report Card that has five specific sections:

The Setup (Context): What was happening before the joke?
The Punchline (Attempt): What did the person actually say?
The Reaction: Did the audience laugh (1) or stay silent (0)?
The Explanation: Why did they laugh? (This is the most important new part).
The "Save" (Recovery): If the joke bombed, how did the person fix it? (e.g., "Oops, my bad," or changing the subject).

3. The Experiment: Teaching the AI to "Think"

The team used a Large Language Model (an advanced AI) to fill out these Report Cards. They tried two different ways of asking the AI to do this:

Prompt 1 (The Optimist): They asked the AI, "Here is the text. Explain why this is funny."
- The Result: The AI tried its best, but when the text was missing clues (like a visual cue or a cultural reference), the AI made up stories to fill the gaps. It was like a student guessing the answer on a test because they forgot to read the question carefully.
- Success Rate: Only about 55% of the explanations were good.
Prompt 2 (The Skeptic): They changed the instructions. They told the AI: "Before you explain, check if the text actually has enough information. If the text is missing a picture, a gesture, or a key detail, admit it and say the explanation is impossible."
- The Result: The AI became much smarter. It stopped making things up. If the transcript was missing a visual cue, it flagged the data as "broken" instead of inventing a fake reason.
- Success Rate: This jumped to 87% correct explanations.

4. The Big Win: Scaling Up

Once they fixed the instructions (the "Prompt"), they used the AI to generate 307 new, high-quality joke explanations.

Why this matters: They didn't just fix the AI; they created a new dataset. This is like building a massive library of "Joke Report Cards" that includes not just the joke, but the reasoning behind it and a warning label if the text was too messy to understand.
The Takeaway: The paper proves that you don't need a smarter robot to understand humor better; you just need to ask the robot the right questions. By forcing the AI to check its own work and admit when information is missing, the quality of the humor explanations skyrocketed.

Summary

The paper is essentially a guide on how to stop AI from "making things up" when explaining jokes. By treating humor as a complex social interaction (with context, reactions, and explanations) rather than a simple "yes/no" label, and by giving the AI strict rules to follow, they created a much more reliable way to study how machines understand human laughter.

Technical Summary: Re-defining Humor Data Objects for AI Humor Research

Problem Statement

Existing AI humor research predominantly treats humor as a binary classification task ("present" or "not present") based on textual objects. This approach ignores the psychological definition of humor as a social behavior involving interaction, context, and explanation. Current datasets typically lack the rich structural elements required to understand why humor succeeds or fails, such as the social roles involved (teller vs. receiver), the specific reasoning behind a reaction, or recovery strategies for failed attempts. Furthermore, existing humor datasets often rely on transcripts from multimodal sources (e.g., TED talks, sitcoms) that omit crucial visual cues, timing, or delivery, leading to incomplete context for AI models.

Methodology

The authors propose a redefinition of the "humor" data object and a pipeline for generating structured humor reasoning data using Large Language Models (LLMs).

1. The Humor Data Object

The paper defines a humor data point as a quintuple $(C, X; Y, R; Z)$ :

$C$ (Context): Social dialogue or preceding context.
$X$ (Attempt): The teller's humorous utterance or punchline.
$Y$ (Reaction): A binary label indicating if the receiver felt humor ($1$) or not ($0$).
$R$ (Reasoning): The receiver's explanation for the reaction, detailing why the humor succeeded or failed.
$Z$ (Recovery): The teller's response to a failed attempt (e.g., apology, deflection), marked as 'n/a' if $Y=1$ .

The preliminary work focused primarily on generating $R$ (the explanation).

2. Dataset Construction and Prompt Iteration

The researchers utilized existing datasets containing observable reactions, specifically the SMILE dataset (TED+sitcoms). They extracted local textual context ( $C$ ), the candidate punchline ( $X$ ), and the observed reaction ( $Y$ ).

To generate the reasoning ( $R$ ), the team employed an iterative prompting strategy with an LLM (base model: gpt-4o-mini):

Prompt 1 (Initial): Asked the model to generate a structured JSON output containing step-by-step reasoning, a causal flow chart, a final explanation, and a hallucination flag. This prompt assumed the text was sufficient and often forced an explanation even when context was missing.
Prompt 2 (Revised): Developed after manual review of 31 examples revealed significant errors. This prompt explicitly required the model to:
1. Assess whether the transcript alone was sufficient to support the humor explanation.
2. Flag data quality issues, such as missing context, alignment errors, or multimodal dependence.
3. Avoid hallucinating explanations when the text was inadequate.

3. Evaluation and Scaling

The team manually evaluated the outputs of both prompts against the same 31 examples. Criteria for evaluation included groundedness (reliance on $C$ and $X$ ), completeness, identification of the true humor mechanism, and the absence of unsupported facts. Following the success of Prompt 2, the team scaled the generation process to produce a dataset of 307 explanation examples.

Key Results

Prompt Improvement: Manual evaluation on the 31-example test set showed a significant improvement with the revised prompt.
- Prompt 1: 17/31 outputs were acceptable; 14 were incorrect.
- Prompt 2: 27/31 outputs were acceptable; only 4 were incorrect.
Error Analysis: The initial failures were categorized into four types:
1. Incomplete Context: The transcript lacked background information necessary to understand the joke.
2. Unexplained Named Entities: The model failed to explain real-world references (e.g., "FedEx days").
3. Multimodal Dependence: The humor relied on visual cues, gestures, or timing not present in the text.
4. Unreliable Transcripts: Mistranscribed words or misaligned laughter markers led to incoherent explanations.
Scaling: The revised prompt successfully generated 307 structured data points. Crucially, it reduced the tendency to hallucinate explanations when the source text was flawed, instead flagging the insufficiency of the data.
Automatic Optimization: An experiment using DSPy/GEPA for automatic prompt optimization did not yield performance gains beyond manual refinement, suggesting that human-informed error analysis was the primary driver of improvement.

Significance and Contributions

The paper claims the following contributions to the field of AI humor research:

Redefinition of Data Objects: Moving beyond binary labels to a structured quintuple that captures humor as a social interaction involving context, attempt, reaction, reasoning, and recovery.
Scalable Data Synthesis: Demonstrating that well-designed LLM prompts can be used to generate large-scale, structured humor reasoning data, even from imperfect source transcripts. This offers a method for data augmentation and synthesis.
Handling Multimodality and Context: Establishing that effective humor explanation generation requires the model to explicitly assess text sufficiency and flag multimodal dependencies, rather than forcing an explanation.
Foundation for Future Work: The generated dataset and the defined data object provide a baseline for future research in humor understanding, data synthesis, and the development of socially aware AI models.

The authors maintain a modest stance, noting that the 307-example dataset is a preliminary resource rather than a gold standard, and that future work is needed to systematically evaluate explanation quality across different failure modes and to explore the dataset's utility in training humor generation models.