Dark & Stormy: Modeling Humor in Sentences from the Bulwer-Lytton Fiction Contest

1. Problem Statement

Computational humor research has traditionally focused on familiar, "good" humor forms such as puns, satirical headlines, and short one-liners. Consequently, existing humor detection models and datasets fail to capture the nuances of intentionally bad humor—a genre where the goal is to write the worst possible opening sentence to a novel. This paper addresses the gap in understanding this specific sub-genre by analyzing the Bulwer-Lytton Fiction Contest (BLFC), where participants craft sentences that are deliberately verbose, convoluted, and stylistically poor. The authors aim to:

1. Characterize how "bad" humor differs structurally and semantically from standard humor datasets.
2. Evaluate how Large Language Models (LLMs) imitate this specific style compared to human writers, specifically looking for distortions or exaggerations in the generated output.

2. Methodology

Data Curation

The authors constructed a novel corpus comprising 1,778 human-written BLFC entries (1996–2024), annotated with metadata such as genre (e.g., Science Fiction, Romance, Vile Puns).

• Baseline Datasets: To establish a comparative framework, the authors gathered:
  • Novel-openings: Crowdsourced first sentences of novels (non-humorous baseline).
  • Combo-humor: A diverse collection of standard humor (puns, satire, Reddit jokes, Twitter jokes).
  • PotD (Pun of the Day): A dataset specifically for pun detection.
• Synthetic Data: The authors generated 4,000 synthetic BL sentences using four different LLMs (DeepSeek-V3.1, GPT-5, GPT-4.1, and gpt-oss-120b) via a minimal one-shot prompt that defined the contest rules and provided an example.

Analytical Framework

The study employed three primary technical approaches:

1. Humor Detection Benchmarking: The authors tested state-of-the-art RoBERTa-based humor detection models (trained on standard humor datasets) on the BL corpus to measure classification performance.
2. Literary Device Extraction: Using a prompt-based framework adapted from TopicGPT, the authors extracted eight specific literary features (Irony, Metaphor, Simile, Satire, Pun, Repetition, Metafiction, Onomatopoeia) from the sentences. This was validated via an intruder task where human annotators identified sentences that did not fit a specific feature label.
3. Incongruity and Semantic Deviance Analysis:
   • Surprisal Analysis: Using the gemma-3-1b-pt language model, the authors calculated token-level surprisal (negative log-probability) to measure expectation violation. They analyzed the distribution of high-surprisal tokens across sentence positions.
   • Novel Adjective-Noun (AN) Bigrams: Inspired by Vecchi et al. (2017), the authors identified "deviant" AN bigrams by querying their frequency in the massive DCLM pre-training corpus using Infini-gram. Low-frequency ANs indicate semantic novelty or jarring combinations.

3. Key Results

A. Failure of Standard Humor Detection

Standard humor detection models trained on "good" humor (e.g., puns or satire) perform poorly on BL sentences.

• BL sentences receive low humor scores, comparable to non-humorous novel openings.
• Even models trained specifically on puns fail to recognize the humor in BL entries labeled "Vile Puns," suggesting that BL humor relies on a complex interplay of devices rather than a single mechanism like a pun.

B. Distinctive Literary Profile

BL sentences exhibit a unique stylistic signature distinct from both standard humor and standard novel openings:

• Higher Usage of Specific Devices: BL sentences significantly exceed baselines in Irony, Metafiction, and Simile.
• Verbosity: BL sentences are substantially longer (avg. 70.5 tokens) than standard jokes (avg. 14–19 tokens) or novel openings (avg. 24.5 tokens).
• Device Distribution: While standard humor relies heavily on puns and satire, BL humor distributes its "badness" across a broader, more complex set of literary devices.

C. LLM Imitation vs. Exaggeration

LLMs successfully imitate the form of BL sentences but consistently exaggerate the features that make them "bad":

• Overuse of Devices: Synthetic sentences show a marked tendency to overuse Simile, Metaphor, and Onomatopoeia compared to human-written entries.
• Surprisal Distribution:
  • Human BL sentences have a flatter distribution of high-surprisal tokens throughout the sentence.
  • LLMs (particularly GPT-5) generate sentences with a significantly higher number of high-surprisal tokens (9.2±2.8 for GPT-5 vs. 5.1±3.1 for humans).
• Semantic Deviance (Novel ANs):
  • BL sentences contain more novel Adjective-Noun bigrams than baselines.
  • LLMs generate even more novel ANs than humans. For example, 10% of human BL ANs occur <100 times in the DCLM corpus, whereas synthetic models generate even higher proportions of rare/unattested combinations.
  • Model Variance: DeepSeek's distribution of surprisal and novelty aligns closer to human writers than GPT-5, which produces the most extreme "bad" humor.

4. Key Contributions

1. Novel Dataset: Introduction of the first large-scale, curated corpus of "intentionally bad humor" from the Bulwer-Lytton Fiction Contest, complete with genre metadata.
2. Empirical Characterization: Demonstration that "bad humor" is a distinct linguistic category that standard NLP humor models cannot detect, requiring new feature sets (e.g., metafiction, complex similes) rather than just pun detection.
3. LLM Style Analysis: A rigorous comparison showing that while LLMs can mimic the style of bad humor, they lack the human ability to calibrate "badness," leading to an exaggeration of semantic deviance and literary device usage.
4. Methodological Framework: A pipeline for analyzing literary devices and semantic incongruity (via surprisal and AN frequency) that can be applied to other stylistic analysis tasks.

5. Significance

This paper challenges the assumption that humor detection is a monolithic task. It reveals that intentionally bad humor operates on different linguistic principles than standard humor, relying on semantic incongruity and stylistic excess rather than simple punchlines.

For the field of Generative AI, the findings are critical: they suggest that LLMs, when prompted to generate "bad" or "creative" text, tend to over-optimize for novelty and complexity, resulting in outputs that are statistically more "deviant" than human attempts. This highlights a fundamental difference in how humans and models navigate the space of "bad" writing, offering a new benchmark for evaluating the nuance and calibration of LLM generation.