💬 NLP

Searching for Sound-Meaning Collisions: Graph-Based Affordance Retrieval and Multi-Evaluator Ranking for Pun Translation at CLEF 2026 JOKER Task 2

This paper presents a graph-based retrieval and multi-evaluator ranking system for pun translation that computationally validates Low's theory by demonstrating that successful translation relies on discovering new sound-meaning collisions in the target language rather than preserving source words, while identifying retrieval as the current primary bottleneck.

Original authors: Russell Taylor, Adam Brikman, Prateek Awate

Published 2026-08-06

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Russell Taylor, Adam Brikman, Prateek Awate

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to translate a joke from one language to another, but the joke relies on a word that sounds like two different things at once. In English, you might say, "I'm on a seafood diet; I see food and I eat it." The humor comes from the sound of "sea" matching "see." If you try to translate this word-for-word into French, the joke dies instantly because the French words for "sea" and "see" don't sound alike. This is the "impossible" problem of pun translation: the magic happens in the accidental collision of sound and meaning, and that accident rarely survives the trip to a new language.

For decades, translators have known the secret isn't to keep the original word, but to find a new word in the target language that creates a similar crash between sound and meaning. It's like trying to build a bridge between two islands; if the original bridge is destroyed, you have to find a new spot where the water is shallow enough to build a fresh one. This paper explores how we can teach computers to do exactly that: stop looking for the same word and start searching for new places where sound and meaning bump into each other.

The Computer's Treasure Hunt for Wordplay

In this paper, researchers from the Georgia Institute of Technology treat translating a pun not as a simple swap of words, but as a three-step adventure: Discovery, Exploration, and Selection. They built a system designed to hunt for "affordances"—a fancy word for "opportunities" or "bridges" in the target language where a sound and a meaning collide in a funny way.

Step 1: The Discovery (Finding the Bridges)
The team realized that to find a pun, you can't just look at single words; you have to look at phrases and sound patterns too. They built a massive digital library of French expressions and a special map of how French words sound (using phonetic codes).

The Semantic Map: They looked for groups of words that share a meaning (like a neighborhood of words about "food").
The Sound Map: They built a graph that connects words that sound similar, not just exact twins, but also cousins that rhyme or share a similar rhythm.

The computer then tries to find a "bridge" connecting the two sides of the original joke. For example, if the joke connects "money" and "time," the computer searches the French language for a word that sounds like a word for "money" but means "time," or vice versa. They call these potential bridges "affordances."

Step 2: The Exploration (Generating Candidates)
Once the computer finds these bridges, it doesn't just pick one. It sends the idea to several different AI "writers" (large language models). These writers are told: "Here is a bridge you found; now write a joke using it."
Instead of writing just one joke, the system asks them to write twelve different versions. Some might be silly, some might be clever, and some might be perfect. This turns the process from "guessing one answer" into "generating a whole menu of options."

Step 3: The Selection (The Panel of Judges)
Now comes the hard part: picking the winner. The system uses a panel of four different AI "personas" to judge the jokes, just like a panel of human experts:

The Comedian: Which one is the funniest?
The Linguist: Which one has the best wordplay?
The Editor: Which one sounds natural and fluent?
The Translator: Which one keeps the original meaning of the joke?

These judges don't just give a score; they rank the jokes. The system then combines their rankings to pick the single best joke for each original pun.

What They Found (and What They Didn't)

The results of this experiment were revealing, suggesting that the computer's process looks a lot like how human translators think.

1. The Hunt is the Hard Part
The biggest bottleneck in the whole process is the "Discovery" phase. Even with their advanced maps, the system only found a usable bridge for about 50.8% of the jokes. For the other half, the computer simply couldn't find a sound-meaning collision in French that worked. This suggests that for many puns, the "bridge" just doesn't exist in the target language, and the joke has to be abandoned.

2. The Generators Love the Bridges
When the system did find a bridge, the AI writers used it. They didn't ignore the clues; they actively built their jokes around the sound-meaning collisions the system found. The more bridges they found, the better the jokes tended to be.

3. Exact Sound Matches Win Big
Here is a surprising twist: while the system found thousands of "near-miss" sound bridges (words that sound almost the same), the final winners were disproportionately "exact" matches. Even though exact sound collisions were rare (only about 10.8% of the found bridges), they ended up making up nearly 27.5% of the final winning jokes. It seems that when a perfect sound-alike exists, it's just too good to pass up.

4. The "Translator" Judge is the Key
When the team tested different ways to combine the judges' opinions, they found that prioritizing the "Translator" (who cares about keeping the original meaning) worked best on the public leaderboard. However, the authors note a tension here: the leaderboard seems to reward keeping the meaning more than it rewards being funny. The "Comedian" judge often picked different jokes than the "Translator," suggesting that what makes a joke funny to a human might not be what the computer's scoring system currently values.

5. It's a Search, Not a Magic Trick
The most important takeaway is that successful pun translation isn't about preserving the original word. It's about discovery. The computer has to search the vast ocean of the target language, find a hidden spot where sound and meaning crash into each other, and then build a new joke there.

The authors conclude that their system successfully mimics the process proposed by a linguist named Low years ago: translators shouldn't look for equivalent words; they should look for new points of contact. While the system still struggles with half the jokes (because the bridges are hard to find), it proves that the path to a translated pun isn't a straight line—it's a treasure hunt for hidden collisions between sound and sense.

Technical Summary: Graph-Based Affordance Retrieval and Multi-Evaluator Ranking for Pun Translation

Problem Definition
Pun translation is widely considered a difficult computational task because the specific linguistic accidents that create wordplay in a source language rarely survive direct translation. Traditional approaches often fail by attempting to preserve the original source word. Instead, drawing on Low's theoretical framework, this paper posits that successful pun translation requires abandoning the search for equivalent words in favor of discovering new points of contact between sound and meaning within the target language. The core challenge is to computationally model this process of discovery, exploration, and selection to find "affordances"—target-language opportunities where sound and meaning collide to support new wordplay.

Methodology
The authors propose a system that treats pun translation as a pipeline of retrieval, generation, and ranking, significantly expanding upon their 2025 CLEF submissions.

Preprocessing and Identification:
- The pipeline begins with data cleaning to normalize Unicode, repair dialogue structures, and remove social media artifacts.
- A specialized stage identifies the pun word, its type, and two associated semantic domain word lists. This stage uses structured JSON outputs from gemini-3-flash to prepare inputs for retrieval.
Graph-Based Affordance Retrieval:
- Expanded Search Space: Moving beyond word-level retrieval, the system utilizes a 370,450-entry French expression bank (combining Wiktionary, PARSEME, and OpenSubtitles data) indexed with BAAI/bge-m3 embeddings.
- Learned Phonetic Embeddings: A large-scale phonetic relevance graph (245k unique word–IPA pairs) was constructed to train a phonetic encoder. This encoder captures broader phonological neighborhoods (rhymes, consonant/vowel skeletons, near-homophones) rather than just exact homophones, using Multiple Negatives Ranking Loss.
- Bridge Mining: The retrieval space is modeled as a heterogeneous graph. The system performs six ordered stages of "bucketized bridge mining" to connect the two semantic domains of the source pun. This involves semantic expansion of source terms followed by phonetic neighborhood expansion to find pairs $(x, y)$ where $x$ and $y$ share phonetic similarity while anchoring to the respective semantic domains.
- Ranking and Pruning: Retrieved affordances are scored based on phonetic similarity, semantic anchorability, naturalness, and utility for pun construction. Heuristic pruning removes trivial variants and enforces diversity.
Candidate Generation:
- The system shifts from a single-solution paradigm to a "generate-and-select" approach.
- For claude-sonnet-4.6 and gemini-3-flash, the generator produces twelve distinct French pun candidates per source, treating retrieved affordances as creative opportunities rather than strict constraints.
- gpt-5.5 and gemini-3.1-pro generate a single candidate per example to evaluate frontier model performance with lower generation costs.
Two-Stage Multi-Agent Ranking:
- Stage 1 (Intra-Model Selection): A single judge model ranks the twelve candidates from a generator using four distinct personas: Comedian (funniest), Linguist (strongest wordplay), Editor (most fluent), and Translator (best intent preservation). Rankings are aggregated via weighted Borda voting to select one winner per generator.
- Stage 2 (Cross-Model Ensemble): The winners from Stage 1, combined with single candidates from other models, form a pool of four. These are re-ranked by the same four personas using the same aggregation strategy.
- The system experiments with various aggregation orders (Judges-first vs. Models-first) and persona weightings to optimize the final ensemble.

Key Results

Retrieval Coverage: The expanded retrieval system achieved 50.8% coverage of the dataset (2,064 of 4,061 puns), a significant increase from previous iterations. However, nearly half of the source puns still yielded no usable affordances.
Affordance Utilization: While retrieval provided thousands of affordances, the selection process progressively concentrated on higher-quality bridges. "Same-sound" (exact phonological) affordances, though only 10.8% of the retrieved inventory, were selected at disproportionately high rates (up to 27.5% of finalists), suggesting they are the most effective for wordplay.
Pipeline Efficiency: The mean quality score of affordances increased at each stage of the pipeline (from 0.677 at retrieval to 0.724 for final winners), indicating that generators and evaluators successfully filter for stronger sound–meaning collisions.
Leaderboard Performance: The system achieved first place on the public CLEF 2026 JOKER Task 2 leaderboard with a score of 37.783. Analysis revealed that "translator-focused" selection strategies (prioritizing semantic fidelity) outperformed "comedian" or "editor" strategies, suggesting the automatic metrics reward meaning preservation over creative wordplay.
Evaluation Dynamics: Significant disagreement was observed between judge personas (e.g., only 16.8% complete agreement for Claude examples), validating the need for multi-perspective aggregation. Position bias was also noted, where candidate order influenced rankings even after randomization.

Significance and Claims
The paper's primary contribution is not merely the leaderboard ranking, but the computational validation of Low's theoretical framework. The authors demonstrate that successful pun translation emerges not from preserving source words, but from a search process that discovers new semantic–phonetic bridges in the target language.

Validation of Theory: The system's architecture—retrieval for discovery, generation for exploration, and ranking for selection—mirrors Low's conceptual "squares, pentagons, and hexagons," translating them into concrete computational steps (semantic expansion, phonetic neighborhoods, affordance discovery).
The Bottleneck: The authors modestly conclude that while retrieval is effective, it remains the central bottleneck. The scarcity of usable affordances (nearly 50% of puns yield none) suggests that current retrieval methods cannot yet fully replicate the intuition of human translators who can navigate vast semantic and phonological spaces.
Nature of Success: The results indicate that successful pun translation is a process of finding "hidden bridges" where sound and meaning collide. The system confirms that exact phonological collisions are disproportionately valuable, yet the difficulty of finding them computationally remains a significant challenge.

The paper concludes that while the search is difficult and many puns resist translation, the realization of Low's insight as a computational process represents the most satisfying outcome of the work.

The Computer's Treasure Hunt for Wordplay

What They Found (and What They Didn't)

Technical Summary: Graph-Based Affordance Retrieval and Multi-Evaluator Ranking for Pun Translation

More like this