💬 NLP

CLIX: Cross-Lingual Explanations of Idiomatic Expressions

This paper introduces CLIX, a task focused on generating cross-lingual explanations for idiomatic expressions to aid language learners, demonstrating the potential of large language models while identifying key challenges through error analysis.

Original authors: Aaron Gluck, Katharina von der Wense, Maria Leonor Pacheco

Published 2026-04-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Aaron Gluck, Katharina von der Wense, Maria Leonor Pacheco

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to learn a new language, like Spanish or German. You come across a phrase like "kick the bucket." If you look this up in a standard dictionary, you might get a dry definition like "to die." But that doesn't really help you understand why someone would say that, or how to use it in a conversation. It's like trying to understand a joke by reading a medical report about the human body; the facts are there, but the feeling is missing.

This paper, titled CLIX, is about building a smarter kind of dictionary for language learners. The authors want to create a system that doesn't just translate words, but explains idioms (those tricky phrases where the meaning isn't the sum of the parts) in the learner's native language.

Here is a breakdown of their work using some simple analogies:

1. The Problem: The "Word-for-Word" Trap

The authors point out that current computer systems are like naive tourists who try to translate a menu by translating every single word. If a menu says "It's raining cats and dogs," a naive translator might say, "It is raining felines and canines." That's confusing!

Language learners struggle because:

Definitions are too hard: The definitions often use big, complicated words that the learner doesn't know yet.
Idioms are tricky: Idioms don't make sense if you look at the individual words. "See eye to eye" has nothing to do with eyes or vision; it means agreeing with someone.

2. The Solution: The "Cultural Guide"

The authors propose a new task called CLIX (Cross-Lingual explanations of Idiomatic eXpressions). Think of this not as a dictionary, but as a local tour guide.

Instead of just giving a definition, the system tries to explain the story behind the phrase.

The Goal: Take an English idiom (like "break a leg") and explain it in Spanish or German in a way that feels natural and helpful to a learner.
The Twist: The explanation doesn't have to be a single, static sentence. It can include examples, context, or even a bit of history, just like a good teacher would.

3. The Toolkit: Two New Datasets

To teach computers how to do this, the authors built two massive "training libraries" (datasets):

EPIE-ME: A smaller, open library of 628 idioms with explanations in English, Spanish, and German.
Oxford-ME: A huge library of over 6,000 idioms from the famous Oxford Dictionary of Idioms.

They manually checked and corrected these explanations to make sure they were accurate, acting like editors ensuring the "tour guide" script was perfect before letting the computer read it.

4. The Experiment: Testing the "Students"

The authors tested different types of AI models to see who could act as the best "tour guide."

The Old School Students (T5/mT5): These are older, specialized AI models. They tried to learn the task but often stumbled, getting confused or repeating words like a broken record. Their "grades" (scores) were around 40%.
The Super-Students (LLMs like GPT and Llama): These are the massive, modern AI models that have read almost everything on the internet. They performed much better, scoring around 70%. They understood the nuance of the idioms much better, almost like a native speaker.

Key Finding: The best strategy wasn't just asking the AI to "translate." It was a two-step process (a pipeline):

First, ask the AI to explain the idiom in English (the source language).
Then, ask it to translate that clear explanation into the target language (Spanish or German).
This worked better than trying to do it all in one giant leap.

5. The Reality Check: Human Judges

The authors didn't just rely on computer scores. They hired native speakers (real humans) to grade the AI's explanations.

The Good News: The humans loved the explanations! They gave high marks for how natural the language sounded (Fluency) and how accurate the meaning was (Accuracy).
The Bad News: The AI still makes mistakes. Sometimes it repeats words weirdly, or it gets too literal (explaining "flesh and blood" as just body parts instead of family).
The Gap: The computer scores didn't perfectly match the human feelings. A computer might say two sentences are "similar," but a human might say one is confusing and the other is clear.

6. The Conclusion: Not Ready for the Classroom Yet

The paper concludes that while these AI models are promising, they aren't quite ready to be the sole teacher in a classroom yet.

They are like apprentice chefs: They can make a delicious meal 70% of the time, but sometimes they forget the salt or burn the toast.
Before we can trust them to teach students, we need to fix the errors where they get the meaning slightly wrong or sound robotic.

In short: The authors built a new way to teach computers how to explain tricky phrases in different languages. They found that modern AI is getting very good at it, but it still needs a human teacher to double-check its work before it can be used in schools.

1. Problem Statement

Automated definition generation systems are widely used to support vocabulary expansion for second language learners. However, these systems face significant barriers:

Complexity of Definitions: Generated definitions often contain unfamiliar vocabulary or complex grammar, hindering comprehension.
Idiomatic Expressions: Standard definition generators often fail with non-standard language (idioms), where the meaning is not the sum of its parts (e.g., "see eye to eye").
Monolingual Limitations: Existing approaches often provide definitions in the target language, which may not be accessible to beginners.
Translation vs. Explanation: Naïve translation of idioms often fails because equivalent idioms do not exist across languages, and literal translation destroys meaning.

The authors propose CLIX (Cross-Lingual Explanations of Idiomatic Expressions), a task where the goal is to generate a natural language explanation of an English idiom in a learner's native language (Target Language, $L_T$ ), rather than a static definition or a direct translation.

2. Methodology

A. Dataset Construction

The authors created the first cross-lingual datasets for this specific task, containing English idioms with explanations in English, Spanish, and German:

EPIE-ME: Derived from the EPIE corpus (Saxena and Paul, 2020). Contains 628 unique idioms with multi-lingual explanations. Includes sentence-level context and thematic categorization.
Oxford-ME: Derived from the Oxford Dictionary of Idioms (4th Edition). Contains 6,218 unique idioms. Since this source lacked sentence context for many entries, the authors used Llama 3.1 to generate context sentences.

Translation Strategy: Non-English explanations were generated by translating gold-standard English explanations using Google Translate, followed by rigorous manual correction by native-speaking computational linguists to ensure accuracy and fluency.

B. Task Formulation

The task is framed as a text-to-text generation problem:

Input: An idiom $I$ (and optional context $C$ ) in English ( $L_S$ ).
Output: An explanation $E$ in the target language ( $L_T$ ).
Flexibility: Unlike strict definitions, "explanations" can include elaborations, usage examples, etymology, and contextual nuances.

C. Experimental Strategies

The authors evaluated several learning and inference strategies using both fine-tuned models and Large Language Models (LLMs):

Direct Explanation: The model is prompted/fine-tuned to generate the explanation directly in the target language.
Pipeline Approach: A two-stage process:
- Stage 1: Generate an explanation in English.
- Stage 2: Translate the English explanation to the target language.
Contextual Enhancements: Adding sentence-level context or thematic categories (based on Rafatbakhsh and Ahmadi's taxonomy) to the prompt.
Few-Shot Prompting: Testing LLMs (Llama 3.1 and GPT-3.5 Turbo) with 1, 3, or 5 in-context examples. They tested random selection vs. category-matched selection for examples.

D. Models and Metrics

Models: T5, mT5 (fine-tuned), Llama 3.1 (8B Instruct), and GPT-3.5 Turbo.
Metrics:
- SBERT (Multilingual): Primary metric for semantic similarity between generated and gold explanations.
- BLEU/ROUGE: Used for surface-level overlap.
- Human Evaluation: Native speakers rated Fluency and Accuracy on a 1-5 Likert scale.

3. Key Contributions

New Task Definition: Formalized the task of generating cross-lingual explanations for idioms, distinguishing it from simple translation or monolingual definition generation.
Dataset Release: Released EPIE-ME, the first open cross-lingual dataset for this task (English $\to$ Spanish/German). (Oxford-ME is available upon request due to copyright).
Comprehensive Evaluation: Provided a benchmark comparing fine-tuned sequence-to-sequence models against state-of-the-art LLMs in both zero-shot and few-shot settings.
Error Analysis & Human Study: Conducted a detailed human evaluation revealing that while automated metrics (SBERT) show moderate performance, native speakers rate LLM outputs highly, though specific error types (redundancy, literalism) persist.

4. Results

Automated Metrics (SBERT Similarity)

Fine-tuned Models (T5/mT5): Performed poorly, with similarity scores hovering around 40%.
LLMs (Zero-Shot): Showed significant improvement.
- GPT-3.5 Turbo: Achieved the highest scores, reaching ~69-70% similarity in the Pipeline strategy.
- Llama 3.1: Performed well but slightly lower than GPT (~60%).
Strategy Comparison: The Pipeline approach (Generate in English $\to$ Translate) generally outperformed the Direct approach for fine-tuned models and was competitive for LLMs.
Few-Shot Learning: Increasing the number of shots (1 to 5) improved performance, but random selection of examples performed as well as or better than category-matched selection, suggesting the models already possess strong internal representations of idioms.

Human Evaluation

Scores: Native speakers rated the best model's Spanish outputs with high scores: 4.70/5.0 for Fluency and 4.78/5.0 for Accuracy.
Discrepancy: There is a notable gap between automated metrics (SBERT ~70%) and human perception of quality (very high). The authors attribute this to SBERT measuring text similarity rather than the pedagogical utility or core meaning representation.

Error Analysis

Common errors identified by human annotators included:

Fluency: Word repetition and redundant phrasing (e.g., repeating "results" twice).
Accuracy:
- Literalism: Explaining the idiom literally rather than figuratively (e.g., explaining "dice with death" as participating in an activity causing death, rather than taking a risk).
- Meaning Mismatch: Correctly identifying a metaphor but linking it to the wrong domain (e.g., linking "flesh and blood" to humanity rather than family).

5. Significance and Future Work

Educational Impact: The study demonstrates that LLMs can generate high-quality, cross-lingual explanations that are understandable to learners, potentially overcoming the "unfamiliar word" barrier in traditional definition systems.
Limitations:
- Dataset Size: The publishable dataset (EPIE-ME) is small (628 idioms) compared to the full Oxford set.
- Language Scope: Currently limited to English source idioms and Spanish/German target languages.
- Metric Gaps: Current automated metrics fail to capture the nuance of "explanation quality" vs. "text similarity."
Future Directions: The authors suggest developing better metrics for explanation quality, investigating the "metaphor gradient" (how easily different types of idioms can be disambiguated), and refining strategies to handle literal vs. figurative interpretations more robustly in educational tools.

In conclusion, CLIX establishes a new benchmark for NLP in language education, showing that while current models are promising, further refinement is needed to ensure reliability in high-stakes educational applications.