💬 NLP

Example-Guided Prompting for Document-Level Text Simplification

This paper demonstrates that augmenting prompts with retrieved document-simplification examples significantly improves the quality of document-level text simplification in large language models, achieving performance competitive with or superior to specialized supervised systems without requiring task-specific fine-tuning.

Original authors: Marina Litvak, Ariel Perstin, Ilan Shtilman, Michael Färber

Published 2026-08-07

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Marina Litvak, Ariel Perstin, Ilan Shtilman, Michael Färber

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to rewrite a complicated story so that a younger child can understand it. This is the world of text simplification, a field where computer scientists teach machines to take dense, adult-level writing and turn it into something clear and easy to read, without losing the original meaning. For a long time, the best way to do this was to train the robot on thousands of specific examples until it memorized the rules. But recently, a new kind of super-smart robot called a Large Language Model (LLM) arrived. These models are like brilliant students who can follow instructions and learn new tricks just by reading a prompt, without needing to be retrained from scratch. However, even these smart robots sometimes get confused when asked to rewrite a whole book or article at once. They might change the meaning, lose the flow of the story, or make it too simple in some parts and too hard in others. The big question researchers are asking is: How do we give these robots the best possible guidance to do a perfect job?

This paper introduces a clever solution called Example-Guided Prompting (EGP). Think of it like this: If you asked a student to write a summary of a complex science article, you could just give them a set of written rules (a "prompt"). But what if, instead, you also handed them a few examples of how other students successfully simplified similar articles? The researchers found that giving the robot these "reference sheets" of real-world examples works wonders. They tested this idea using a collection of educational news articles called the OneStopEnglish corpus. They took complex documents and asked five different state-of-the-art robots to simplify them. In one group, the robots only got the written instructions. In the other group, the robots got the instructions plus three similar examples of documents that had already been simplified correctly.

The results were quite revealing. When the robots were shown these examples, they generally did a much better job. Specifically, the robot named Claude Haiku 4.5 saw its performance score jump from 40.78 to 44.84, and it also kept the original meaning of the text much better than before. Other robots like Gemini 2.0 Flash and Sonar also improved significantly. In fact, this simple trick of showing examples allowed these robots to beat or match the performance of older, specialized systems that had been trained specifically for this task for years. However, the paper also discovered that this trick doesn't work for every robot. One model, Llama 3.2, actually got worse when shown the examples, suggesting that not all robots know how to use a "reference sheet" effectively. The study suggests that while showing examples helps the robots stay true to the original meaning, it sometimes makes the text slightly less "easy to read" than if they had just guessed on their own, creating a delicate balance between keeping the story accurate and making it simple. Ultimately, the authors show that for many powerful robots, looking at how others solved similar problems is a better teacher than just reading a list of rules.

Technical Summary: Example-Guided Prompting for Document-Level Text Simplification

Problem Statement

Document-level text simplification presents a significant challenge for Large Language Models (LLMs). Unlike sentence-level rewriting, which focuses on local transformations, document-level simplification requires coordinated changes across multiple sentences to maintain discourse coherence, semantic fidelity, and consistent readability throughout an entire text. While prompt-based LLMs offer a flexible, training-free alternative to supervised models, they often struggle with complex document-level transformations. Relying solely on textual instructions frequently leads to inconsistent simplification quality, as instructions alone provide insufficient guidance for the nuanced structural and stylistic shifts required.

Existing approaches, such as Retrieval-Augmented Generation (RAG), typically utilize retrieved documents to provide factual knowledge or external context. However, document simplification requires guidance on how to transform text rather than additional factual information. There is a gap in utilizing retrieval to provide transformation patterns specifically for document-level simplification without task-specific fine-tuning.

Methodology: Example-Guided Prompting (EGP)

The authors propose Example-Guided Prompting (EGP), an inference-time framework designed to augment standard prompts with retrieved document-simplification examples.

Core Mechanism

Retrieval: Given a complex input document, the system retrieves the top- $k$ (where $k=3$ ) most semantically similar document-simplification pairs from a parallel simplification corpus. This retrieval utilizes dense embeddings and cosine similarity.
Augmentation: The retrieved examples are incorporated into the prompt alongside the target document and the textual instruction.
Generation: The LLM processes this enriched prompt, leveraging the retrieved examples to identify and apply relevant simplification patterns observed in similar contexts.
Constraint: The evaluated input document is explicitly excluded from the retrieval pool to prevent data leakage.

Experimental Setup

Dataset: The OneStopEnglish corpus (Vajjala and Luˇci´c, 2018), specifically the advanced-to-elementary simplification setting.
Models: Five state-of-the-art LLMs were evaluated: GPT-4o-mini, Claude Haiku 4.5, Gemini 2.0 Flash, Sonar, and Llama 3.2.
Baselines:
- Prompt-only: A constraint-based prompting strategy was selected as the baseline after preliminary experiments showed it achieved the highest SARI scores.
- Supervised/Planning: Comparisons were made against T5 (Raffel et al., 2020) and PlanSimp (Cripwell et al., 2023), a planning-based system.
Evaluation Metrics:
- Simplification Quality: SARI (Xu et al., 2016).
- Semantic Preservation: BERTScore (Zhang et al., 2019) and LENS (Maddela et al., 2023).
- Readability: Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level (FKGL).

Key Results

The experiments demonstrate that EGP consistently improves prompt-only generation across most models and achieves competitive or superior performance compared to specialized supervised systems.

Performance Gains: Incorporating retrieved examples improved simplification quality (SARI) for four of the five evaluated LLMs.
- Claude Haiku 4.5 showed the most significant improvement, increasing SARI from 40.78 to 44.84, while also improving semantic preservation (BERTScore and LENS).
- Gemini 2.0 Flash and Sonar also demonstrated consistent gains.
- GPT-4o-mini showed smaller but consistent improvements.
- Llama 3.2 was an outlier, experiencing a notable performance degradation across all metrics, suggesting that not all LLMs can effectively integrate retrieved examples.
Comparison with Baselines: EGP models generally outperformed or matched the specialized T5 and PlanSimp systems.
- Claude Haiku 4.5 + EGP achieved the highest SARI score (44.84), surpassing PlanSimp (42.02) and T5 (30.0).
- In terms of semantic preservation, GPT-4o-mini and Claude Haiku 4.5 substantially outperformed both T5 and PlanSimp on LENS, improving over the planning-based baseline by more than 20 points.
Trade-offs: A distinct trade-off was observed between readability and semantic preservation. While EGP consistently improved semantic-oriented metrics (BERTScore, LENS), it did not always yield the largest gains in readability metrics (FKGL, FRE). The retrieved examples appear to encourage "conservative" simplifications that remain closer to the source document's structure and meaning, prioritizing fidelity over aggressive readability reduction.

Significance and Contributions

The paper makes three primary contributions:

Proposal of EGP: The introduction of Example-Guided Prompting as a novel inference-time approach for document-level text simplification. Unlike traditional RAG which provides factual grounding, EGP uses retrieval to provide transformation guidance, enabling LLMs to exploit simplification patterns without task-specific fine-tuning.
Empirical Validation: The demonstration that EGP consistently enhances prompt-only generation across multiple state-of-the-art LLMs. The results show that this retrieval-based approach can achieve performance competitive with or superior to representative supervised (T5) and planning-based (PlanSimp) systems, despite not requiring training data specific to the simplification task.
Analysis of Model Dependence: The identification that the efficacy of example-guided prompting is highly dependent on the underlying LLM. The study highlights that successful integration of retrieved examples is not guaranteed; it relies on the model's specific ability to utilize contextual information during generation. This finding suggests that retrieval quality alone is insufficient and points to architectural or training characteristics as critical factors for future research.

The authors conclude that retrieved examples serve as an effective mechanism for controlling text generation through transformation guidance. This perspective extends beyond text simplification, potentially applying to other controlled generation tasks where examples illustrate desirable transformations rather than providing factual information.

Limitations

The authors acknowledge several limitations:

Dataset Scope: Experiments were limited to the OneStopEnglish corpus (educational news in English), and generalizability to other domains, languages, or styles remains unverified.
Evaluation Metrics: Reliance on automatic metrics (SARI, BERTScore, etc.) which may not fully capture human judgments of coherence, factual consistency, or audience suitability.
Retrieval Configuration: The study used a single retrieval configuration and corpus. Variations in retrieval strategies, corpus sizes, or similarity measures could influence outcomes.
Model Variability: The significant performance drop observed in Llama 3.2 indicates that the method is not universally effective across all LLM architectures, necessitating further investigation into why some models integrate examples better than others.