Translate or Simplify First: An Analysis of Cross-lingual Text Simplification in English and French

1. Problem Statement

Cross-Lingual Text Simplification (CLTS) aims to make content accessible to diverse audiences (e.g., language learners, individuals with reading difficulties) by simultaneously translating text from a source language to a target language and reducing its linguistic complexity.

While Machine Translation (MT) and Monolingual Text Simplification (TS) are mature fields, CLTS presents unique challenges:

• Interdependence: Simplification strategies vary by language due to grammatical and cultural differences.
• Ordering Ambiguity: It is unclear whether one should translate first then simplify, simplify first then translate, or perform both simultaneously.
• Evaluation Gaps: Traditional metrics (like BLEU) often penalize legitimate simplification operations, and existing evaluations lack systematic comparisons of prompting strategies for Large Language Models (LLMs) in a cross-lingual context.

2. Methodology

Experimental Setup

• Corpora: The study utilized five diverse datasets covering general knowledge (Wikipedia) and medical texts:
  • ASSET (English)
  • WikiAuto (English)
  • MultiCochrane (Multilingual: EN, FR, ES, FA)
  • CLEAR (French medical)
  • WikiLarge-Fr (French)
• Models: Seven state-of-the-art LLMs were evaluated, including proprietary models (GPT-3.5, GPT-4o-mini, Gemini 2.5 Flash-Lite) and open-source models (DeepSeek, Aya101, Mistral-NeMo, mT5).
• Prompting Strategies: The core of the study compares five distinct approaches to handling the CLTS task:
  1. Direct Prompting: A single prompt instructing the model to translate and simplify simultaneously.
  2. T→S Composition: A single prompt instructing the model to "Translate then Simplify" (sequential reasoning within one prompt).
  3. S→T Composition: A single prompt instructing the model to "Simplify then Translate."
  4. T→S Decomposition: Two separate consecutive prompts (Step 1: Translate; Step 2: Simplify).
  5. S→T Decomposition: Two separate consecutive prompts (Step 1: Simplify; Step 2: Translate).

Evaluation Framework

The authors employed a multi-faceted evaluation approach:

1. Automatic Metrics:
   • BLEU: Measures translation fidelity (n-gram overlap).
   • SARI: Measures simplification quality (additions, deletions, retentions).
   • Semantic Similarity: BERTScore (for EN→FR) and CamemBERTScore (for FR→EN) to assess meaning preservation.
2. Linguistic Feature Analysis: A comprehensive set of 37 features (based on the BATS framework) was extracted, including lexical richness, syntactic tree depth, sentence length, readability scores (Flesch-Kincaid, Flesch Reading Ease), and grammatical structures.
3. Human Evaluation: Annotators rated outputs on Meaning Preservation (information addition/removal) and Simplicity (relative complexity) using Likert scales.

3. Key Contributions

1. First Systematic Comparison: This is the first study to systematically compare five different prompting strategies for CLTS between English and French across multiple LLMs and domains.
2. Directional Asymmetry Discovery: The study reveals that the optimal strategy depends heavily on the translation direction (EN→FR vs. FR→EN).
3. Comprehensive Evaluation Protocol: The paper establishes a robust protocol combining automatic metrics, deep linguistic feature analysis, and human judgment, providing a template for future CLTS research.
4. Trade-off Quantification: It explicitly quantifies the trade-off between fidelity (keeping the original meaning) and simplicity (reducing complexity), showing that different strategies optimize for different goals.

4. Key Results

Performance by Metric

• Translation Fidelity (BLEU): Direct Prompting consistently achieved the highest BLEU scores across all models and corpora. This suggests that asking the model to perform both tasks simultaneously yields the most fluent and accurate translations, preserving the source structure best.
• Simplification Quality (SARI & Linguistic Features): Sequential approaches, particularly Translate-then-Simplify (T→S), demonstrated superior simplification capabilities.
  • T→S Composition and Decomposition resulted in lower syntactic tree depth, shorter sentences, higher ratios of short sentences, and significantly better Flesch Reading Ease scores compared to Direct Prompting.
  • Simplify-then-Translate (S→T) approaches generally performed worse in terms of BLEU, especially in the French-to-English direction, indicating significant semantic drift when simplifying before translating.
• Semantic Similarity: Direct prompting maintained the highest semantic similarity scores. However, in French-to-English tasks, T→S approaches showed lower BERT scores, suggesting that translating complex French text first, then simplifying, preserves meaning better than simplifying French first.

Human Evaluation Findings

• Simplicity vs. Fidelity Trade-off: Human annotators rated Decomposition approaches as the most "simple" (highest simplicity scores). However, this came at the cost of higher rates of information addition and removal.
• Direct Prompting: Rated as the least simple but the most faithful to the original source text, confirming its strength in meaning preservation.
• Operational Order: The order of operations significantly impacted the final output. "Translate-then-Simplify" generally produced more readable text than "Simplify-then-Translate."

Model Performance

• Proprietary models (GPT-4o, Gemini) generally outperformed open-source models (Aya, Mistral, mT5) across all metrics.
• The fine-tuned mT5 baseline showed competitive SARI scores but significantly lower BLEU and semantic scores compared to the prompted LLMs, highlighting the zero-shot capabilities of modern LLMs.

5. Significance and Implications

• Strategy Selection: The paper concludes that there is no "one-size-fits-all" strategy.
  • If the goal is maximum accuracy and meaning preservation (e.g., legal or medical translation where fidelity is critical), Direct Prompting is the superior choice.
  • If the goal is maximum accessibility and readability (e.g., educational materials for non-native speakers), Translate-then-Simplify (T→S) approaches are recommended, as they yield the most structurally simplified text.
• Prompt Engineering: The study validates that prompt engineering (specifically the ordering of tasks and the use of composition vs. decomposition) is a critical lever for controlling the output characteristics of LLMs in complex, multi-objective tasks.
• Future Directions: The authors highlight the need for research into other language pairs (especially low-resource or typologically distant languages) and the development of metrics that better capture cultural appropriateness and actual reader comprehension.

In summary, this paper provides a foundational analysis of how to best leverage LLMs for Cross-Lingual Text Simplification, demonstrating that the choice of prompting strategy fundamentally alters the balance between translation accuracy and text simplification.