Cross-Preference Learning for Sentence-Level and Context-Aware Machine Translation

1. Problem Statement

Current Machine Translation (MT) systems typically operate under two distinct modes: Sentence-Level (Sent) translation (using only the source sentence) and Context-Aware (Ctx) translation (using the source sentence plus preceding/following context).

• The Core Issue: While context-aware translation is theoretically superior, empirical evidence shows it does not consistently outperform sentence-level translation. Contextual signals are unevenly beneficial; for some sentences, context improves quality, while for others, it introduces noise, redundancy, or even degrades performance.
• Limitation of Existing Methods: Standard training objectives (like Maximum Likelihood Estimation) or existing preference optimization methods (like DPO, CPO, SimPO) treat input conditions as fixed. They implicitly assume "more context is always better" or optimize for a single condition, failing to explicitly model when and how context should be utilized. Consequently, models lack the adaptability to switch between or leverage the strengths of both modes effectively.

2. Methodology: Cross-Preference Learning (CPL)

The authors propose Cross-Preference Learning (CPL), a training framework that unifies sentence-level and context-aware translation within a single model by explicitly modeling the preference relationships between them.

A. Input Conditions

The model is trained to handle two input configurations sharing the same parameters ($\theta$):

1. Sentence-Level: Input $x$ (source sentence).
2. Context-Aware: Input $(x, c)$ (source sentence + context).

B. Preference Construction

For each source instance, the model generates candidate translations for both conditions. Preference pairs are constructed using automatic metrics (average of sentence-level COMET and document-level COMET):

• Intra-Condition Pairs: Within a single condition (e.g., comparing two context-aware outputs), a "preferred" ($y^+$) and "dispreferred" ($y^-$) pair is selected.
• Cross-Condition Pairs: This is the novel contribution. For a given source $x$, the best translation among all candidates (from both Sent and Ctx modes) is identified as the "winning" output ($y^+_w$). The other condition's output is the "losing" output ($y_l$).
  • Two types of cross-condition pairs are formed: $(y^+_w, y^+_l)$ and $(y^+_w, y^-_l)$.
  • This forces the model to learn that the winning condition's output is superior regardless of whether it came from a sentence-only or context-rich input.

C. The CPL Objective

The total loss function combines two components:

1. Intra-Condition Loss ($L_{intra}$): Standard Contrastive Preference Optimization (CPO) applied independently to sentence-level and context-aware preference pairs.
2. Cross-Condition Loss ($L_{cross}$): A modified CPO (Cross-CPO) that compares translations generated under different input conditions. It encourages the model to assign higher probability to the winning condition's output over the losing condition's output.

$$L_{CPL} = L_{intra} + L_{cross}$$

Key Technical Feature: CPL operates entirely at the training objective level. It requires no architectural changes to the underlying Large Language Model (LLM) and can be applied to any shared-parameter translation system.

3. Key Contributions

1. Unified Framework: Extends preference-based optimization to jointly support sentence-level and context-aware translation, addressing the variability in context utility.
2. Cross-Condition Preference Optimization: Introduces a mechanism to explicitly model interactions between different input conditions, allowing the model to learn a shared preference structure and adaptively leverage context.
3. Architecture Agnostic: The method improves performance without modifying model architecture, making it applicable to various pre-trained LLMs.
4. Robustness: Demonstrates that the model learns to selectively use context, improving quality when context is helpful and remaining robust when context is noisy.

4. Experimental Results

The approach was evaluated on six translation directions (English to German, Spanish, French, Italian, Dutch, Russian) using three LLM backbones: Qwen3-4B, Qwen3-8B, and Llama-3-8B-Instruct.

• Performance Gains: CPL consistently outperformed all baselines, including:
  • Standard fine-tuned models (Sent-Level, Ctx-Aware).
  • Jointly fine-tuned models (Sent+Ctx).
  • Preference-optimized models trained on single conditions (Sent-Level CPO, Ctx-Aware CPO).
  • Result: CPL achieved significant improvements in COMET scores (e.g., +1.10 average gain for Qwen3-4B) across both sentence-level and context-aware inputs.
• Context Adaptivity:
  • On sentences where context was labeled as "High Necessity," CPL with context input significantly outperformed sentence-level translation.
  • On sentences with "Noisy/Random" context, CPL's performance remained stable and nearly identical to sentence-level translation, proving it does not blindly rely on context.
• Quality Metrics: Beyond COMET, CPL showed improvements in discourse coherence, faithfulness (measured by ALTI+), and fluency (measured by GPT-as-Judge).
• Ablation Studies: Removing either the intra-condition or cross-condition loss degraded performance, confirming that both components are necessary. The cross-condition loss was found to be crucial for stabilizing the model's ability to handle variable context utility.

5. Significance

• Solving the "Context Paradox": The paper addresses the long-standing issue where context-aware MT fails to consistently outperform sentence-level MT. CPL provides a principled way to teach models to be "context-adaptive" rather than "context-dependent."
• Efficiency: By avoiding architectural changes and leveraging existing LLMs with a new training objective, CPL offers a cost-effective path to robust, document-level translation.
• Generalizability: The framework is not limited to translation; the concept of cross-condition preference learning could be applied to other NLP tasks where multiple input modalities or contexts exist with varying utility.

In conclusion, Cross-Preference Learning successfully aligns translation behavior across different input conditions, enabling a single model to dynamically determine whether to rely on context or not, resulting in state-of-the-art translation quality and robustness.