Technical Summary: The Price of Reasoning in RLVR for Neural Machine Translation

Problem Statement

Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a superior training paradigm for Large Language Models (LLMs) in Neural Machine Translation (NMT), particularly for legal documents where reasoning capabilities are critical. However, a practical tension exists: while reasoning-enhanced models often yield higher translation quality, they generate significantly more output tokens, leading to increased inference latency and computational costs.

The core question addressed is whether the inclusion of explicit reasoning traces (thinking processes) is genuinely necessary during both the training and inference phases. Specifically, the paper investigates the cost-quality tradeoff: does the marginal gain in translation quality justify the substantial increase in token generation and inference costs? Furthermore, it explores whether mismatched configurations—such as training with reasoning but inferring without it, or vice versa—result in suboptimal performance or wasted resources.

Methodology

The authors conducted a systematic study using the SwiLTra-Bench dataset, a Swiss legal translation benchmark. The experimental design involved fine-tuning Qwen3.5 models (4B and 9B parameter variants) using Group Relative Policy Optimisation (GRPO).

Experimental Configurations

The study manipulated two independent variables:

1. Training Phase: Models were trained either with reasoning traces (requiring the model to output thoughts within <thoughts> tags) or without (enforcing empty thought tags).
2. Inference Phase: Trained models were evaluated by either enabling or disabling the generation of reasoning traces during inference.

This resulted in four distinct configurations for each model size:

• [T:✓, I:✓]: Trained with reasoning, inferred with reasoning.
• [T:✓, I:✗]: Trained with reasoning, inferred without reasoning.
• [T:✗, I:✓]: Trained without reasoning, inferred with reasoning.
• [T:✗, I:✗]: Trained without reasoning, inferred without reasoning.

Evaluation Metrics and Cost Estimation

• Quality: Translation quality was assessed using chrF, METEOR, COMET, and MetricX. The authors prioritized COMET due to its strong correlation with human expert judgment on legal texts.
• Cost: Inference costs were estimated based on execution time and a standardized hourly GPU cost ($0.79/hour), derived from depreciation, power consumption, and maintenance. Training costs were similarly estimated based on the number of training samples (checkpointed every 100 samples) and total execution time.
• Efficiency: The study tracked the number of generated tokens to quantify the overhead introduced by reasoning.

Key Contributions and Findings

1. The Necessity of Consistent Reasoning Configurations

The experiments revealed that mismatched configurations are detrimental:

• Training with Reasoning, Inference without ([T:✓, I:✗]): This configuration resulted in translation quality below the baseline of models trained without reasoning. The model failed to leverage the reasoning capabilities it learned during training when they were suppressed at inference.
• Training without Reasoning, Inference with ([T:✗, I:✓]): While this configuration improved quality over the baseline, it incurred a massive increase in inference costs (2.3x to 2.5x more tokens) without achieving the same quality levels as models trained with reasoning.

2. The Superiority of [T:✓, I:✓]

The configuration where reasoning is enabled in both training and inference ([T:✓, I:✓]) consistently achieved the highest translation quality (e.g., a COMET score of 82.50 for Qwen3.5 9B). Crucially, training with reasoning acts as a "taming" mechanism: it forces the model to generate more compact reasoning traces.

• Models trained with reasoning generated significantly fewer tokens during inference compared to those trained without it but forced to reason at inference time.
• For Qwen3.5 9B, enabling reasoning at inference for a model trained without reasoning increased token count by 230% compared to a model trained with reasoning.

3. Cost-Quality Tradeoffs and the Pareto Frontier

The study mapped the relationship between inference cost and translation quality, identifying a Pareto frontier.

• The [T:✓, I:✓] configuration for the 9B model sits on the frontier, offering the best balance of quality and cost.
• The [T:✗, I:✓] configuration (training without, inferring with) is strictly dominated; it costs significantly more for lower quality than the [T:✓, I:✓] counterpart.
• The authors note that while training costs are a one-time fixed expense, inference costs accumulate over time. Therefore, optimizing for inference efficiency (by training with reasoning to reduce token bloat) is economically superior for long-term deployment.

4. Data Efficiency

The study observed that translation quality plateaus after approximately 1,000 to 1,500 training samples on the SwiLTra-Bench dataset. Beyond this point, additional training data yields diminishing returns that do not justify the disproportionate increase in computational cost.

Significance and Claims

The paper claims that the "Price of Reasoning" is not merely a function of the inference phase but is fundamentally shaped by the training paradigm.

• Reasoning is not just an inference-time artifact: The ability to reason effectively and compactly must be cultivated during training.
• Economic Viability: For legal machine translation, the most economically defensible approach is to train models with reasoning traces and maintain that mode during inference. This approach maximizes quality while minimizing the "token tax" associated with unstructured or untrained reasoning generation.
• Strategic Shift: The findings support a shift in the industry from pursuing ever-larger models toward engineering efficient, measurable deployments where the cost-quality tradeoff is explicitly managed through training-inference consistency.

The authors conclude that for legal domains, limiting training to high-quality samples (approx. 1,500) and maintaining a consistent "thinking" mode between training and inference offers the optimal balance of legal reliability and operational cost.