💬 NLP

Analysis of Numerical Localisation in LLM Translations

This paper extends prior research on numerical translation by evaluating five commodity-compatible large language models, finding that embedding localisation principles directly into the prompt context yields statistically significant improvements in accuracy for translating times, numbers, and dates compared to direct translation or other strategies.

Original authors: Patrizia Kaye

Published 2026-08-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Patrizia Kaye

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to send a postcard to a friend in a different country. You write the date as "07/04/2024." To you, it's July 4th. To your friend in the UK, it's April 7th. If you are sending a recipe, a bank transfer, or a scientific formula, getting the numbers and dates wrong isn't just a typo; it's a disaster. This is the world of localisation. While translation is like swapping words (changing "dog" to "Hund"), localisation is like changing the entire outfit of the information to fit the local culture. It's deciding whether to wear a suit or a tuxedo, or in this case, whether to use a comma or a dot to separate decimals.

Enter Large Language Models (LLMs). Think of these as incredibly smart, super-fast robots that have read almost everything on the internet. They are great at writing stories and translating sentences, but they sometimes stumble when it comes to the strict rules of numbers and dates. They might know that "1,000" means a thousand, but they might forget that in Germany, that same number is written as "1.000." This paper dives into a specific corner of AI science: Can these robot brains not just translate words, but also correctly "dress up" numbers and dates for a new country? It's a crucial question because if an AI gets the localisation wrong, it could turn a safe medical dose into a dangerous one or a small profit into a massive loss.

The Great Number Makeover

In this study, a researcher named Patrizia Kaye put five different AI models to the test. These weren't the massive, supercomputer-sized models that cost millions to run; instead, she picked five smaller, "commodity" models that could run on standard hardware, like a powerful gaming laptop. She wanted to see if these models could handle the tricky task of translating English text into German (and vice versa) while getting the numbers, dates, and times exactly right according to local rules.

To do this, she set up a little experiment. She took 700 sentences containing dates, times, and numbers from real-world sources like European Parliament records and medical documents. Then, she asked the AI models to translate them using four different "strategies" or methods:

Direct Translation: Just asking the AI, "Translate this sentence."
In-Context Learning (ICL): Giving the AI a reference in the prompt, like saying, "Remember, in German, we use a dot for thousands and a comma for decimals."
Chain-of-Thought (CoT): Asking the AI to "think out loud" and explain its steps before giving the answer.
Post-Editing: Letting the AI translate first, then using a separate computer script to hunt down the numbers and fix them manually.

The Surprise Winner

The results were a bit of a plot twist. In a previous study on translating numbers between English and Chinese, researchers found that the "Post-Editing" method (fixing the numbers after the fact) was the best. But when Patrizia tested this on English and German, Post-Editing was actually the worst strategy, often scoring below 30% accuracy. It seems that when the AI gets confused, trying to fix the numbers afterward just makes a mess.

The "Chain-of-Thought" method, where the AI explains its reasoning, also didn't help much. It often led the models to give wrong answers while trying to sound smart about how they got there.

The real champion turned out to be In-Context Learning. When the researcher simply added a few clear instructions to the prompt—telling the AI exactly how to format the numbers and dates—the models performed significantly better. For the best-performing model (Qwen3-4B-Instruct-2507), this simple "reference" boosted accuracy to 91.7% for numbers in the English-to-German direction, a slight improvement over the 92.2% achieved by direct translation alone. The study found that this improvement wasn't just a lucky guess; statistical tests confirmed that adding these rules to the prompt made a real, measurable difference compared to other strategies.

What About Bigger Brains?

You might think that a bigger AI model with more "brain power" (parameters) would always do a better job. However, the paper found that size doesn't always equal success. The models tested ranged from 2 billion to 7 billion parameters. Surprisingly, the 4-billion-parameter model actually outperformed the 7-billion-parameter models in many cases. It suggests that for this specific task, having the right instructions is more important than having the biggest brain.

The study also tested whether telling the AI exactly which version of English or German to use (like "British English" vs. "German from Germany") helped. It did help a little bit, but not as much as simply giving the formatting rules in the prompt.

The Bottom Line

This paper suggests that if you want an AI to handle numbers and dates correctly when translating, don't just ask it to "do its best." Don't rely on it to fix its own mistakes later, and don't force it to explain its thinking. Instead, give it a clear set of rules right at the start. By embedding the localisation principles directly into the conversation, you can turn a confused robot into a precise translator. While the results are specific to the models and language pairs tested, the method offers a reproducible way to make AI translations safer and more accurate for anyone dealing with money, science, or schedules across borders.

Technical Summary: Analysis of Numerical Localisation in LLM Translations

Problem Statement
While Large Language Models (LLMs) have become the dominant method for human language translation, their ability to handle "localisation"—the adaptation of formatting conventions for dates, times, and numbers to specific regions—remains under-studied. Unlike programming languages, natural languages vary by culture and region; for instance, the decimal and thousands separators differ between English (UK) and German (e.g., 123,456 vs. 123.456). Improper localisation can result in errors of magnitude, rendering economic or scientific translations nonsensical. Previous research by Tang et al. focused on numerical translation (English $\leftrightarrow$ Chinese) and found that post-editing strategies were most effective. This paper investigates whether those findings hold for numerical localisation (English $\leftrightarrow$ German) and tests strategies to improve accuracy on commodity hardware.

Methodology
The study evaluated five LLMs capable of running on commodity hardware (8GB VRAM, 64GB system memory), comprising two generic models (Qwen2-7B, Qwen3-4B) and three translation-specific models (TowerInstruct-7B, SalamandraTA-7b, SalamandraTA-2b). The experiments utilized three parallel corpora (Statmt-europarl, Tilde-ema, Statmt-news_commentary) to extract 100 samples each of times, dates, and numbers, totaling 700 inputs per model.

Four strategies were tested:

Direct Translation: Baseline instruction to translate without specific localisation guidance.
In-Context Learning (ICL): Providing the model with explicit localisation principles (e.g., digit and decimal separators) within the prompt context.
Chain-of-Thought (CoT): Instructing the model to explain its translation process.
Post-Editing (PE): Extracting numeric pairs from direct translations using regular expressions, re-translating them via a dateparser module, and reinserting them.

Experiments were conducted with both unspecified and explicitly defined locales (en_GB and de_DE). Accuracy was measured by comparing model outputs against authoritative translations using the dateparser and babel libraries, awarding points for correct values and correct formatting. Statistical significance was determined using Wilcoxon tests.

Key Contributions

Empirical Analysis: A systematic evaluation of English $\leftrightarrow$ German numerical localisation across two generic and three translation-specific LLMs.
Strategy Determination: Identification of the most accurate method for ensuring localisation accuracy, challenging previous findings regarding numerical translation.
Reproducibility: Provision of a reproducible methodology, including scripts and code, applicable to additional language pairs and future models.

Results
Contrary to the findings of Tang et al. regarding numerical translation, In-Context Learning (ICL) proved to be the most effective strategy for numerical localisation.

Performance: The Qwen3-4B-Instruct-2507 model achieved the highest accuracy across all data types and directions.
Strategy Comparison: ICL yielded the highest mean accuracy (53.3% for unspecified locales, 53.8% for explicit locales), significantly outperforming Direct translation (approx. 50%), Chain-of-Thought (approx. 40%), and Post-Editing (approx. 27-28%).
Statistical Significance: Wilcoxon tests confirmed that the improvement of ICL over other strategies was statistically significant ( $\alpha = 8.33 \times 10^{-3}$ ).
Model Variance: Parameter count did not correlate linearly with performance; for instance, the 7B SalamandraTA model performed worse than the 2B variant, and both struggled significantly with Post-Editing, often hallucinating unrelated text.
Locale Specification: Explicitly specifying the locale (e.g., "German (de_DE)") had a small but measurable impact, improving accuracy in 66/120 cases by a mean of 4.2%, though it caused a mean loss of 3.2% in others.

Significance and Claims
The paper claims that embedding localisation principles directly into the prompt context (ICL) provides a statistically significant improvement in accuracy compared to direct translation or alternative strategies like Post-Editing. This suggests that for localisation tasks—which rely on applying specific formatting rules—explicitly providing those rules to the model is more effective than relying on the model's implicit knowledge or attempting to correct outputs post-hoc. The author notes that while Post-Editing was superior for numerical translation (a task of conversion), it is less effective for localisation (a task of rule application). The study concludes that while LLMs are powerful, their localisation capabilities require specific prompting strategies to ensure accuracy, particularly when operating under hardware constraints that limit model size.

The Great Number Makeover

The Surprise Winner

What About Bigger Brains?

The Bottom Line

More like this