💬 NLP

Building Large-Scale English-Romanian Literary Translation Resources with Open Models

This paper introduces the TinyFabulist Translation Framework (TF2), a unified pipeline that leverages synthetic data and a two-stage fine-tuning process to produce a cost-effective, open 12B-parameter model for high-quality English-to-Romanian literary translation, alongside the release of large-scale datasets and evaluation tools.

Original authors: Mihai Nadas, Laura Diosan, Andreea Tomescu, Andrei Piscoran

Published 2026-07-29

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Mihai Nadas, Laura Diosan, Andreea Tomescu, Andrei Piscoran

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where stories are the currency of culture, but some languages are like islands with very few bridges connecting them to the rest of the world. This is the challenge of machine translation, a branch of artificial intelligence that tries to teach computers to speak different languages. Usually, these computers learn by reading millions of books and news articles written by humans. But for many languages, especially those spoken by fewer people, there just aren't enough books to teach the computer. It's like trying to learn to cook Italian food when you've never seen a recipe or tasted a tomato.

To solve this, scientists have started using Large Language Models (LLMs). Think of these as super-smart, digital chefs who have tasted almost every dish in the world and can guess what a new recipe should taste like. However, teaching these chefs to cook literature—stories with feelings, jokes, and cultural secrets—is much harder than translating a news report. And doing this for a language like Romanian, which has fewer digital resources than English, is like trying to bake a perfect cake in a kitchen with very limited ingredients and a tight budget. The big question researchers are asking is: Can we build a high-quality library of translated stories without spending a fortune or needing a super-computer?

The Story of the Tiny Fabulist

In this paper, a team of researchers from Romania introduces a new project called TF2 (TinyFabulist Translation Framework). Their goal was to create a massive library of translated fables—short, moral stories like Aesop's fables—moving from English to Romanian. They wanted to see if they could do this using "open" models (free, public AI tools) instead of expensive, private ones, and if they could do it on a shoestring budget.

The Recipe: Building a Library from Scratch

The researchers knew they couldn't just ask humans to translate millions of stories; it would take too long and cost too much. Instead, they built a four-step assembly line to create the data themselves:

The Taste Test (Stage 1): First, they tested 13 different AI models (some free, some paid) to see which one was the best at translating a few sample fables. They didn't just look at whether the words matched; they asked the AI to grade the translations on five things: accuracy, flow, story logic, style, and cultural fit. The winner was a powerful model called GPT-o3, which became their "gold standard" for creating the rest of the library.
The Silver Standard (Stage 2): Using that winning model, they translated 15,000 English fables into Romanian. Even though these were made by a robot, they were so good that the researchers treated them as a "silver standard" (almost as good as gold) to train their own custom models.
The Specialized Training (Stage 3): This is the magic part. They took open-source AI models (specifically versions of Gemma, ranging from small 1-billion-parameter models up to a 12-billion-parameter one) and gave them a special "finishing school." They used a technique called LoRA (Low-Rank Adaptation), which is like giving a general-purpose chef a specific cookbook for fables without having to rebuild the whole kitchen. They trained these models on the 15,000 translated stories.
The Mass Production (Stage 4): Finally, they used their newly trained, specialized models to translate the remaining 3 million fables from the original English collection. The result is a massive new dataset called DS-TF2-EN-RO-3M, containing three million pairs of English and Romanian fables.

The Results: Small Models, Big Surprises

The researchers then put their new, trained models to the test against the big, expensive, private models (like GPT-4 and DeepL) and the original, untrained open models.

The Gap Closed: Their best open model, TF2-12B (the 12-billion-parameter version), performed incredibly well. It scored an average of 4.83 out of 5 on their quality rubric, while the top private model (GPT-o3) scored 4.92. The gap between the free, custom-trained model and the expensive, proprietary giant was tiny—less than 0.1 points.
The Cost Difference: This is where the story gets really exciting. Translating all 3 million fables using the expensive private APIs would have cost between $1,800 and $32,400 (depending on which model you picked). In contrast, their open-source TF2 model did the exact same job for roughly $350. That's a savings of 97% to 99%.
The Small Models: Even their smallest model (1 billion parameters), after training, jumped from a score of 2.02 to 3.75. It wasn't perfect, but it went from "barely understandable" to "quite good," proving that even small, cheap computers can learn to tell stories if they are taught the right way.

What They Found (and What They Didn't)

The paper suggests that you don't need to be a billionaire to build high-quality literary translation tools. By using a smart, step-by-step process and focusing on a specific type of story (fables), they showed that open models can compete with the giants.

However, the authors are careful to note a few things:

It's not perfect yet: While the open models are close, the private models (like GPT-o3) still scored slightly higher, especially in style and cultural nuance. The open models are a great alternative, but they haven't completely "solved" the problem of matching human-level translation quality in every single case.
The "Gold Standard" is Silver: The reference translations they used to train the models were made by another AI, not a human. This means the models are learning to sound like other AIs, not necessarily like human translators. The researchers checked this by having a human expert look at a small sample of the stories, and the human agreed that the AI rankings were generally correct, but they admit that a larger human study is needed to be 100% sure.
Fables are Special: Fables are short and have a clear structure. The researchers suggest that while this method works great for fables, it might be harder to apply to complex novels with deep metaphors or historical dialogue.

The Takeaway

The TF2 project is like showing that you can build a magnificent library using recycled materials and a little bit of ingenuity, rather than needing a vault of gold. They proved that with the right training data and a clever, cost-effective approach, open-source AI can translate literary content for languages like Romanian at a fraction of the usual cost. They released all their code, their 3-million-story dataset, and their trained models to the public, inviting everyone else to build on their work. It's a step toward a future where AI can help preserve and share stories from every culture, regardless of how much money that culture has to spend on technology.

Technical Summary: Building Large-Scale English–Romanian Literary Translation Resources with Open Models

Problem Statement

Literary translation into low-resource languages, specifically Romanian, faces a dual challenge: a scarcity of high-quality parallel data and the difficulty of preserving stylistic nuance, narrative coherence, and cultural context. Existing machine translation (MT) benchmarks (e.g., WMT) predominantly focus on news or informational text, offering limited guidance for creative, narrative-driven tasks. While large language models (LLMs) have enabled synthetic data generation, most existing work focuses on monolingual story generation or back-translation rather than the systematic creation of large-scale bilingual literary datasets. Furthermore, proprietary models optimized for general domains often fail to capture the specific stylistic and cultural requirements of literary translation, while their deployment is often cost-prohibitive for low-resource research communities.

Methodology: The TinyFabulist Translation Framework (TF2)

The authors introduce TF2, a unified, end-to-end framework for dataset creation, fine-tuning, and evaluation in English-to-Romanian literary translation. The pipeline consists of four distinct stages:

Stage 1: Model Benchmarking and Selection:
- The authors evaluated 13 diverse systems (including proprietary APIs like GPT-4.1, Gemini, DeepL, and open-weight models like EuroLLM and Gemma) on a held-out set of English fables.
- Evaluation utilized a five-dimension LLM-based rubric (Accuracy, Fluency, Coherence, Style, Cultural/Pragmatic Adaptation) scored on a 1–5 Likert scale.
- GPT-o3 was selected as the reference translator for generating "silver-standard" parallel data due to its highest average rubric score (4.92). Gemma-3-12B-it was selected as the backbone for open-model fine-tuning due to its superior performance among open baselines (4.43 avg score) and permissive licensing.
Stage 2: Parallel Corpus Creation (DS-TF2-EN-RO-15K):
- Using the selected reference translator (GPT-o3), 15,000 English fables from the existing DS-TF1-EN-3M corpus were translated into Romanian.
- This resulted in a high-quality, machine-generated parallel corpus (DS-TF2-EN-RO-15K) serving as a training resource and a consistent reference set for subsequent evaluation.
Stage 3: Parameter-Efficient Fine-Tuning:
- Open instruction-tuned LLMs (Gemma-3 variants: 1B, 4B, and 12B parameters) were fine-tuned on the 15k parallel dataset using Low-Rank Adaptation (LoRA).
- Adapters were injected into all major projection matrices ($q, k, v, o, gate, up, down$) with rank $r=32$ and scaling $\alpha=32$ .
- The process included training with AdamW, cosine learning rate scheduling, and the generation of 8-bit quantized variants for efficient deployment.
Stage 4: Large-Scale Corpus Generation (DS-TF2-EN-RO-3M):
- The best-performing fine-tuned model (TF2-12B) was used to translate the remaining ~3 million English fables from DS-TF1-EN-3M.
- This produced DS-TF2-EN-RO-3M, a large-scale bilingual corpus of moral fables, released openly for research.

Evaluation Protocol

The paper employs a hybrid evaluation strategy to address the limitations of traditional metrics in literary contexts:

Primary Metric: A multi-dimensional LLM-based rubric (Accuracy, Fluency, Coherence, Style, Cultural Adaptation) implemented via GPT-o3-mini. To mitigate self-preference bias, a cross-family check was performed using Grok-3-mini.
Secondary Metric: Bilingual Evaluation Understudy (BLEU) is reported as a consistency check against the GPT-o3 reference, though the authors explicitly note its limitations in capturing stylistic variation and paraphrasing inherent in literary translation.
Human Validation: A pilot study involving one independent Romanian linguist evaluated 40 fables, finding that automated rankings were directionally consistent with human judgment regarding fluency, entity fidelity, and narrative closure.

Key Results

1. Performance of Fine-Tuned Open Models

Fine-tuning significantly improved the performance of open-weight models, narrowing the gap with proprietary systems:

TF2-12B: Achieved an average rubric score of 4.83 (at temperature $T=0.0$ ), compared to the untuned Gemma-3-12B-it baseline of 4.43. This represents a gap of only 0.09 points to the GPT-o3 reference (4.92).
TF2-4B: Improved from a baseline of 3.81 to 4.74.
TF2-1B: Improved from 2.02 to 3.75.
Quantization: 8-bit quantized variants maintained performance within 0.01–0.03 points of their FP16 counterparts, demonstrating viability for resource-constrained deployment.

2. Cost Efficiency

The paper highlights a dramatic reduction in cost for generating large-scale datasets:

Translating the full 3-million-fable corpus using proprietary APIs (e.g., GPT-4.1, DeepL) was estimated to cost between $2,700 and $32,400 (depending on the model and reasoning depth).
Using the fine-tuned open TF2 model on local hardware, the estimated cost was approximately $350, representing a 97–99% reduction in expenses.

3. Evaluation Robustness

Cross-Family Bias: System rankings remained stable when evaluated by an independent judge (Grok-3-mini), confirming that the performance gains of TF2 models were not artifacts of evaluator bias.
Human Alignment: The pilot human evaluation confirmed that TF2-12B produced more natural phrasing and better narrative closure than the untuned baseline, though GPT-o3 remained the stylistic gold standard.

Significance and Claims

The paper claims that TF2 provides a reproducible, cost-effective pipeline for advancing literary machine translation in low-resource settings. Its primary contributions are:

Resource Creation: The release of two large-scale datasets (DS-TF2-EN-RO-15K and DS-TF2-EN-RO-3M) and three fine-tuned open models (TF2-1B, TF2-4B, TF2-12B).
Methodological Validation: Demonstrating that parameter-efficient fine-tuning (LoRA) on synthetic, silver-standard data can substantially close the quality gap between open and proprietary models for specific literary domains.
Evaluation Framework: Establishing a rubric-based LLM evaluation protocol that prioritizes stylistic and cultural dimensions over simple lexical overlap, while validating its stability through cross-family checks.

The authors maintain a modest stance, acknowledging that their results are specific to the fable genre (which has structured narratives and explicit morals) and that the "ground truth" references are machine-generated rather than human-translated. They position TF2 not as a replacement for professional human translation, but as a practical, accessible solution for generating culturally significant literary content in low-resource languages where such resources are otherwise unavailable.