💬 NLP

"Intelegi Româneste?'' A Recipe for Romanian Vision-Language Models

This paper presents a systematic study on building Romanian-specific Vision-Language Models by translating English corpora, curating the culturally grounded HoraVQA benchmark, and demonstrating that models adapted with Romanian language backbones outperform larger multilingual counterparts across all evaluated tasks.

Original authors: Mihai Masala, Marius Leordeanu, Mihai Dascalu, Traian Rebedea

Published 2026-06-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mihai Masala, Marius Leordeanu, Mihai Dascalu, Traian Rebedea

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant, multilingual robot chef. This chef is famous for cooking amazing meals using English recipes (English text) and looking at pictures of food (images). But if you ask the chef to cook a traditional Romanian dish using a Romanian recipe, they might get confused, burn the food, or just serve you a generic American burger instead.

This paper, titled "A Recipe for Romanian Vision-Language Models," is essentially a guide on how to teach that robot chef to cook authentic Romanian cuisine. The authors, researchers from Romania, didn't just tell the chef "try harder"; they built a whole new kitchen, translated the recipes, and even changed the pictures on the menu to match the local culture.

Here is the breakdown of their work in simple terms:

1. The Problem: The "English-Only" Kitchen

Most advanced AI models (called Vision-Language Models) are like chefs who only know English. They are great at describing a picture of a dog in English, but if you show them a picture of a Romanian street sign or a traditional folk dance, they often fail. They might misread the text on the sign or not understand the cultural context of the dance.

The researchers found that simply translating the English questions into Romanian wasn't enough. The AI still struggled because it hadn't "seen" enough Romanian things or learned how to read Romanian text inside images (like on a bus ticket or a menu).

2. The Solution: A Complete Romanian Makeover

The team built a specialized version of these AI models, which they call RoVLM. To do this, they followed a strict "recipe":

Translating the Menu (Data): They took thousands of existing English image-and-text datasets and translated them into Romanian. But they didn't just translate the words; they used special tools to translate the text inside the images too. If a picture had a sign saying "Open," they edited the image to say "Deschis."
Adding Local Flavor (HoraVQA): They realized that translating English tests wasn't fair. So, they created a brand new test called HoraVQA. Think of this as a "Romanian Culture Quiz." Instead of asking about generic things, they asked questions about Romanian landmarks, traditional food, folk customs, and local history. They hired native speakers to write these questions and pick the photos, ensuring the test was truly grounded in Romanian life.
The "OCR" Challenge: A big part of the recipe was teaching the AI to read text inside pictures (OCR). They found that standard AI "eyes" (vision backbones) were too good at understanding the general vibe of a picture but bad at reading the tiny letters on a sign. They discovered that using a specific type of "eye" (called CLIP) was much better at reading Romanian text than the newer, more "multilingual" eyes.

3. The Results: Small Chef, Big Appetite

The most surprising part of their findings is that specialization beats size.

Usually, in AI, bigger is better. A giant model with billions of parameters is expected to beat a smaller one. However, the authors found that their smaller, Romanian-specialized models performed better than:

The original English-only versions of the same size.
Even some models that were twice as big but not specialized for Romanian.

It's like a small, local Romanian grandmother who knows exactly how to make mămăligă (polenta) perfectly, beating a giant, high-tech international food factory that tries to make it but gets the recipe wrong.

4. Key Lessons Learned

The paper highlights three main things that made their "Romanian Chef" successful:

The Language Backbone: Changing the "brain" of the AI to a Romanian-specific one helped a little, but it wasn't the magic ingredient.
The Vision Backbone: Choosing the right "eyes" (specifically CLIP) was crucial for reading text in images.
The Data Mix: The most important factor was the data. They needed a mix of translated data, real Romanian documents (like PDFs), and synthetic charts. Without this specific "diet" of data, the model couldn't learn to read or understand Romanian culture.

Summary

In short, this paper proves that you can't just take a general AI and expect it to work perfectly in a low-resource language like Romanian. You have to feed it specific, culturally relevant food (data), teach it to read local signs (OCR), and test it on local culture (HoraVQA). When you do that, a smaller, specialized AI can outperform massive, generic giants.

The authors have released their "recipe" (code, data, and models) to the public, so anyone can try to cook up their own Romanian AI chef.

Technical Summary: "Înțelegi românește? A Recipe for Romanian Vision-Language Models"

Problem Statement

Vision-Language Models (VLMs) have achieved significant success on English benchmarks but suffer from sharp performance degradation in low-resource languages. This gap stems from a lack of large-scale, culturally grounded image-text corpora and evaluation frameworks for languages like Romanian. Existing multilingual VLMs often rely on post-hoc machine translation of English-centric datasets, which fails to capture localized visual semantics, cultural context, and the nuances of in-image text (OCR). Consequently, it remains unclear how effectively multilingual capabilities transfer to grounded multimodal understanding in these settings, particularly for tasks involving document analysis, cultural reasoning, and text recognition within images.

Methodology

The authors present a systematic study of building a language-specific VLM for Romanian, covering the full pipeline from data construction to architectural ablation.

1. Data Construction and Adaptation

The authors constructed a training corpus of approximately 3.17 million samples by adapting 11 established English datasets (e.g., LAION, LLaVA-Mix, PixMo, CoSyn, FinePDFs).

Translation Strategy: Unlike prior works that only translated textual annotations, this approach involves "translating" the visual content. This includes extracting text from images, translating it, and regenerating the image with the translated text while preserving layout, font, and size.
Native Data: The pipeline incorporates native Romanian data, specifically the FinePDFs dataset for OCR and synthetically regenerated charts from CoSyn where source code was available.
Translation Models: A comparative study selected GPT-4.1-mini for benchmark translation and Seed-X-PPO for training data translation based on performance on the MMMU dataset.
Quality Control: Human annotators validated translations using length-difference filters (flagging translations >1.5x the original length) and random sampling.

2. Native Evaluation Benchmark: HoraVQA

To address the lack of culturally grounded evaluation, the authors introduced HoraVQA, a benchmark containing 580 question-answer pairs across 232 unique images.

Content: Images depict Romanian cultural concepts (landmarks, cuisine, customs, history) sourced from Wikimedia Commons.
Annotation: Questions were authored natively in Romanian by seven volunteers, categorized into multiple-choice and open-ended formats.
Cultural Grounding: Each question is tagged with a is_cultural flag to distinguish between generic visual perception and knowledge requiring Romanian cultural, historical, or linguistic context.

3. Model Training and Architecture

The study utilizes a modular VLM architecture (vision encoder + language backbone + projection adapter), primarily based on LLaVA-NeXT, but also testing Qwen2-VL, Qwen2.5-VL, Qwen3-VL, and Gemma3.

Training Protocol: Models undergo supervised fine-tuning (SFT) on the Romanian-centric multimodal data. The vision backbone is kept frozen, while the adapter and language backbone are trainable.
Hyperparameters: Training uses a single epoch, AdamW optimizer, cosine learning rate scheduler (peak $1.0 \times 10^{-5}$ ), and a batch size of 64.

4. Ablation Studies

The authors systematically isolated the contributions of:

Language Backbones: Comparing multilingual LLMs against Romanian-adapted LLMs (e.g., RoLlama3).
Vision Backbones: Evaluating CLIP, SigLIP, and SigLIP2 encoders.
Data Composition: Analyzing the impact of OCR-rich data and translated in-image text.

Key Contributions

RoVLM Models: A suite of Romanian-adapted VLMs released with a fully open, reproducible training and evaluation pipeline.
Comprehensive Evaluation Suite: A benchmark suite of 19 tasks spanning OCR, visual question answering (VQA), captioning, and reasoning, including the novel HoraVQA cultural benchmark.
Systematic Analysis of Adaptation: A detailed investigation into how data composition (specifically OCR and translated in-image text) and architectural choices affect low-resource multimodal performance.
Translation of Visual Content: A novel pipeline for translating and regenerating in-image text to preserve visual grounding while adapting linguistic content.

Results

Performance Gains: Romanian-adapted VLMs (RoVLMs) consistently outperform their English-only counterparts across all 19 benchmarks. Notably, RoVLMs often surpass models from the next larger size category (e.g., a 3B Romanian-adapted model outperforming a 7B English-only model).
Architecture Generalization: The performance improvements generalize across different vision encoders and language backbones (Qwen, Gemma, Llama).
Vision Backbone Findings: Contrary to trends in multilingual pretraining where SigLIP2 excels, CLIP outperformed SigLIP and SigLIP2 on OCR and document tasks. The authors attribute this to CLIP's preservation of high-frequency visual structures necessary for character transcription, whereas SigLIP variants optimize for semantic abstraction which can suppress fine-grained details.
Language Backbone Findings: Switching to a Romanian-adapted language backbone (RoLlama3) yielded negligible improvements over the standard multilingual Llama3, suggesting that the 3.1M sample training mix provided sufficient capacity for adaptation, and the limitation was not the pre-trained language model but the data composition.
Data Composition: Removing OCR data led to performance drops across all categories, not just OCR-specific ones, indicating that text recognition capabilities are foundational for general multimodal understanding.
Translated Images: Training with images containing translated text (target language) yielded a slight but consistent performance boost (approx. 1.76 points average) over using native English images, with the largest gains observed in grounding tasks.

Significance and Claims

The paper argues that the dominant trajectory of scaling a single generalist VLM across all languages is insufficient for low-resource languages. Instead, specialized, language-targeted adaptation is necessary and effective.

The authors claim that:

Data Composition is Critical: The primary driver of performance is not the language backbone itself, but the composition of the training data—specifically the inclusion of OCR-heavy supervision and translated in-image text.
Representation vs. Grounding: "Multilingual" representation at the pre-training level does not automatically guarantee grounded, OCR-faithful, or culturally aware behavior.
Cultural Benchmarks are Essential: Standard translated benchmarks are insufficient; native cultural benchmarks like HoraVQA are required to evaluate and improve a model's understanding of local visual culture.
Efficiency: A smaller, specialized model trained on curated low-resource data can outperform larger, generalist models, offering a more efficient path for multimodal AI in non-English contexts.

The work concludes that while translation-based adaptation is a viable strategy, it must be combined with native data, rigorous visual translation, and culturally specific evaluation to achieve robust multimodal understanding in low-resource languages.