💬 NLP

LuxInstruct: A Cross-Lingual Instruction Tuning Dataset For Luxembourgish

This paper introduces LuxInstruct, a high-quality cross-lingual instruction tuning dataset for Luxembourgish created by leveraging aligned data from English, French, and German to avoid machine translation pitfalls, thereby improving both cross-lingual alignment and the model's generative capabilities in this low-resource language.

Original authors: Fred Philippy, Laura Bernardy, Siwen Guo, Jacques Klein, Tegawendé F. Bissyandé

Published 2026-07-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Fred Philippy, Laura Bernardy, Siwen Guo, Jacques Klein, Tegawendé F. Bissyandé

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a super-smart robot how to talk to humans. In the world of computer science, this is called "Instruction Tuning." Think of a Large Language Model (LLM) as a brilliant but shy student who has read almost every book in the library but doesn't quite know how to answer a specific question like, "Write me a poem about a sad toaster." Instruction tuning is the process of showing this student thousands of examples of questions and perfect answers, training them to follow instructions rather than just reciting facts.

However, there's a big problem: most of these training books are written in English, French, or German. For languages spoken by fewer people, like Luxembourgish, the library is almost empty. To fill the shelves, researchers often try to translate English instructions into the local language using a machine translator. But this is like trying to teach someone to cook Italian food by translating a recipe from English word-for-word; the machine might miss the cultural "spice," the local idioms, or the specific way a grandmother would say "simmer gently." The result is a robot that speaks the language but sounds robotic, awkward, or culturally confused. This paper dives into how to fix that for Luxembourgish without relying on those clunky machine translations.

The researchers behind this study, working at the University of Luxembourg and the Luxembourg Institute of Science and Technology, decided to build a brand-new training dataset called LUXINSTRUCT. Instead of taking English instructions and blindly translating them into Luxembourgish, they took a clever, cross-lingual approach. They gathered high-quality, human-written content in Luxembourgish—like articles from Wikipedia, news stories, and dictionary entries—and then asked a powerful AI to write the instructions in English, French, or German, while keeping the answers in perfect, natural Luxembourgish.

Think of it like a cooking class where the teacher speaks English, but the students are learning to cook a traditional Luxembourgish dish. The teacher gives the recipe and the steps in English (the instruction), but the students practice making the dish and describing the taste in their native Luxembourgish (the output). Because the "answers" come from real human sources rather than a machine translator, the language retains its natural flavor, cultural nuances, and correct grammar. The team created over 391,000 of these cross-lingual pairs, plus another 145,000 examples where both the question and answer were in Luxembourgish.

When they tested this new method, the results were surprisingly good. They found that teaching the robot with these mixed-language instructions actually helped it understand Luxembourgish better than if they had just used instructions entirely in Luxembourgish. It was as if the robot learned to connect the dots between the big, well-understood languages (English, French, German) and the smaller one, creating a stronger bridge in its brain. Specifically, using English or French instructions seemed to help the most, while using German instructions sometimes didn't help as much as expected, suggesting that sometimes a little distance between languages helps the learning process.

The paper also showed that this method made the robot much better at following instructions in a "few-shot" setting, which is like giving the robot a few examples of a task before asking it to do a new one. When the robot saw examples written in English or French but had to reply in Luxembourgish, it performed better than when it saw examples entirely in Luxembourgish. This suggests that the cross-lingual approach doesn't just fix the grammar; it helps the model understand the intent of the question more deeply.

In short, the authors suggest that for low-resource languages like Luxembourgish, the best way to teach an AI isn't to force a machine translation on it, but to let it learn from high-quality human content while using other languages as a guide. They didn't claim this solves every problem in the world, and they admitted their dataset is still limited to about seven types of tasks, but they proved that this "cross-lingual" strategy is a powerful, high-quality alternative to the usual machine-translation shortcuts. It's a fresh recipe for teaching robots to speak local languages with a human touch.

Technical Summary: LUXINSTRUCT

Problem Statement

Instruction tuning has proven essential for aligning Large Language Models (LLMs) with human intent, yet this technique faces significant barriers in low-resource languages. For Luxembourgish, a West Germanic language with approximately 400,000 speakers, the scarcity of high-quality instruction-following datasets is a primary bottleneck. Traditional approaches to overcoming this scarcity rely heavily on Machine Translation (MT) of existing English-based instruction datasets. However, the authors argue that MT introduces semantic misalignment, cultural inaccuracies, and unnatural phrasing, which can degrade model performance. Furthermore, manual creation of native instruction datasets is labor-intensive and expensive, often requiring native speakers with domain expertise. Consequently, there is a lack of robust, culturally grounded instruction data for Luxembourgish, limiting the development of effective LLMs for the language.

Methodology

To address these challenges, the authors introduce LUXINSTRUCT, a cross-lingual instruction tuning dataset for Luxembourgish. The core innovation is a construction strategy that avoids generating Luxembourgish text via machine translation. Instead, the dataset leverages aligned data from high-resource languages (English, French, and German) to create instruction-output pairs where the instruction is in a source language and the output is in Luxembourgish.

The dataset is constructed from three primary human-curated sources:

Wikipedia: Using a reverse instruction generation approach, the authors prompted gpt-4.1-mini to extract coherent excerpts from Luxembourgish Wikipedia articles and generate corresponding instructions in English. This method bypasses the need to translate the source text into English first, preserving the original Luxembourgish content. The resulting pairs were filtered via heuristics (length, language consistency, extraction validity). A subset of instructions was subsequently translated to German and French.
News Articles: Leveraging RTL Luxembourg, which publishes parallel content in Luxembourgish, French, and English, the authors used embedding models (text-embedding-3-small) to retrieve semantically similar bilingual article pairs. These were used to create tasks such as generating Luxembourgish headlines from foreign articles (Article-To-Title) and generating hypothetical articles from foreign headlines (Title-To-Article). Additionally, a cross-lingual paraphrasing task (CL-Paraphrase) was derived from the LUXALIGN corpus.
Online Dictionary: A public Luxembourgish dictionary provided lexical entries with translations and examples. This source supported tasks including generating example sentences for specific words (Word-To-Example), simplifying colloquial sentences (Colloquial-To-Standard), and word translation (Word-Translation).

The final dataset comprises 391,551 cross-lingual samples (instructions in English, French, or German; outputs in Luxembourgish) and 145,793 monolingual Luxembourgish samples.

Key Contributions

LUXINSTRUCT Dataset: The first cross-lingual instruction tuning dataset tailored for Luxembourgish, containing nearly 540,000 samples across seven task types.
Cross-Lingual Construction Strategy: A methodology that generates high-quality Luxembourgish outputs by aligning them with instructions in English, French, and German, thereby avoiding the pitfalls of machine-translated Luxembourgish text.
Empirical Validation: The authors provide evidence that cross-lingual instruction tuning improves both representational alignment across languages and the model's generative capabilities in Luxembourgish.

Results

The authors conducted two primary experiments to evaluate the efficacy of LUXINSTRUCT:

Cross-Lingual Alignment: Using Centered Kernel Alignment (CKA) scores on FLORES-200 data, the study measured the alignment between Luxembourgish embedding spaces and those of English, French, and German after fine-tuning. Results indicated that cross-lingual tuning (e.g., English instructions $\to$ Luxembourgish output) was at least as effective, and often more effective, than monolingual tuning (Luxembourgish instructions $\to$ Luxembourgish output). Notably, pairing Luxembourgish with English or French yielded higher alignment gains than pairing it with German, suggesting that distant languages may offer better alignment benefits than closely related ones in this context.
Few-Shot In-Context Learning: The authors evaluated instruction-following performance using Gemma 3 (12B and 27B) models. They compared settings where few-shot examples used instructions in Luxembourgish, English, French, German, or a mixed-language setting. Performance was assessed via G-Eval (using LLM judges) across four dimensions: Clarity, Coherence, Fluency, and Relevance. The results demonstrated that cross-lingual few-shot examples consistently outperformed monolingual Luxembourgish examples, with English and French instruction languages often yielding the highest scores.

Significance

The paper posits that LUXINSTRUCT represents a significant step forward for resource development in Luxembourgish NLP. By prioritizing human-generated seed data and a cross-lingual design, the dataset ensures natural, reliable targets and embeds a culturally grounded understanding of the language within LLMs. The authors claim that this approach avoids the common noise and mistranslations associated with large-scale machine translation, offering a viable path for developing robust LLMs for low-resource languages. The work serves as a foundation for future advancements, suggesting that cross-lingual data curation can directly benefit language development without relying on synthetic translation artifacts.