💬 NLP

From Data to Device: ELMOD An Efficient German-First 2.7B Language Model for Mobile Inference

ELMOD is a compact 2.7B German language model designed for efficient on-device inference that, through specialized data preprocessing and quality filtering, achieves performance comparable to 7B-parameter models while operating under a limited computational budget.

Original authors: Darina Gold, Alexander Schwirjow, Viktor Haag, Viktor Hangya, Joel Schlotthauer, Fabian Küch, Luzian Hahn

Published 2026-07-28

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Darina Gold, Alexander Schwirjow, Viktor Haag, Viktor Hangya, Joel Schlotthauer, Fabian Küch, Luzian Hahn

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the world of artificial intelligence as a massive, bustling library where giant robots are trying to learn how to speak every language in existence. For a long time, these robots needed to live in huge, super-cooled data centers far away, connected to the internet like a lifeline. They were like brilliant scholars who could only think if they were plugged into a massive power grid. But what if you wanted a scholar who could live right in your pocket, on your phone, working even when you're in a cave with no signal? This is the dream of "on-device" AI. To make this happen, scientists are trying to shrink these giant brains down to fit into small spaces without losing their smarts. The big challenge is that smaller brains usually mean dumber robots, unless you can teach them incredibly well using very specific, high-quality lessons.

Enter ELMOD, a new project by researchers at the Fraunhofer Institute who decided to build a tiny, super-smart language model specifically for the German language. Think of language models as students who learn by reading billions of books. Usually, to get a student to be a genius, you throw a mountain of books at them. But ELMOD's creators wanted to see if they could make a student just as smart by being much more careful about which books they read. They didn't just grab random pages from the internet; they acted like strict librarians, filtering out the noise, fixing the grammar, and even rewriting boring parts to make them more educational. Their goal was to create a 2.7-billion-parameter model (a measure of its brain size) that could run efficiently on mobile devices, proving that you don't need a supercomputer to have a smart German-speaking assistant.

The Recipe for a Pocket-Sized Genius

The story of ELMOD begins with a simple question: Can we build a German-speaking AI that is small enough to fit on a phone but smart enough to compete with much larger models? The researchers started with a massive pile of data, like a chaotic junkyard of text from the internet. They had 4.83 trillion words to work with, but most of it was junk—ads, broken links, and repetitive nonsense.

Step 1: The Great Cleanup
First, they had to clean the data. Imagine trying to cook a gourmet meal, but your ingredients are mixed with rocks and plastic wrappers. The team used a series of filters to remove the "rocks." They threw out pages that were just lists of numbers, removed text that was mostly in English when they wanted German, and even filtered out words that were too rude or inappropriate. They also used a clever trick called "deduplication," which is like finding and removing the exact same recipe card that someone pasted into the cookbook a thousand times. This process was crucial because it saved them from wasting time and energy reading the same boring stuff over and over.

Step 2: The Quality Check
Once the junk was gone, they faced a new problem: not all the remaining books were good for learning. Some were just news headlines, others were deep scientific articles. The researchers wanted their AI to learn from the "best" books. They used a smart AI judge (a larger model) to grade the text on a scale of 0 to 5, based on how educational it was. They found that training on the highest-quality books (score 2 and above) made the model smarter, but going even higher (score 3 and above) didn't provide any extra boost. However, here's the clever part: they realized that even books with a "medium" score (around 1.5 to 2) could be upgraded.

Step 3: The Rewrite Magic
This is where the magic happened. The team took the "medium-quality" texts and asked another AI to rewrite them. They told the AI: "Take this boring blog post and rewrite it like it's a chapter from a textbook for experts, or a fun story for children, or a professional blog." By doing this, they turned average data into high-quality lessons. It was like taking a rough draft of a story and polishing it until it shined. They generated about 73 billion new tokens (chunks of text) this way, effectively upgrading their library without needing to find new books.

Step 4: The Training
With their cleaned and upgraded library ready, they started training the model. They had a strict budget: they could only use 55,000 hours of powerful H100 GPUs (the engines that power AI training). To make the most of this, they experimented with how the model learned. They tried different ways of handling numbers and different mixtures of German, English, and code. They found that a specific mix—50% English, 40% German, and 10% code—worked best. They also discovered that slowing down the learning process at the very end (a technique called "annealing") helped the model settle into its knowledge better, much like how a student reviews notes calmly before a big test.

The Results: Small but Mighty

When they finished, they had ELMOD-2.7B, a model with 2.7 billion parameters. To put that in perspective, it's tiny compared to the giants that usually dominate the field. But when they put it to the test on German language challenges, the results were surprising.

ELMOD didn't just do well; it was the strongest performer in its size class (under 3 billion parameters) and performed on par with models that are nearly three times bigger (7 billion parameters) specifically on German benchmarks. It was like a compact sports car that could race as fast as a massive truck. The researchers tested it on various tasks, from answering tricky logic questions to understanding complex stories, and it held its own against much larger models that had been trained on double the amount of data.

They also made sure the model was safe and ready for real life. They scrubbed the training data of personal information like email addresses and credit card numbers, ensuring the AI wouldn't accidentally memorize private details. Finally, they proved it could actually run on a phone. They built a demo app that ran the model on different Android phones and even on a MacBook Pro, showing that it could process text quickly enough to be useful for a human user, even without a supercomputer nearby.

Why This Matters

The paper shows that you don't need to be a giant tech company with unlimited money to build a great AI. By being smart about data quality—filtering out the noise, upgrading the average stuff, and being careful about how you teach the model—you can build a powerful tool that fits in your pocket. ELMOD proves that for the German language, a small, efficient model can be just as capable as the big, expensive ones, opening the door for privacy-friendly, offline AI assistants that work anywhere, anytime.

Technical Summary: ELMOD – An Efficient German-First 2.7B Language Model for Mobile Inference

Problem Statement

The scaling of Large Language Models (LLMs) has yielded significant performance gains but has created a reliance on remote servers, persistent connectivity, and energy-intensive infrastructure. This limits reliability, accessibility, and user privacy in real-world deployments, particularly for edge and mobile devices. Furthermore, major LLM providers often treat non-English languages, such as German, as secondary by-products, failing to prioritize local deployment or meet specific European regulatory requirements regarding transparency. There is a lack of comprehensive, step-by-step methodologies for building efficient, high-quality language models from scratch in low-resource settings for non-English languages.

Methodology

The authors present ELMOD (Efficient Language Model for On-Device Deployment), a 2.7-billion parameter model trained on a limited computational budget of 55,000 H100 GPU hours using exclusively publicly available data. The development process is divided into three principal phases: Data Preparation, Pre-training, and Post-training.

1. Data Preparation

The data pipeline was designed to maximize compute efficiency and data quality, specifically addressing German linguistic characteristics (morphological variation, compounding, orthography).

Data Sources: The dataset comprises 45% German text, 45% English text, and 10% code. Sources include CodeParrot (GitHub files), Common Crawl (German raw dumps), and DCLM (English data), covering up to March 2025.
Filtering Pipeline: A rule- and gazetteer-based approach was used to minimize computational cost. Steps included:
- Language & Line Filters: Removed non-German content and low-information lines (e.g., repetitive text, excessive numbers).
- Document Filters: Enforced minimum word counts, valid word fractions, and stop-word patterns while removing punctuation-heavy content.
- URL & Profanity Filters: Excluded harmful categories and texts exceeding a specific profanity ratio (0.005).
- Deduplication: Applied Bloom Filters and MinHash to eliminate redundant documents.
Quality Bucketing & Synthesis:
- An "LLM-as-a-Judge" (Mistral-Small-24B) annotated a subset of data on an educational quality scale (0–5).
- A regression model (FineWebEdu German or FWe-de) was trained to score the entire corpus.
- Rephrasing: Data scoring between 1.5 and 2.0 was synthetically rephrased using Qwen3-8B into three domains (expert textbooks, children's textbooks, blogs) to elevate quality. This generated 73.33B additional tokens.
Anonymization: The pipeline explicitly anonymized PII (emails, phone numbers, IBANs, credit cards) using Microsoft's Presidio, a step the authors note is rarely detailed in model training pipelines.

2. Pre-training

Architecture: A 32-layer transformer with 32 attention heads (using Grouped-Query Attention), 2560 embedding size, and GELU activation. The model uses AdamW optimization and $\mu$ P scaling.
Tokenizer Optimization: Extensive experiments (66 configurations) determined the optimal tokenizer for German. The final choice utilized:
- Digit Handling: 3-digit left-to-right representation.
- Vocabulary Size: 65,000 tokens.
- Data Mix: 50% English, 40% German, 10% Code.
Learning Rate Strategy: The authors compared cosine schedules against polynomial schedules. They found that a polynomial schedule with an explicit annealing phase (stable rate for 180B tokens, followed by linear annealing over the final 20B tokens) outperformed standard cosine schedules, particularly on German tasks.

3. Post-training

The authors performed a standard instruction fine-tuning (SFT) using the Sigma dataset (synthesized via self-instruct), supplemented with SystemChat2.0, GSM8K, and WinoGrande. This phase was intentionally minimal to benchmark the approach rather than exhaustively optimize the instruction-tuned model.

Key Results

Performance vs. Size: ELMOD-2.7B achieves the best performance within its size class (<3B parameters).
Performance vs. Larger Models: On German benchmarks (German versions of ARC, HellaSwag, and MMLU), ELMOD-2.7B matches the performance of significantly larger 7B-parameter models (e.g., Llämmlein-7B) despite being trained on fewer tokens.
Data Quality Impact: Experiments showed that training on data filtered for educational quality (FWe-de $\ge$ 2) yielded measurable improvements over raw filtered data. Rephrased data (FWe-de 1.5–2) performed on par with originally high-quality data (FWe-de $\ge$ 3).
On-Device Deployment: The model was successfully deployed on an Android demo app. Inference speeds ranged from 4.38 TPS (Snapdragon 7s Gen 3, CPU) to 22.7 TPS (NVIDIA Jetson Orin Nano, GPU, quantized), demonstrating viability on resource-constrained hardware.

Significance and Claims

The paper claims to be the first to outline a complete model creation process—from data collection to on-device deployment—specifically for a German language model. Its primary contributions are:

Efficiency: Demonstrating that a 2.7B model can match 7B model performance in German through careful data curation and architectural choices, rather than brute-force scaling.
Transparency: Providing a detailed, reproducible pipeline for German-specific data preprocessing, including quality filtering, rephrasing, and explicit anonymization.
Regulatory Compliance: Addressing EU-AI Act criteria for transparency and local deployment, which the authors argue are currently unmet by other sub-3B models.
Resource Constraints: Proving that high-quality German language models can be built within a constrained compute budget (55k H100 hours) without relying on proprietary data.

The authors maintain a modest tone regarding the post-training phase, acknowledging that the instruction-tuned model did not surpass all competitors in the <3B class, likely due to the limited scope of the fine-tuning optimization. They also note that the model's performance in languages other than German and English is likely limited.