💬 NLP

Robust Language Identification for Romansh Varieties

This paper presents a novel SVM-based language identification system capable of distinguishing between the various regional idioms of Romansh and the supra-regional Rumantsch Grischun, achieving 97% average accuracy on a newly curated benchmark to enable applications like idiom-aware spell checking and machine translation.

Original authors: Charlotte Model, Sina Ahmadi, Jannis Vamvas

Published 2026-03-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Charlotte Model, Sina Ahmadi, Jannis Vamvas

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are walking through a small, mountainous valley in Switzerland. In this valley, people speak a language called Romansh. But here's the twist: it's not just one language. It's more like a family of five cousins who grew up in different valleys. They all speak the same "family language," but they have different accents, slang, and even spell things slightly differently. Sometimes, if a cousin from the north talks to a cousin from the south, they might struggle to understand each other!

There's also a sixth "cousin" called Rumantsch Grischun. This one is a bit special; it's an artificial "super-cousin" created by mixing bits and pieces of the other five to act as a standard written language for official business.

The Problem: The "Who Are You?" Game

For a long time, computers were terrible at telling these cousins apart. If you asked a computer, "What language is this?" it would usually just say, "Oh, it's Romansh," and stop there. It couldn't tell if it was the northern cousin or the southern one.

This is a big problem because if you want to build a spell-checker or a translator for Romansh, you need to know exactly which cousin you are talking to. If you try to spell-check a northern text with a southern dictionary, it will look like gibberish.

The Solution: The "Detective" System

The researchers in this paper (Charlotte, Sina, and Jannis) decided to build a digital detective (a computer program) whose only job is to look at a piece of text and shout out, "I know who you are! You are the Puter cousin!" or "You are the Sursilvan cousin!"

Here is how they did it, using some simple analogies:

1. Gathering the Evidence (The Data)

To train their detective, they didn't just look at one type of book. They gathered a massive pile of clues from:

Dictionaries: Like a giant encyclopedia of words.
Newspapers: Daily news written in the local dialects.
Schoolbooks: Texts used to teach kids.
Radio Scripts: Transcripts from the local radio station.

They cleaned this data up, removing things like HTML tags (the invisible code behind websites) and fixing typos, so the detective could focus on the real language.

2. The Training Method (The "SVM" Coach)

They used a classic machine learning technique called an SVM (Support Vector Machine).

The Analogy: Imagine a coach trying to sort a pile of mixed-up socks. The coach doesn't need to understand why a sock is blue; they just need to find the patterns.
The Trick: The coach didn't look at whole words (like "house" or "cat"). Instead, they looked at tiny fragments of letters (like "th", "ing", or "a.").
- Why? Because the cousins often use the same words but spell them differently. For example, one cousin might write "o." with a dot, while another writes "o" without it. These tiny letter patterns are the "fingerprints" that give the cousins away.

3. The Big Test

They tested their detective on four different scenarios:

Scenario A & B (The Practice Run): They tested it on news and radio scripts it had seen before (or very similar to what it saw).
- Result: The detective was a superstar! It got 97% to 98% of the answers right. It could perfectly distinguish between the cousins.
Scenario C (The Wildcard): They tested it on schoolbooks.
- Result: Still very good, though a few tricky short sentences confused it.
Scenario D (The Curveball): They tested it on rough, informal notes written by journalists before they went on air.
- Result: The detective stumbled a bit (dropping to about 69% accuracy). Why? Because these notes were messy, full of abbreviations, and sometimes mixed with German. It's like trying to identify a person by their handwriting when they are scribbling in a hurry.

The Surprising Discoveries

Names Don't Matter: The researchers tried to hide all the names of people and places (like "Zurich" or "John") to see if the computer was just cheating by memorizing locations. It turned out, the computer didn't need the names at all. It was listening to the sound and spelling of the words.
The "Dot" Clue: They found that for some cousins, a tiny dot under a letter (like o.) was the biggest giveaway. It's like a secret handshake that only that specific cousin uses.

Why Does This Matter?

This isn't just a cool science experiment. It's a lifeline for a language that is at risk of disappearing.

Better Tools: Now, we can build spell-checkers that know exactly which dialect you are typing in.
Smarter Translators: We can route your text to the right translator automatically, so you don't get a translation that sounds like a different language.
Preservation: It proves that even for small, low-resource languages, we can build smart technology to help them survive and thrive in the digital world.

In short: The researchers built a super-smart sorter that can tell the difference between five very similar Swiss dialects and a "super-dialect" with nearly perfect accuracy, using nothing but the tiny patterns in how people spell their words. It's like teaching a computer to recognize a family member's unique laugh, even if they are all whispering in the same room.

1. Problem Definition

The paper addresses the challenge of Language Identification (LID) for Romansh, a low-resource language spoken in the Swiss canton of Grisons. Unlike standard LID tasks that distinguish between unrelated languages, this task involves distinguishing between closely related regional varieties (idioms) that often have limited mutual intelligibility.

The specific classification problem involves identifying one of six varieties:

Five historical idioms: Sursilvan, Sutsilvan, Surmiran, Puter, and Vallader.
One supra-regional standard: Rumantsch Grischun (RG), an artificial standard created in 1982 that combines elements of the other idioms.

Key Challenges:

Linguistic Similarity: The idioms share significant phonological, morphological, and lexical features, making automatic differentiation difficult.
Data Scarcity: As a low-resource language, there is a lack of annotated datasets and existing NLP tools that treat Romansh as a monolithic language rather than distinct varieties.
Domain Variability: The system must handle diverse text types, from formal dictionaries and news articles to informal broadcast notes and school textbooks.

2. Methodology

Data Curation and Preprocessing

The authors curated a novel benchmark dataset by aggregating five distinct sources:

Pledari Grond (PG): A comprehensive Romansh-German dictionary.
La Quotidiana (LQ): A daily newspaper with idiom-annotated articles (2021–2025).
Radiotelevisiun Svizra Rumantscha (RTR): Validated speech transcripts.
RTR Telesguard Notes (TG): Pre-broadcast journalist notes (out-of-domain).
Mediomatix Textbooks (TB): Parallel scholastic material.

Preprocessing Steps:

Removal of duplicates, HTML/markup, and non-letter artifacts.
Named Entity (NE) Masking: To prevent models from relying on memorizing location-specific proper nouns (which are common across regions), the authors created two training variants: unmasked and masked (where NEs are replaced by $NE$ using a fine-tuned SwissBERT model).

Experimental Setup

Task Formulation: Supervised multi-class classification (6 classes).
Model Architecture: The study primarily utilizes Support Vector Machines (SVM), specifically Linear SVM, alongside Logistic Regression and Naïve Bayes as baselines.
Feature Extraction: The models use Bag-of-n-grams features, combining:
- Word unigrams.
- Overlapping character n-grams ( $n \in \{1, 2, 3, 4\}$ ).
- Represented as TF-IDF vectors with sublinear term frequency scaling and L2 normalization.
Data Splits:
- Training: Unbalanced mix of PG, LQ, RTR, and TB.
- Validation: Balanced, in-domain RTR data.
- Testing: Four distinct test sets to evaluate domain generalization and balance:
  - Test-A: In-domain, unbalanced (News).
  - Test-B: In-domain, balanced (Broadcast transcripts).
  - Test-C: In-domain, unbalanced (Textbooks).
  - Test-D: Out-of-domain, unbalanced (Journalist notes).

3. Key Contributions

Novel Benchmark: Creation of the first curated dataset for Romansh LID spanning two domains (news and education) and six varieties, enabling systematic evaluation.
High-Performance Classifier: Development of an SVM-based system achieving 97% average in-domain accuracy.
Qualitative Analysis: A detailed investigation into discriminative features, revealing that character n-grams (specifically orthographic patterns and diacritics) are more effective than word unigrams for this task.
Open Source: The classifier and dataset are publicly available to facilitate further research on low-resource regional languages.

4. Results

Overall Performance

In-Domain Accuracy: The optimized Linear SVM achieved an average accuracy of 97% and a Macro F1 score of 98.1% on the balanced in-domain test set (Test-B).
Out-of-Domain Performance: Performance dropped on the out-of-domain journalist notes (Test-D), with Macro F1 falling to 69.1%, highlighting challenges with informal, noisy text.
Baseline Comparison: The SVM significantly outperformed the majority-class baseline (35.1% accuracy) and other classifiers like Logistic Regression and Naïve Bayes.

Per-Idiom Analysis

Sutsilvan: Achieved perfect classification (100% recall) on the balanced test set.
Puter: Showed the most variance, particularly in the out-of-domain set where recall dropped to 63.2%, often misclassified as Rumantsch Grischun.
Confusion Patterns: Errors were often attributed to data quality issues (e.g., samples containing mostly named entities, German text, or numbers) rather than inherent model failure.

Feature Importance

Character N-grams: These were identified as the most discriminative features.
Orthographic Markers: Specific features included underdotted characters (e.g., o., u.) unique to Puter and Vallader in dictionary data, and whitespace patterns for Rumantsch Grischun.
Named Entities: Masking named entities had a negligible impact on performance (difference $\le 0.7$ F1 points), confirming the model relies on genuine linguistic features rather than location memorization.

5. Significance and Limitations

Significance:

Practical Application: The system enables downstream applications such as idiom-aware spell checking and machine translation routing, addressing a critical gap in Romansh digital services where users previously had to manually select their dialect.
Low-Resource Paradigm: The work demonstrates that effective LID for closely related, low-resource varieties is achievable using classical machine learning (SVM) with carefully engineered character-level features, without requiring massive deep learning datasets.
Model for Others: Provides a replicable framework for language identification in other endangered or regional language contexts.

Limitations:

Class Imbalance: The training data is heavily skewed toward Rumantsch Grischun (due to dictionary data), which may have caused artifacts (e.g., whitespace becoming a top feature).
Generalization: The model struggles with out-of-domain, informal text (journalist notes), suggesting a need for more diverse training data.
Orthographic Variance: Some discriminative features (like specific underdots) are artifacts of dictionary formatting and may not generalize to standard orthography in other contexts.
Modality: The study is restricted to written text; spoken language identification remains unexplored.

In conclusion, this paper establishes a robust baseline for Romansh language identification, proving that with careful data curation and feature engineering, even highly similar, low-resource language varieties can be distinguished with high accuracy.