The Morphological Core of Dungan: A Two-Dialect Finite-State Model and a Multi-Genre Evaluation

Imagine you are a detective trying to solve a mystery, but instead of a crime scene, your crime scene is a language. This paper lives in the world of computational linguistics, a field where scientists build digital tools to understand how human languages work. To solve the mystery, they use a specific type of tool called a finite-state model. Think of this model like a super-organized library card catalog or a train station map. It doesn't try to understand the meaning of a sentence like a human does; instead, it just checks if a word fits a specific pattern. It asks: "Does this word have a stem (the main part) and a tag (a little piece added on the end)?" If the pattern matches, the tool says, "Got it!" If not, it says, "I don't know this one."

Why does anyone care about this? Because for many languages, we have dictionaries and grammar books, but we don't have these digital maps. Without them, computers can't translate, spell-check, or search through texts in those languages effectively. The question this paper tackles is simple but tricky: How much of a language is actually made of these little patterned pieces, and how much is just a giant list of unique words? If a language is mostly unique words, the "map" needs to be huge. If it's mostly patterns, the map can be small and neat.

---

The Mystery of the Dungan Language

Meet Dungan, a language spoken by a community of Central Asian Muslims who moved there over a hundred years ago. It's a cousin of Mandarin Chinese, but with a twist: instead of using Chinese characters, they write it using the Cyrillic alphabet (the same script used for Russian). Because they dropped the characters and the musical tones that usually help tell words apart, Dungan is a bit like a game of "Guess Who?" where many characters look exactly the same on paper.

The authors of this paper, Anton and Sergey, decided to build a digital "map" (a finite-state analyzer) for Dungan to see exactly how its grammar works in real life. They didn't want to invent new rules; they wanted to take the rules already written in old grammar books and turn them into a working machine to measure the language. They built this map for two main dialects: the Gansu variety (the standard, literary version) and the Shaanxi variety.

The Big Discovery: A Tiny Core and a Huge Lexicon

When they ran their map over thousands of sentences from three different types of texts (encyclopedias, folk stories, and religious narratives), they found something surprising.

1. The "Invisible" Grammar
In many languages, words change shape constantly (like "walk" becoming "walked," "walking," "walks"). In Dungan, this happens very rarely. The authors found that in the encyclopedia text, only 9.3% of the words actually had a visible "tag" attached to them. In folk stories, it was 13.4%, and in the religious narrative, it was 27.1%.

• The Analogy: Imagine a city where 90% of the buildings are just plain, unadorned blocks. Only a few have special flags or decorations on top. The "grammar" of Dungan is those few flags. Most of the time, the words just sit there, unchanged.

2. The "Two-Clitic" Ambiguity
Because so few words change, when they do change, it can get confusing. The authors found that 78.1% of the words the tool recognized had only one possible meaning. That's a very clear signal! However, the remaining confusion was almost entirely stuck on just two tiny word-pieces (clitics): -di and -ni.

• The Analogy: Imagine a traffic light that works perfectly 78% of the time. The only time it gets confusing is when the light is stuck on "Yellow" or "Red," and nobody knows if it means "Stop" or "Go." In Dungan, the little piece -di can mean "belonging to" (genitive) or "happening right now" (progressive), and -ni can mean "in a place" (locative) or "will happen soon" (prospective). The computer can't tell the difference without more context, but at least it knows where the confusion is.

3. The "Closed" Core
The most important finding is about how "complete" their map is. They asked: "When the computer fails to recognize a word, is it because the grammar rule is missing, or because the word itself is new?"
They discovered that between 78% and 95% of the words the computer couldn't handle were simply missing from the dictionary. The grammar rules themselves were actually "closed" and complete. The things the model didn't include (like some rare past-tense habits or specific sound changes) accounted for at most 4.5% of the failures.

• The Analogy: Imagine you are trying to identify animals in a zoo. You have a perfect guidebook for how to spot a lion, a tiger, or a bear. If you see an animal you don't know, it's almost certainly because it's a new species you haven't seen before, not because your guidebook is missing a rule about how lions walk. The "frontier" of Dungan isn't the grammar; it's the vocabulary.

What the Numbers Tell Us

The authors were very careful to measure exactly what their tool could do.

• Coverage: When they tested their map on new texts it had never seen before, it successfully recognized 80–85% of the words.
• The Power of Morphology: If they had just used a list of words without any grammar rules, they would have recognized 67.4% of the words. By adding the grammar rules (the "morphology"), they boosted that number to 72.6%. That's a 5.2-point gain. It's a helpful boost, but it proves that the grammar is "thin"—the heavy lifting is done by the list of words, not the rules.
• Accuracy: For the words they did recognize, the tool was very good at finding the right meaning, though it sometimes got stuck on those tricky -di and -ni pieces.

The Conclusion: A Map for the Future

The paper concludes that the "morphological core" of Dungan is compact, productive in only a few categories, and effectively closed. The grammar is simple and finished; the challenge is just learning all the words.

The authors released their "map" (the analyzer), the code, and the tests for anyone to use. They admit their work is a hypothesis based on written books and dictionaries, not a final verdict from native speakers. They even included a "worksheet" for native speakers to come along and correct any mistakes. It's a starting point, a digital foundation built to help computers finally understand this unique, Cyrillic-written language. The open frontier isn't the rules of the game; it's the players themselves.