💬 NLP

A Computational Approach to Language Contact -- A Case Study of Persian

This study investigates how historical language contact influences the intermediate representations of a monolingual Persian language model, revealing that while universal syntactic information remains largely unaffected, morphological features like Case and Gender are significantly shaped by contact-induced structural changes.

Original authors: Ali Basirat, Danial Namazifard, Navid Baradaran Hemmati

Published 2026-01-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ali Basirat, Danial Namazifard, Navid Baradaran Hemmati

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart robot that has only ever read books written in Persian. It has never been taught English, Arabic, Turkish, or Japanese. In fact, it doesn't even know those languages exist.

Now, imagine you hand this robot a sentence written in Turkish. You ask it: "Hey, what language is this? And what are the grammar rules here?"

You might expect the robot to say, "I don't know, I've only read Persian!" But this paper asks a deeper question: Does the robot's internal "brain" (its digital representations) secretly show signs that it understands the history of Persian, even if it was never taught those other languages directly?

Since Persian has been neighbors with, traded with, and influenced by many other languages for centuries, the authors wondered if the robot's "Persian brain" had absorbed some invisible fingerprints of those neighbors.

The Experiment: A Detective Story

The researchers used a tool called ParsBERT (a Persian-only AI model) and fed it sentences from 8 different languages, ranging from Arabic (a major historical neighbor) to Japanese (a stranger with no real contact).

They used two main "detective tools" to peek inside the robot's brain:

The Information Meter: This measures how much the robot knows about a specific feature (like "Is this a noun?" or "Is this word masculine?"). Think of it like checking how loud a radio signal is.
The Spotlight: This tries to find where in the robot's brain that knowledge is hiding. Is it stored in one specific neuron (a single lightbulb), or is it scattered across the whole room like a diffuse glow?

The Big Discovery: The "Universal" vs. The "Specific"

The results revealed a fascinating split in how the robot thinks, which the authors describe as a difference between universal rules and local habits.

1. The Universal Rules (The "Hard Drive")

When the researchers asked the robot about Universal Part-of-Speech (UPOS) tags—basically, "Is this a noun, a verb, or an adjective?"—the robot performed consistently well across all languages.

The Analogy: Imagine the robot has a universal dictionary of "types of words." Whether the word is in Persian, English, or Japanese, the robot knows that a "verb" is a verb.
The Result: The robot didn't care about language contact here. History didn't change how it saw the basic building blocks of sentences. These rules are too deep and universal to be shaken by centuries of borrowing words.

2. The Local Habits (The "Furniture")

When the researchers asked about Morphology (the specific shapes of words, like Case or Gender), the robot's behavior changed drastically based on the language's history with Persian.

The "Case" Test (Who did what to whom?):
- The Problem: Persian doesn't use "cases" (changing word endings to show if someone is the subject or object). It uses word order instead.
- The Result: When the robot looked at languages with complex case systems (like German or Russian, which have many word endings), it was completely lost. It couldn't find the information.
- The Exception: When the robot looked at languages that also rely on word order or simple markers (like English or French), it did much better.
- The Takeaway: The robot only "understands" the grammar of other languages if that grammar looks like Persian's. If the other language uses a system Persian never used (like complex case endings), the robot's brain doesn't have a map for it.
The "Gender" Test (Masculine vs. Feminine):
- The Problem: Persian has no grammatical gender (words aren't "male" or "female").
- The Result: The robot struggled with languages that have complex gender systems (like Russian with three genders). However, it did okay with languages that have simpler gender systems or rely on natural sex (like Arabic or French), likely because Persian has borrowed many words from Arabic, and those borrowed words sometimes kept their "flavor."
- The Takeaway: The robot's understanding of gender is shaped by what it has actually seen in Persian. It can't invent a complex gender system it never learned.

The "Ghost" in the Machine

One of the most interesting findings was how this information was stored.

For languages with no contact (like Japanese), the robot had a few very specific "lightbulbs" that lit up only for Japanese. It was easy to spot.
For languages with heavy contact (like Arabic or Turkish), the information wasn't in a single lightbulb. Instead, it was a diffuse glow spread across the whole brain.
The Metaphor: Think of it like a house. If you invite a stranger (Japanese) in, they sit in a specific chair. If you invite a long-time family friend (Arabic) who has lived there for centuries, they don't sit in one chair; they have touched everything, moved the furniture, and left their scent all over the house. You can't point to one spot and say, "This is the Arabic part." The influence is everywhere, but it's mixed in with the Persian structure.

The Conclusion

The paper concludes that a language model trained on just one language (Persian) does show traces of its history, but only in a very selective way.

It keeps the universal rules (like what a verb is) safe and unchanged.
It adapts the specific rules (like gender or case) only if they match the "shape" of the Persian language.

If a language contact changed the deep structure of Persian (like adding complex case endings), the robot would have learned it. But since Persian mostly borrowed words and kept its own sentence structure, the robot's brain reflects that: it knows the borrowed words, but it still thinks in Persian grammar.

In short: The robot is a Persian speaker who knows a lot of foreign words, but it still thinks in Persian. It hasn't learned to speak the grammar of its neighbors, only to recognize the shapes of words it has borrowed from them.

Technical Summary: A Computational Approach to Language Contact – A Case Study of Persian

Problem Statement
Language contact is a primary driver of structural diversification and change, ranging from lexical borrowing to deep grammatical restructuring. While traditional linguistics has extensively documented these phenomena, particularly in Persian (Farsi)—a language with millennia of contact with Turkic, Arabic, and other families—it remains unclear whether such contact-induced changes are detectable, quantifiable, and interpretable within the internal representations of monolingual neural language models. Specifically, it is unknown if a model trained exclusively on Persian data encodes latent structural traces of its contact languages, or if its representations are solely shaped by the training language's native structure and universal syntactic properties.

Methodology
The study employs a computational framework to probe the intermediate representations of ParsBERT, a monolingual, encoder-only Transformer model trained on approximately 3.9 billion tokens of Persian text. The methodology involves passing sentences from eight contact languages (Arabic, English, French, German, Hindi, Japanese, Russian, and Turkish) into the Persian model and analyzing the resulting embeddings.

The analysis utilizes two primary techniques:

Information-Theoretic Probing: Using Variational Usable Information ( $\hat{IV}$ ), the study quantifies the amount of linguistic information encoded in the model's intermediate layers regarding specific target variables ( $Y$ ), such as language identity, Universal Part-of-Speech (UPOS) tags, CASE, and GENDER. The metric is normalized by the marginal entropy of the target feature to reflect the proportion of uncertainty recoverable from the representation.
Attribution Analysis: Using Language Activation Probability Entropy (LAPE), the study identifies which specific elements (neurons) within the representations contribute most strongly to encoding these features. Low LAPE scores indicate high selectivity (neurons responding preferentially to specific conditions), while high scores suggest uniform or non-discriminative activation.

The dataset consists of the Parallel Universal Dependencies (PUD) treebanks, ensuring cross-linguistic comparability through consistent annotation and aligned sentence structures. The selected test languages vary in their degree and type of contact with Persian (historical, modern, regional, or minimal).

Key Results
The investigation reveals a distinct asymmetry between how universal syntactic categories and morphological features are encoded in the presence of language contact:

Language Identification: ParsBERT encodes substantial information about non-Persian languages despite monolingual training. Information gain peaks after the third layer. Notably, the model is most informative about Arabic, Hindi, and Russian, likely reflecting significant lexical influence or genealogical ties. However, LAPE analysis shows that language-specific signals are rarely localized into compact, selective features; instead, they are distributed across shared representations, except for Japanese, which shows high selectivity likely due to orthographic mismatch.
Universal Syntax (UPOS): The recoverable information for UPOS tags remains largely stable across layers and is insensitive to the degree of historical contact. Neither typological similarity nor contact intensity significantly impacts the encoding of universal syntactic categories, suggesting these representations are robust to contact-induced variation.
Morphological Features (CASE and GENDER):
- CASE: The model exhibits a strong bias toward languages with minimal or residual case marking (e.g., English, French, Japanese), where information gain is high. Conversely, it fails to encode rich inflectional case systems (e.g., German, Russian, Turkish) that are absent in Persian. Arabic shows intermediate performance, likely due to lexical overlap and the limited realization of case in contemporary written Arabic. Hindi's postpositional case is poorly captured due to structural mismatches.
- GENDER: ParsBERT effectively captures cues related to the absence of grammatical gender (Japanese, Turkish) and the marginal realization of gender in pronouns (English). For languages with fully grammaticalized gender systems, performance is lower. Two-gender systems (French, Hindi, Arabic) show moderate recoverability, while three-gender systems show almost none. The model's selectivity is concentrated in features associated with neuter nouns, reflecting the semantic diversity of that category.

Key Contributions

First Computational Study of Persian Contact: This work presents the first computational investigation of Persian language contact at the level of neural representations.
Selective Encoding of Contact: The study provides evidence that monolingual language models encode selective and structurally constrained traces of language contact. Contact effects are more readily reflected in morphological and constructional patterns aligned with the training language (analytic/configurational) rather than in universal syntactic categories.
Methodological Framework: By combining information-theoretic probing with attribution analysis, the paper offers a principled framework for investigating classical questions in contact linguistics using neural models.

Significance and Claims
The paper claims that monolingual language models do not merely reflect the training data but implicitly encode traces of historical language contact, provided those traces align with the structural properties of the training language. The findings suggest that while universal syntactic representations remain robust to contact-induced variation, morphological features are highly sensitive to language-specific structure.

The authors conclude that contact effects in these models are not a result of direct exposure to contact languages but arise from:

Structural features of Persian itself that reflect historical contact.
The statistical imprint of borrowings, calques, and hybrid constructions.
Universal linguistic properties shared across languages.

The study highlights that neural models capture surface-level or semantic alignment (e.g., lexical borrowing, analytic strategies) rather than deep grammatical restructuring (e.g., transfer of inflectional paradigms). This underscores the limits and possibilities of using neural language models as tools for studying language change, indicating that they reflect the "structural constraints" of the training language when processing contact phenomena.

Limitations
The authors acknowledge that the PUD dataset excludes several languages crucial to Persian's history, such as Azerbaijani, Turkmen, and Urdu, which limits the ability to evaluate the strongest cases of sustained bilingualism and structural convergence. Consequently, the analysis is constrained to the languages available in the PUD framework.