💬 NLP

Is Cross-Lingual Transfer in Bilingual Models Human-Like? A Study with Overlapping Word Forms in Dutch and English

This study investigates whether bilingual Dutch-English language models replicate human cross-lingual activation patterns during reading, finding that while shared embeddings can induce facilitation or interference effects, the models' alignment with human processing critically depends on specific vocabulary-sharing configurations and is primarily driven by word frequency rather than form-meaning consistency.

Original authors: Iza Škrjanec, Irene Elisabeth Winther, Vera Demberg, Stefan L. Frank

Published 2026-04-09

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Iza Škrjanec, Irene Elisabeth Winther, Vera Demberg, Stefan L. Frank

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Question: Do AI Brains Read Like Human Brains?

Imagine you are a bilingual person who speaks both Dutch and English. When you read a word that looks the same in both languages, your brain gets a little "boost" or a little "glitch."

The Boost (Cognates/Friends): If you see the word "winter," it means the same thing in both languages. Your brain says, "Oh, I know this! It's easy!" You read it faster.
The Glitch (False Friends): If you see the word "brand," it means "fire" in Dutch but "to want" in English. Your brain gets confused. "Wait, is it fire or desire?" This usually slows you down or causes a moment of hesitation.

The Study: The researchers asked: Do AI language models (like the ones powering chatbots) behave like human bilinguals when they read these words? To find out, they built a special AI that speaks both Dutch and English and tested it under four different "rulebooks."

The Four Rulebooks (Vocabulary Conditions)

Think of the AI's vocabulary as a giant dictionary. The researchers changed how this dictionary was organized to see how it affected the AI's "brain."

The "One-Size-Fits-All" Dictionary (Full Overlap):
- The Rule: If a word looks the same in Dutch and English (like "winter" or "brand"), the AI uses one single entry for it. It doesn't care which language you are speaking; it's just one word.
- The Analogy: Imagine a library where the book Harry Potter is on the shelf. If you ask for it in English or Dutch, the librarian hands you the exact same physical book, regardless of the language you spoke.
The "True Friends Only" Dictionary (Friends Overlap):
- The Rule: The AI only shares entries for words that mean the same thing (True Friends like "winter"). For words that look the same but mean different things (False Friends like "brand"), it keeps two separate entries.
- The Analogy: The librarian shares the Harry Potter book (same meaning) but keeps two different books for "Brand" (one for fire, one for desire) because they are totally different stories.
The "False Friends Only" Dictionary (False Friends Overlap):
- The Rule: The AI shares entries only for the confusing words (False Friends) and keeps separate entries for the helpful ones.
- The Analogy: The librarian thinks, "Let's merge the confusing books together and keep the easy ones separate." (This is a weird rule, but they had to test it!)
The "Strictly Separate" Dictionary (Minimal Overlap):
- The Rule: The AI treats Dutch and English as completely different languages. Even if a word looks the same, it gets two different entries.
- The Analogy: The librarian has two completely separate libraries. You can't borrow a book from the English section if you are in the Dutch section, even if the titles are identical.

What Happened? (The Results)

The researchers measured how "surprised" the AI was by a word.

Low Surprise = Easy processing (The AI predicted it easily).
High Surprise = Hard processing (The AI was confused).

Here is what they found:

1. The AI is usually a "Language Purist"
Unless the researchers forced them to share, the AI kept Dutch and English completely separate in its brain. It didn't naturally "activate" the other language like humans do.

2. The "Shared Entry" Effect
When the AI was forced to share a single entry for a word (whether it was a True Friend or a False Friend), it got faster at predicting that word.

Why? It wasn't because the AI understood the meaning was shared. It was simply because the word appeared more often in the training data.
The Analogy: Imagine you are practicing a magic trick. If you practice the trick 100 times, you get good at it. If you practice it 200 times (because you practiced it in two different languages), you get even better. The AI got better just because it saw the word more often, not because it understood the nuance.

3. The Frequency Trap
The study found that the AI's "helpfulness" was driven almost entirely by frequency, not by meaning.

For True Friends, the AI got faster because it saw the word often.
For False Friends, the AI also got faster (which is weird, because humans usually get slower/confused). The AI didn't get confused because it didn't really "know" the English meaning of the Dutch word well enough to get tripped up. It just saw the shape of the word often and guessed it.

4. The Winner: The "True Friends Only" Rule
The only condition that made the AI act like a human bilingual was Rule #2 (Friends Overlap).

In this setup, the AI got a "boost" for True Friends (like humans).
It did not get a boost for False Friends (like humans).
The Catch: The researchers had to manually tell the AI, "Hey, these specific words are True Friends, share them. These are False Friends, don't share them." The AI didn't figure this out on its own; it was programmed to do it.

The Big Takeaway

Can AI mimic human bilingual reading?
Yes, but only if we manually build the dictionary exactly the way human brains seem to work.

Human Brains: Naturally figure out that "winter" is the same in both languages, but "brand" is different. They use meaning to decide how to read.
AI Brains: Currently, they are like a super-fast statistician. They notice that "winter" appears a lot, so they get good at it. They don't truly understand that "brand" is a trap unless we explicitly tell them to treat it differently.

The Metaphor:
Think of a human bilingual reader as a detective who looks at clues (meaning) to solve a case.
Think of the current AI as a super-fast accountant who just counts how many times a number appears. If the number appears often, the accountant is confident. If we want the accountant to act like a detective, we have to manually write rules for them, because they don't naturally "get" the clues.

Conclusion:
While AI models can reproduce the patterns of human reading, they do so for the wrong reasons (frequency vs. meaning). To make them truly "human-like," we need to figure out how to teach them to understand meaning, not just count words.

1. Problem Statement

The paper investigates whether bilingual Neural Language Models (LMs) exhibit cross-lingual activation patterns similar to human bilinguals during reading. Specifically, it focuses on words with overlapping surface forms across two languages (Dutch and English):

Cognates (Friends): Words with similar form and meaning (e.g., winter in both languages). Humans typically show facilitation (faster processing) for these.
Interlingual Homographs (False Friends): Words with identical form but different meanings (e.g., Dutch brand means "fire," English brand means "trademark"). Humans typically show interference (slower processing) or no effect for these.

While standard bilingual LMs often share vocabulary (Full Overlap), it is unclear if this architectural choice accurately models human cognitive mechanisms or if specific vocabulary encoding strategies are required to replicate human-like behavior.

2. Methodology

2.1 Experimental Design: Vocabulary Manipulation

The authors trained four distinct Dutch-English causal Transformer models (GPT-2 small architecture) to test how vocabulary sharing affects cross-lingual transfer. They manipulated the embedding layer under four conditions (visualized in Figure 1 of the paper):

Full Overlap: All identical surface forms share a single, language-unspecific embedding.
Friends Overlap: Only cognates/loan words share embeddings; false friends and other words have language-specific embeddings.
False Friends Overlap: Only false friends share embeddings; cognates have language-specific embeddings.
Minimal Overlap: Only punctuation and named entities are shared; all other words have language-specific embeddings.

2.2 Training Data

Corpus: A mix of Wikipedia, OpenSubtitles, TedTalks, and web-crawled data (CC100).
Bilingual Simulation: Simulated an unbalanced late bilingual (75% Dutch tokens, 25% English tokens).
Training Regime: Sequential presentation within epochs (Dutch first, then English) to mimic L1/L2 exposure.
Tokenization: Custom Byte-level BPE tokenizers were used. Crucially, target words (friends and false friends) were forced to be single tokens to avoid segmentation bias.

2.3 Evaluation Stimuli

The models were evaluated on psycholinguistic stimuli from two human studies involving Dutch-English bilinguals:

Friend Study (Bultena et al., 2014): English sentences containing cognates vs. controls.
False Friend Study (Huisman, 2025): Dutch sentences containing false friends vs. controls.

2.4 Metrics

Surprisal: Used as a proxy for processing effort ( $-\log_2 p(w|context)$ ). Lower surprisal indicates facilitation; higher surprisal indicates interference.
Embedding Similarity: Cosine similarity between Dutch and English contextual and word embeddings to measure cross-lingual alignment.
Regression Analysis: Mixed-effects models to disentangle the effects of word frequency (sentence language vs. other language) and word category on surprisal.

3. Key Results

3.1 Language Separation vs. Sharing

General Separation: In conditions where embeddings were separated (Minimal Overlap), the models maintained distinct representations for Dutch and English, showing no cross-lingual effects.
Shared Embeddings Drive Effects: Cross-lingual activation (facilitation) occurred only when embeddings were shared.
- In Full Overlap and Friends Overlap, cognates showed significantly lower surprisal (facilitation) compared to controls.
- Surprisingly, in Full Overlap and False Friends Overlap, false friends also showed facilitation (lower surprisal) compared to controls, contrary to human data which often shows interference or no effect.

3.2 The Role of Frequency

Regression analyses revealed that the observed facilitation effects were driven primarily by word frequency, not by semantic consistency.

Shared embeddings receive training signals from both languages, effectively doubling the frequency count of the word.
Higher frequency leads to lower surprisal regardless of whether the word is a cognate or a false friend.
In the "Friends Overlap" condition, the facilitation for cognates was explained by English frequency (the sentence language), while Dutch frequency was redundant due to high correlation.
In the "False Friends" condition, both Dutch and English frequencies contributed to the model's prediction, but the model still failed to replicate human interference.

3.3 Alignment with Human Data

Best Match: The Friends Overlap condition was the only one that qualitatively matched human behavior: it produced facilitation for cognates but did not produce facilitation for false friends (because false friends had language-specific embeddings in this condition).
Mismatch: The Full Overlap condition (commonly used in prior work) produced facilitation for both cognates and false friends, failing to capture the human distinction where false friends often cause interference or neutral effects.

4. Key Contributions

Vocabulary Sensitivity: Demonstrated that the "human-likeness" of bilingual LMs is critically dependent on how lexical overlap is encoded in the vocabulary, not just the model architecture.
Frequency vs. Semantics: Showed that in current Transformer architectures, cross-lingual facilitation is largely a statistical artifact of increased frequency (shared counts) rather than a result of semantic alignment or competition.
Refutation of Full Overlap: Challenged the standard practice of using Full Overlap for bilingual training, showing it leads to unrealistic facilitation for false friends.
Methodological Framework: Provided a controlled framework for testing cross-lingual transfer by manipulating embedding sharing for specific word classes.

5. Significance and Limitations

Significance: The study suggests that while LMs can reproduce certain cross-lingual patterns, they lack the explanatory adequacy to model human bilingual reading unless the vocabulary is explicitly constrained to match human lexical representations (i.e., sharing only cognates). It highlights that current models rely on distributional statistics (frequency) rather than grounded semantic understanding.
Limitations:
- The study used a small-scale vocabulary manipulation (affecting only ~2-4% of the vocabulary).
- The "False Friend" human study did not show significant interference in humans, making it harder to validate the model's failure to replicate interference.
- The models were trained on text only, lacking the grounded, sensorimotor, and social cues humans use to distinguish meaning.

Conclusion

The paper concludes that bilingual LMs generally keep languages separate unless explicitly tied by shared embeddings. When tied, they exhibit facilitation driven by frequency. To align with human bilingual processing, models require a vocabulary design that shares embeddings only for cognates, not for false friends. This suggests that current standard bilingual LMs (using Full Overlap) may not be accurate models of the human bilingual lexicon.