💬 NLP

Biasless Language Models Learn Unnaturally: How LLMs Fail to Distinguish the Possible from the Impossible

Contrary to previous claims, this study demonstrates that GPT-2 fails to systematically distinguish between natural and "impossible" languages, learning both with equal ease and showing no innate bias toward humanly possible linguistic structures.

Original authors: Imry Ziv, Nur Lan, Emmanuel Chemla

Published 2026-04-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Imry Ziv, Nur Lan, Emmanuel Chemla

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Question: Do AI and Humans Think Alike?

Imagine you are teaching a robot to speak. You give it a million sentences from real life (like a storybook) and a million sentences that are completely scrambled nonsense (like a book where every word is thrown in a blender).

The Old Theory: For a long time, linguists believed humans have a special "biological filter" in our brains. We are born knowing that some patterns make sense (like "The cat sat on the mat") and others are impossible (like "On the mat sat the cat" if the grammar rules are broken in a specific, unnatural way). This is called the Generative Hypothesis. It suggests that if you showed a human a "scrambled" language, they would instantly say, "Nope, that's not a real language."

The New Controversy: Recently, some researchers claimed that Large Language Models (LLMs) like GPT-2 act just like humans. They said, "Look! The AI learns the real language much faster than the scrambled nonsense. This proves AI has the same 'biological filter' as us, and maybe we don't need to assume humans have special innate biases."

This Paper's Verdict: The authors of this paper say, "Hold on a minute. We tested this again, but with more languages and more types of scrambling. And guess what? The AI doesn't care. It learns the real language and the nonsense language at the exact same speed. Sometimes, it even prefers the nonsense!"

The Experiment: The "Language Gym"

To test this, the researchers set up a giant gym for the AI (GPT-2).

The Weights (The Data): They took 9 different languages (English, French, Russian, Hebrew, etc.).
The Scramble (The Impossible): For every real language, they created "impossible" versions using a few tricks:
- The Shuffle: Taking a sentence and mixing up the words randomly.
- The Reverse: Reading the sentence backward.
- The Hop: Moving words around in a weird, unnatural pattern.
The Workout (Training): They trained the AI on the real language and the scrambled version separately.
The Scorecard (Perplexity): They measured how "confused" the AI got. In AI terms, this is called Perplexity.
- Low Perplexity = The AI understands it easily (like a smooth run).
- High Perplexity = The AI is struggling (like running through mud).

The Results: The AI is "Biasless"

If the old theory were true, the AI should have been like a human: it should have run smoothly on the real language and stumbled badly on the scrambled one.

What actually happened?

The "Twin" Effect: The AI's learning curve for the real language and the scrambled language were almost identical. They were twins.
The "Preference" Glitch: In many cases, the AI actually found the scrambled nonsense slightly easier to learn than the real language!
The Big Picture: When they looked at all the languages together, the AI didn't separate "Real Languages" from "Fake Languages." It treated them all as just another set of patterns to memorize.

The Analogy: The Music Student vs. The DJ

Imagine two students learning music:

The Human Student (The Linguist): You play them a beautiful, structured symphony. Then you play them a track where the drums are playing backwards and the violin is screaming randomly. The human student immediately says, "That's not music; that's noise. I can't learn that because it breaks the rules of harmony." They have an innate bias for structure.
The AI Student (The DJ): You play them the symphony. Then you play the noise. The AI student just starts memorizing the notes. "Okay, in the symphony, the violin goes high-low. In the noise, the violin goes low-high. Got it." The AI doesn't care if the music is "real" or "impossible." It just sees patterns.

The paper concludes: The AI is a brilliant DJ, but it is not a human musician. It lacks the "biological filter" that makes humans instinctively reject impossible languages.

Why Does This Matter?

For AI: It suggests that current AI models are just really good at pattern matching, but they don't actually "understand" language the way humans do. They don't have the same internal rules that guide human speech.
For Linguistics: It supports the idea that humans do have special, innate biases that shape how we speak. If AI (which has no biology) can't tell the difference between a real language and a broken one, but humans can, then those human biases must be real and unique to us.
For the "Easy Learning" Myth: The paper debunks the idea that we can just look at how fast an AI learns to prove it thinks like a human. Sometimes, learning fast just means you are good at memorizing nonsense, not understanding truth.

The Bottom Line

The paper argues that LLMs are "biasless" in a way that humans are not. They can learn anything, even things that are logically impossible for a human to speak. This means we can't use AI as a perfect mirror to understand how the human brain learns language. The human brain has a special "grammar filter" that AI simply doesn't have.

1. Problem Statement

The paper addresses a central debate in cognitive science and linguistics: Do Large Language Models (LLMs) share the same innate learning biases as humans?

The Generative Hypothesis: Linguists argue that human language is constrained by innate biological biases (e.g., a preference for hierarchical structures over linear ones). This explains why certain "impossible" languages (those violating these biases) do not exist in nature, while others do.
The Challenge: Recent studies (e.g., Kallini et al., 2024) suggested that LLMs (specifically GPT-2) can distinguish between natural and "impossible" languages. They found that LLMs learn natural languages faster (lower perplexity) than artificially perturbed, impossible variants. If true, this would undermine the generative hypothesis, suggesting that complex linguistic typology emerges from statistical learning without innate biases.
The Research Gap: The authors question whether the previous findings are robust across different languages and perturbation types. They aim to test if LLMs truly possess a "sensitivity to impossibility" or if they learn unnatural patterns just as easily as natural ones.

2. Methodology

The authors extend the methodology of Kallini et al. (2024) by scaling it to a cross-linguistic and multi-perturbation framework.

A. Experimental Setup

Model: GPT-2 (12 layers, 12 heads, 768 dimensions) trained from scratch.
Languages: 9 languages representing distinct branches: English, Danish, German, French, Italian, Greek, Hebrew, Finnish, and Russian.
Data: 90M token subsets from Wikipedia dumps, processed with a uniform whitespace tokenizer (vocab size 50,257) to eliminate tokenization biases across languages.
Training: Standard hyperparameters (batch size 512, LR $5 \times 10^{-4}$ ) for 3,000 batches. Validation perplexity is recorded every 100 batches.

B. Perturbation Functions (Creating "Impossible" Languages)

The authors applied specific transformations to natural language datasets to create "impossible" variants that violate linguistic universals (specifically hierarchical structure and linear dependencies):

SHUFFLE (Global/Local): Randomly reordering tokens or swapping adjacent tokens, destroying constituent structure.
REVERSE (Full/Partial): Reversing entire sentences or segments, violating binding and anaphora rules.
SWITCH: Swapping specific tokens (indices 0 and 2).
HOP: Inserting a marker three tokens after a verb (requiring linear counting rather than hierarchical dependency).

C. Evaluation Metrics

The authors analyze sensitivity to impossibility from two perspectives:

Intralinguistic Perspective: Comparing the learning curve (perplexity) of a natural language vs. its specific impossible counterpart.
- Metric: Mean Error (ME) = Average difference between baseline and perturbed curves. A negative ME implies the model prefers the natural language; a positive ME implies a preference for the impossible one.
Interlinguistic Perspective: Comparing the entire set of natural languages against the entire set of impossible languages.
- Metrics: Minimum validation perplexity and Area Under the Curve (AUC).
- Logic: If LLMs have innate biases, the variance in learning difficulty between natural and impossible languages (within-language variance) should be significantly larger than the variance between different natural languages (across-language variance).

3. Key Contributions

Cross-Linguistic Expansion: Unlike previous work limited to English, this study tests 9 languages across 7 distinct language families, ensuring results are not artifacts of English-specific properties.
Uniform Tokenization: By using a single whitespace tokenizer for all languages, the authors control for variables like morphological complexity and tokenization bias, which were potential confounds in prior studies.
Dual-Perspective Analysis: The paper introduces a rigorous interlinguistic test (comparing variances across the whole set of languages) to determine if LLMs can separate the "possible" from the "impossible" as a class, not just in isolated pairs.
Replication and Refutation: The study systematically replicates the Kallini et al. (2024) setup but finds contradictory results, challenging the claim that LLMs naturally learn human-like linguistic biases.

4. Key Results

Intralinguistic Results (Language vs. Counterpart)

No Systematic Preference: In the vast majority of cases (28 out of 54 test cases showed the expected direction, but this was not statistically significant, $p=0.892$ ), GPT-2 did not learn the natural language faster than the impossible variant.
Preference for Impossible: In many cases (e.g., REVERSE full, SWITCH), the model actually learned the impossible variants better (lower perplexity) than the natural language.
Conclusion: The model fails to distinguish the possible from the impossible on a sentence-by-sentence or language-by-language basis.

Interlinguistic Results (Set vs. Set)

Clustering by Language, Not Typology: When plotting minimal perplexity and AUC, the data points clustered by language identity (e.g., all Russian variants grouped together) rather than by typology (natural vs. impossible).
Variance Analysis: The variance in learning difficulty between different languages (across-language variance) was significantly higher than the variance between a natural language and its impossible variants (within-language variance).
Implication: The model treats an impossible version of Russian as "Russian-like" (similar difficulty) rather than "impossible-like" (significantly harder). There is no stable boundary separating the set of possible languages from impossible ones.

5. Significance and Conclusion

Refutation of LLM Sensitivity: The study provides strong empirical evidence that GPT-2 lacks the "sensitivity to impossibility" previously claimed. It learns unnatural, structurally broken patterns just as easily as natural ones.
Support for the Generative Hypothesis: Since LLMs (which are theoretically "bias-free" statistical learners) fail to replicate the human distinction between possible and impossible languages, the paper argues that human linguistic typology cannot be explained solely by statistical learning. Instead, it supports the view that humans possess innate, domain-specific learning biases that shape what languages are learnable.
Critique of Perplexity as a Proxy: The authors conclude that "ease-of-learning" measured by perplexity curves is an insufficient metric for modeling human linguistic cognition or deriving language typology. The method used in previous studies (Kallini et al., 2024) appears to be fragile and not generalizable across languages.
Future Directions: The authors suggest that while GPT-2 fails, future work should investigate if larger, state-of-the-art models exhibit different behaviors, and whether cultural evolution models might amplify small learning asymmetries that pure statistical models miss.

In summary, the paper argues that LLMs do not share human learning biases, and therefore, the existence of "impossible languages" remains a strong indicator of innate human cognitive constraints, not a limitation of statistical modeling.