💬 NLP

Polish phonology and morphology through the lens of distributional semantics

This study demonstrates that distributional semantic vectors for Polish words inherently encode sub-lexical phonological and morphological information, enabling accurate predictions of form-based linguistic features and supporting the view that semantic space is isomorphic with structural form space.

Original authors: Paula Orzechowska, R. Harald Baayen

Published 2026-04-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Paula Orzechowska, R. Harald Baayen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the Polish language as a massive, intricate Lego castle. For a long time, linguists believed that the shape of the bricks (the sounds and letters) and the meaning of the castle (what it represents) were built by two completely separate teams. One team just stacked bricks according to strict physical rules, and the other team just decided what the castle meant, with no connection between the two.

This paper, written by Paula Orzechowska and R. Harald Baayen, argues that those two teams were actually whispering to each other the whole time. They found that in Polish, the shape of a word actually hints at its meaning, even in ways we didn't think possible.

Here is the breakdown of their discovery using simple analogies:

1. The "Magic Mirror" of Meaning

The researchers used a computer tool called Distributional Semantics. Think of this as a giant, invisible map where every word in the Polish language is a city.

The Old Idea: Words with similar meanings (like "dog" and "puppy") are close together on the map. Words with different meanings are far apart. The sound of the word doesn't matter for this map.
The New Discovery: The researchers found that if you look closely at the map, the shape of the word also affects where the city is located.
- If a word has a very complex, "clunky" sound at the beginning (like a long string of consonants), it tends to cluster in a specific neighborhood on the map.
- If a word has a simple sound, it lives in a different neighborhood.
- The Analogy: Imagine walking into a library. Usually, books are sorted by topic (History, Science, Fiction). This study found that the library also secretly sorts books by the shape of their spines. If a book has a very jagged, complex spine, it ends up on the same shelf as other jagged-spine books, even if they are about different topics. The "jaggedness" of the spine tells you something about the book's content.

2. The "Sound-Clumps" (Consonant Clusters)

Polish is famous for having words that start with long strings of consonants, like grzbiet (back) or wschód (east). In English, we usually say, "That's just a weird sound rule."

The Study: The researchers asked: "Does the 'weirdness' of that sound tell us anything about the word's meaning?"
The Answer: Yes! They found that words with specific types of sound clumps (like a "falling" sound pattern vs. a "rising" one) tend to have related meanings or grammatical functions.
The Analogy: Think of a musical instrument. A drum might sound "thud-thud," while a violin sounds "squeak-squeak." You don't need to read the sheet music to know the drum is for rhythm and the violin is for melody; the texture of the sound gives it away. In Polish, the "texture" of the consonant clump gives a clue about the word's grammar or meaning.

3. The "Secret Decoder Ring" (The Computer Model)

The authors used a computer model called the Discriminative Lexicon Model (DLM).

How it works: Imagine a robot that has never been taught Polish grammar rules. It has never been told what a "prefix" is or what "past tense" means. It only knows two things:
1. What the word looks like (the letters).
2. What the word means (based on how it's used in millions of sentences).
The Magic: The robot was able to guess the grammar and sound rules of Polish with 90% to 99% accuracy.
The Analogy: It's like giving a child a box of 10,000 mixed-up puzzle pieces. You don't tell them what the picture is. But because the pieces that fit together have similar shapes and colors, the child can eventually figure out the whole picture just by looking at the pieces. The computer didn't need a rulebook; the "shape" of the words and their "meaning" were already perfectly aligned in the data.

4. Why This Matters

This study challenges a famous old idea from linguistics (by Ferdinand de Saussure) that the link between a word's sound and its meaning is arbitrary (random).

The Old View: The word "dog" sounds like "dog" just by chance. It could have been "cat" or "zorp."
The New View: In complex languages like Polish, the sound isn't random. The way a word is built (its "architecture") is tightly woven into what it means.
The Analogy: Think of a key and a lock. For a long time, we thought keys were just random metal shapes that happened to fit locks. This study suggests that the shape of the key is actually designed to fit the lock, and if you look closely at the key's grooves, you can predict exactly which door it opens.

The Bottom Line

The researchers proved that in Polish, form and meaning are best friends, not strangers.

You can predict a word's grammar (like whether it's past or future tense) just by looking at its sound.
You can predict a word's sound complexity just by knowing its meaning.
The "meaning" of a word isn't just an abstract idea; it leaves a fingerprint on the sound of the word itself.

This suggests that our brains (and our computers) don't need to memorize separate lists for "sounds" and "meanings." Instead, they are stored in one big, interconnected web where the shape of a word helps you understand its soul.

1. Problem Statement and Research Questions

The study challenges the traditional Saussurean view of the linguistic sign as arbitrary and the structuralist view of "dual articulation" (where phonology and syntax operate as independent systems). Specifically, the authors investigate whether the phonological (sound patterns) and morphological (word structure) properties of complex Polish words are systematically mirrored in their semantic representations (meaning).

Polish is chosen as the testbed due to its:

Rich consonant clusters: Often exceeding universal phonotactic constraints (e.g., word-initial CCCC clusters).
Complex morphology: High degree of inflection, syncretism (form overlap), and productive prefixation that triggers complex phonotactic sequences (morphonotactics).

Key Research Questions:

Can the grammatical functions of morphemes (tense, case, aspect) be inferred from semantic vectors without explicit morphological parsing?
Can phonotactic patterns (cluster size, markedness) and morphonotactic properties (prefix parsability) be modeled from semantic space without consulting phonemic representations?
Is there an isomorphism (structural similarity) between the "form space" and the "semantic space" that allows simple linear models to predict complex linguistic features?

2. Methodology

Data

Source: A subset of 456 morphologically complex words (featuring initial consonant clusters) extracted from The Basic Dictionary of Polish for Foreigners.
Expansion: These words were expanded into their full inflectional paradigms using the NKJP300M (National Corpus of Polish, 300 million tokens).
Final Dataset: 21,417 inflected forms reduced to 8,015 unique word forms (after removing homographs and keeping the most frequent syncretic form).
Annotations: Words were tagged for:
- Morphosyntax: Part of speech, tense, aspect, person, number, gender, case.
- Phonology/Morphonology: Cluster size (CC, CCC, CCCC), sonority profile (rise, fall, plateau), and prefix parsability (clear, active stem, bound root).

Embeddings

Two types of pre-trained word embeddings were utilized:

fastText: Uses subword n-grams (character sequences), capturing morphological structure.
Word2vec: Treats words as indivisible units, capturing only whole-word co-occurrence.

Computational Models

Discriminative Lexicon Model (DLM):
- A connectionist model using linear mappings (matrix algebra) to learn the relationship between form cues (orthographic trigrams) and semantic vectors.
- Comprehension: Maps form vectors $\to$ semantic vectors.
- Production: Maps semantic vectors $\to$ form vectors.
- Training: Used "end-state learning" (Widrow-Hoff rule) without iterative frequency weighting to test theoretical learnability.
Linear Discriminant Analysis (LDA):
- A supervised classification algorithm used to predict linguistic categories (e.g., "Is this word Past Tense?") directly from the semantic embeddings.
- Evaluated using Leave-One-Out Cross-Validation (LOOCV).
t-SNE (t-distributed Stochastic Neighbor Embedding):
- An unsupervised dimensionality reduction technique used to visualize high-dimensional semantic spaces in 2D to observe clustering patterns.
Principal Component Analysis (PCA):
- Used to determine how many dimensions of the semantic space are required to capture specific linguistic features (e.g., does "tense" require the first 10 dimensions or the last 200?).

3. Key Contributions

Empirical Evidence for Form-Meaning Isomorphism: The study demonstrates that semantic vectors contain sufficient information to predict not just grammatical categories, but also phonotactic complexity and morphonotactic transparency.
Decoupling Form and Meaning in Modeling: The DLM achieved high accuracy in comprehension and production tasks without being explicitly provided with stem boundaries, morpheme lists, or phonological rules. It learned these structures purely from the statistical alignment of form and meaning in the data.
Subword vs. Whole-word Sensitivity: The comparison between fastText and Word2vec reveals that while whole-word co-occurrence (Word2vec) captures significant morphosyntactic structure, subword information (fastText) is crucial for capturing fine-grained phonological and morphological details (e.g., cluster markedness).
Dimensionality of Semantic Space: The study maps which linguistic features reside in the "high-variance" dimensions of semantic space (e.g., tense, person) versus the "low-variance" dimensions (e.g., cluster size, markedness), suggesting a hierarchical organization of linguistic information in meaning.

4. Results

A. Discriminative Lexicon Model (DLM) Performance

Comprehension: Achieved 97.2% – 98.3% accuracy across full, training, and held-out datasets.
Production: Achieved 97.3% on training data but dropped to 77.8% on held-out data (still significantly above chance).
Error Analysis: Production errors were mostly grammatical (e.g., wrong tense or case) rather than phonotactically illegal, suggesting the model learned the system's constraints.

B. Predictability of Linguistic Features (LDA on Embeddings)

The LDA successfully predicted various features from semantic vectors, with accuracy far exceeding majority baselines:

Feature Category	Specific Feature	Accuracy (fastText)	Baseline	Observation
Morphosyntax	Person	99%	60%	Very clear clustering.
	Tense	99%	49%	Past vs. Non-past highly distinct.
	Aspect	97%	60%	High accuracy despite t-SNE overlap.
	Number	96%	65%	Robust despite syncretism.
	Case	89%	28%	Confusion reflects syncretism (e.g., Nom/Acc).
	Gender	87%	38%	Lower due to animacy sub-genders.
Phonology	Prefix Type	97%	40%	Surprisingly high; semantics encode prefix choice.
	Cluster Parsability	82%	56%	Less parsable clusters are harder to predict.
	Cluster Markedness	82%	30%	Sonority profiles are encoded in meaning.
	Cluster Size	81%	57%	Length is predictable from semantics.

Note: Word2vec (whole-word) embeddings performed consistently lower (approx. 10% drop) but still significantly above baseline, proving that even without subword analysis, semantic distribution encodes structural constraints.

C. Dimensionality Analysis (PCA)

High-Varience Features: Tense, Person, and Number are captured by the first 10–50 principal components. These features drive the major structure of semantic space.
Low-Varience Features: Phonotactic properties (Cluster Size, Markedness) and Morphonotactic Parsability require 200–300 dimensions to be accurately predicted. They are represented as subtle, distributed patterns across the entire semantic space rather than distinct clusters.

D. Visualization (t-SNE)

Morphosyntax: Clear, tight clusters were observed for tense, person, and number.
Phonology: t-SNE plots for phonological features (cluster size, markedness) showed significant overlap, failing to separate classes visually. However, LDA (supervised) could still classify them accurately. This indicates that while these features do not dominate the variance (making them invisible to unsupervised 2D projection), they are linearly separable in the high-dimensional space.

5. Significance and Implications

Challenge to Arbitrariness: The findings provide strong evidence against the strict arbitrariness of the sign. The fact that phonotactic complexity (e.g., a 4-consonant cluster) and morphological transparency can be predicted from meaning suggests a deep, systematic coupling between form and meaning.
Mechanism of Deep Learning Success: The authors argue that the success of modern Large Language Models (LLMs) may stem from this inherent isomorphism. If form and meaning are aligned in the data, simple linear mappings (or shallow neural networks) can effectively learn complex grammatical and phonological rules without explicit symbolic programming.
Re-evaluating Morphonotactics: The study supports the view that "marked" phonological clusters (which violate universal constraints) are not random but are semantically motivated by morphological operations (prefixation). The semantic space "remembers" the morphological history of the word.
Methodological Insight: The divergence between t-SNE (unsupervised) and LDA (supervised) results highlights that linear separability does not require clustering. Linguistic features can be encoded in the semantic space as distributed, low-variance signals that are invisible to dimensionality reduction but accessible to linear classifiers.

In conclusion, the paper demonstrates that the "form space" and "semantic space" of Polish are highly isomorphic. The mental lexicon (modeled here computationally) does not need to explicitly store rules for phonotactics or morphology; these constraints emerge naturally from the statistical distribution of word meanings.