The Distribution of Phoneme Frequencies across the World's Languages: Macroscopic and Microscopic Information-Theoretic Models

1. Problem Statement

While the frequency distributions of words across languages are well-studied (often following power laws like Zipf's Law), the frequency distributions of phonemes remain under-explored. Previous studies (e.g., Sigurd, 1968; Martindale et al., 1996; Macklin-Cordes and Round, 2020) have attempted to model phoneme frequencies using power laws or geometric series. However, these approaches face significant limitations:

• Structural Mismatch: Phoneme inventories are small, closed sets (rarely exceeding a few dozen contrasts), unlike open-ended vocabularies. Consequently, they do not exhibit the extended linear behavior in double-logarithmic space characteristic of true power laws.
• Lack of Mechanism: Existing models describe how frequencies rank but fail to explain why specific phonemes are more frequent than others based on linguistic or physical constraints.
• Gap in Understanding: There is no unified information-theoretic account that explains both the global structural patterns of phoneme distributions across languages and the specific probabilities of individual phonemes within a language.

2. Methodology

The authors propose a two-level information-theoretic framework: a Macroscopic model for global rank-frequency patterns and a Microscopic model for language-specific phoneme probabilities.

A. Datasets

The study utilizes three distinct datasets to ensure robustness across typological and geographical diversity:

1. Sigurd (1968): Five manually transcribed distributions (American English, Bengali, Kaiwá, Samoan, Swedish).
2. Macklin-Cordes and Round (2020): 166 Australian language varieties (highly curated but typologically limited).
3. UDHR Dataset: 53 languages compiled semi-automatically using the Cross-linguistic Phonological Frequencies (XPF) corpus applied to the Universal Declaration of Human Rights. This dataset offers broad genetic and geographical coverage, though with lower accuracy due to automatic transcription.

B. Macroscopic Model: Symmetric Dirichlet Distribution

• Approach: The authors model the probability distribution of phonemes as a sample from a Symmetric Dirichlet distribution, $Dir(\alpha)$, where $\alpha$ is the concentration parameter.
• Assumption: In the absence of language-specific constraints, all phonemes are a priori equally probable, implying a symmetric distribution where $\alpha_1 = \alpha_2 = ... = \alpha_n = \alpha$.
• Analysis: They estimate the optimal $\alpha$ for each language and analyze its relationship with the phonemic inventory size ($n$). They utilize order statistics to map the theoretical Dirichlet distribution to observed rank-frequency plots.

C. Microscopic Model: Maximum Entropy (MaxEnt)

• Approach: To explain deviations from the macroscopic average and predict specific phoneme probabilities, the authors apply the Principle of Maximum Entropy. This finds the distribution that maximizes entropy subject to specific constraints derived from linguistic theory.
• Constraints (Feature Functions): Three distinct factors were operationalized as constraints:
  1. Physical Costs: Measured by cross-linguistic incidence probability (rarer phonemes globally imply higher articulatory/perceptual cost).
  2. Phonotactic Factors: Measured by segmental information (surprisal). This captures how predictable a phoneme is within a specific context (e.g., word onsets).
  3. Lexical Factors: Measured by lexical information gain. This quantifies how much a specific phoneme reduces uncertainty regarding word identity.
• Optimization: The model solves for Lagrange multipliers ($\lambda_k$) that satisfy the expected values of these constraints, generating a generative probability distribution for each phoneme.

3. Key Results

Macroscopic Findings

• Dirichlet Fit: Phoneme rank-frequency distributions across all datasets closely follow the order statistics of a symmetric Dirichlet distribution.
• Scaling Law: There is a robust, negative correlation between the inventory size ($n$) and the concentration parameter ($\alpha$). The relationship is modeled as:
  $$\hat{\alpha}(n) \approx 19.47 \cdot n^{-0.95}$$
• Compensation Hypothesis: As the phonemic inventory size increases, the concentration parameter $\alpha$ decreases. This leads to a decrease in relative entropy (distributional evenness).
  • Interpretation: Languages with larger inventories (higher complexity) compensate by having less uniform frequency distributions (lower entropy), thereby attenuating the total information processing cost. This provides quantitative support for the Compensation Hypothesis (Hockett, 1955).

Microscopic Findings

• Predictive Accuracy: The MaxEnt model, incorporating physical, phonotactic, and lexical constraints, accurately predicts language-specific phoneme probabilities. The correlation between observed and predicted probabilities is extremely high (non-linear regression closely follows the identity line).
• Constraint Effects:
  • Physical Cost: Negative coefficient ($\lambda < 0$). Phonemes with high global rarity (high cost) are less frequent within a language.
  • Phonotactic Surprisal: Positive coefficient ($\lambda > 0$). Counter-intuitively, phonemes occurring in less predictable (more surprising) contexts are more frequent. This suggests a diachronic mechanism where highly predictable phonemes are prone to elision/dropping over time.
  • Lexical Information Gain: Positive coefficient ($\lambda > 0$). Phonemes that contribute more to distinguishing words (higher information gain) are more frequent.
• Micro-Macro Link: The microscopic constraints (specifically phonotactic surprisal) show a negative correlation with inventory size, reinforcing the macroscopic compensation effect: larger inventories tend to have more predictable phonotactic structures.

4. Key Contributions

1. Unified Framework: The paper provides the first unified information-theoretic account of phoneme frequencies, bridging the gap between macroscopic statistical patterns and microscopic linguistic constraints.
2. Rejection of Power Laws: It demonstrates that power laws are inappropriate for phoneme distributions due to the closed nature of phonemic inventories, proposing the Symmetric Dirichlet distribution as a superior, virtually parameter-free alternative.
3. Quantitative Compensation: It offers empirical evidence for the Compensation Hypothesis at the level of unigram phoneme distributions, showing that increased inventory size is offset by decreased distributional evenness.
4. Generative Modeling: Unlike previous descriptive models, the MaxEnt approach is generative, deriving phoneme probabilities from first principles (physical, phonotactic, and lexical constraints) rather than fitting parameters to observed frequencies.

5. Significance

• Cognitive Insights: The findings suggest that human language processing is optimized to balance articulatory/perceptual costs, phonotactic predictability, and lexical discriminability.
• Methodological Advancement: The study introduces a rigorous methodology for analyzing linguistic phenomena using constraints and entropy maximization, moving beyond simple curve-fitting.
• Theoretical Impact: It challenges the assumption that all linguistic frequency distributions follow power laws, highlighting the unique statistical properties of closed systems like phoneme inventories.
• Practical Application: The ability to reconstruct phoneme distributions from inventory size alone (via the scaling law) or predict specific phoneme frequencies based on linguistic features has implications for language modeling, historical linguistics, and the study of language evolution.