Beyond cognacy

1. Problem Statement

Computational phylogenetics has become a standard tool in historical linguistics for reconstructing language evolution. However, current state-of-the-art methods rely heavily on expert-annotated cognate sets (groups of historically related words). This reliance creates three major bottlenecks:

• Scalability: Expert annotation is labor-intensive and sparse, limiting analyses to specific language families rather than global scales.
• Replicability: Manual annotation introduces subjectivity and hinders reproducibility.
• Scope: Cognate-based methods are inherently constrained to individual language families, making it difficult to test hypotheses about macro-families or global language evolution.

The paper addresses the need for fully automated methods that can extract phylogenetic signals directly from raw lexical data (word lists) without requiring manual cognate identification.

2. Methodology

The study compares three distinct approaches for generating binary character matrices used in Maximum Likelihood (ML) phylogenetic inference (using raxml-ng with the BIN+G model).

Data Sources

• Lexical Data: 113,671 entries from 928 languages (928 languages present in both Lexibank and Grambank), covering 110 high-coverage concepts.
• Gold Standard: Glottolog expert classification for tree topology validation.
• Typological Data: Grambank (195 binary features) for model fit validation.

The Three Methods Compared

1. Cognate Classes (cc): The baseline method. Uses expert-annotated cognate sets. Each cognate class is treated as a binary character (1 if present, 0 if absent/different). This method is limited to family-level signals by definition.
2. PMI (Pointwise Mutual Information): An automated method based on Jäger (2018).
   • Uses an SVM to cluster words into automatic cognate groups.
   • Generates features based on unigrams (sound classes) and concepts.
   • Relies on the statistical association between sound classes and concepts.
3. MSA (Multiple Sequence Alignment): A fully automated method based on Akavarapu and Bhattacharya (2024), with specific modifications by the author.
   • Step 1 (Pairwise Distance): Computes Levenshtein distances on ASJP transcriptions to identify potentially related language pairs.
   • Step 2 (Training): Trains a Pair-Hidden Markov Model (pHMM) combined with logistic regression to distinguish cognates from non-cognates. The model learns emission probabilities for sound class matches and gaps.
   • Step 3 (Alignment): Uses the trained pHMM and Viterbi algorithm to align synonymous word pairs across languages.
   • Step 4 (MSA Construction): Aggregates pairwise alignments into a global Multiple Sequence Alignment using T-Coffee.
   • Step 5 (Binarization): Converts the MSA into a binary matrix. Presence of a specific sound class in a column is a "1"; gaps are treated as missing or flipped depending on the specific character definition. This captures both cognacy and sound correspondence patterns implicitly.

Evaluation Metrics

• Generalized Quartet Distance (GQD): Measures topological agreement between inferred trees and the Glottolog gold standard (0 = perfect, 0.67 = random).
• Akaike Information Criterion (AIC): Measures how well the inferred tree explains the evolution of typological features in Grambank (lower is better).
• Phylogenetic Difficulty: Estimated via PyPythia to quantify the intrinsic signal strength in the data (0 = strong signal, 1 = no signal).

3. Key Contributions

• Development of a Scalable MSA Pipeline: The paper refines and implements a fully automated pipeline that uses pHMMs to generate MSAs from raw lexical data, bypassing the need for manual cognate coding.
• Comprehensive Benchmarking: It provides the first large-scale comparison (928 languages) between traditional cognate-based methods, automatic clustering (PMI), and MSA-based inference.
• Demonstration of Global Applicability: The study proves that automated methods can successfully reconstruct phylogenies across diverse, unrelated language families, a task previously difficult without expert intervention.

4. Results

The evaluation was conducted on the full dataset, 100 random samples of 100 languages, and 14 specific large language families.

• Global Performance (Cross-Family):

  • MSA significantly outperformed both cc and PMI.
  • GQD: MSA achieved the lowest distance to Glottolog (0.054 vs. 0.188 for cc and 0.062 for PMI), indicating superior topological accuracy.
  • AIC: MSA showed the best fit to typological data (lowest AIC), suggesting the inferred trees better explain the evolution of grammatical features.
  • Signal Strength: MSA yielded the lowest "phylogenetic difficulty" score (0.21), indicating a much clearer phylogenetic signal in the data compared to the other methods.

• Individual Family Performance:

  • Within specific language families, the results were more mixed. MSA was competitive with or slightly better than the other methods in many cases (e.g., Indo-European, Sino-Tibetan), but in some families, PMI or cc performed comparably.
  • However, MSA consistently produced the lowest phylogenetic difficulty scores across families.

• Qualitative Findings:

  • The pHMM trained in the MSA pipeline successfully learned linguistically plausible sound correspondences (e.g., high probability of matching /p/ with /f/ or /b/).
  • The MSA approach captures information about both cognacy and sound changes simultaneously, providing a richer feature set than simple binary cognate presence.

5. Significance

• Overcoming the Annotation Bottleneck: The study demonstrates that high-quality phylogenetic trees can be generated without expert cognate annotation. This opens the door to analyzing the entire world's languages (global-scale phylogenies) rather than just well-studied families.
• Robustness of Automated Methods: The MSA-based approach is shown to be not only a viable alternative but often superior to traditional methods when dealing with heterogeneous datasets spanning multiple families.
• Future Applications: By enabling the construction of global trees, this method facilitates downstream tasks such as statistical modeling of global language evolution, testing macro-family hypotheses, and understanding the interplay between lexical and typological evolution.

In conclusion, Jäger argues that MSA-based inference is a promising, scalable, and robust alternative to traditional cognate-based methods, effectively "going beyond cognacy" to utilize raw lexical data for historical linguistics.