Phonological Fossils: Machine Learning Detection of Non-Mainstream Vocabulary in Sulawesi Basic Lexicon

1. Problem Statement

The study addresses a persistent challenge in Austronesian linguistics: the identification of non-mainstream vocabulary in Sulawesi languages. A significant portion of basic vocabulary in these languages resists reconstruction to Proto-Austronesian (PAn) and lacks plausible cognates.

• The Hypothesis: Traditionally, such "non-conforming" forms are interpreted as pre-Austronesian substrate (remnants of languages displaced by Austronesian expansion).
• The Gap: This interpretation assumes that phonological non-conformity implies a shared non-Austronesian ancestry. However, this has rarely been tested systematically. It is possible that these forms represent parallel independent innovations rather than a single shared substrate.
• The Challenge: Traditional comparative methods are labor-intensive, require deep specialist knowledge, and do not scale well. Furthermore, using cognacy data to identify non-cognates creates a circular logic problem when training machine learning models.

2. Methodology

The authors propose a novel two-model machine learning framework to detect phonological non-conformity without relying on circular cognacy data for the primary scientific claim.

Data

• Source: Austronesian Basic Vocabulary Database (ABVD) in CLDF format.
• Corpus: 1,357 lexical forms from six Sulawesi languages (Muna, Bugis, Makassar, Wolio, Toraja-Sa'dan, Tolaki) covering 210 Swadesh concepts.
• Labeling (Rule-Based Baseline - E022):
  • Forms with ABVD cognate codes or rescued via PAn cross-checking are labeled "Austronesian".
  • Residual forms (no cognates) are labeled "Candidate Substrate".
  • Note: This is treated as a Positive-Unlabeled (PU) learning problem, where "Candidate Substrate" contains both true substrates and missing-data Austronesian forms.

Feature Engineering (Model B)

To avoid circularity, the study employs a Phonological-Only Model (Model B) trained on 27 features derived strictly from surface orthography (no morphological decomposition):

1. Phonological Features (10): Form length (syllable/character count), vowel ratios, initial character class, presence of glottal stops, nasal clusters, consonant clusters, reduplication, and absence of canonical Austronesian prefixes.
2. Initial-Character Encoding (8): One-hot encoding of frequent onsets.
3. Semantic Features (2): Core vocabulary membership (Swadesh-100 vs. 210) and semantic domain.
4. Control Features (2): Language identity and language-level cognacy coverage (to control for documentation depth).

Experimental Design

• Model A (Full Features): Includes distributional/cognacy features. Serves as a circular baseline (expected AUC $\approx$ 1.0) to validate the labeling process.
• Model B (Phonological Only): Excludes all cognacy data. If this model performs above chance, it proves that phonological patterns alone distinguish inherited vs. non-mainstream forms.
• Classifiers: XGBoost, Random Forest, and Logistic Regression.
• Validation:
  • Stratified 5-fold Cross-Validation (10 seeds).
  • Leave-One-Language-Out (LOLO): Tests generalization across languages.
  • Robustness Checks: Approximate IPA conversion tests and feature ablation (removing the "cognacy coverage" confound).
• Consensus Analysis: Cross-tabulation of Rule-Based (E022) and ML (E027) predictions to identify high-confidence candidates.
• Clustering Analysis (E029): Tests the "Shared Substrate Hypothesis" by clustering consensus candidates using Levenshtein distance. If they share a common ancestor, they should form coherent clusters.

3. Key Contributions

1. Methodological Innovation: Introduction of a two-model design that separates "circular" cognacy-based features from "non-circular" phonological features, allowing for the detection of phonological signals independent of etymological assumptions.
2. The "Phonological Fingerprint": Identification of a distinct set of phonological traits characterizing non-mainstream vocabulary in Sulawesi, detectable by ML.
3. Negative Result on Shared Substrate: Providing computational evidence that non-mainstream vocabulary in Sulawesi does not derive from a single pre-Austronesian language layer, but rather represents parallel independent innovations.
4. Script Convergence: Correlating ML findings with historical script adaptation (Javanese Hanacaraka), showing that the phonological categories eliminated by the script match the "non-mainstream" features detected by the model.

4. Key Results

Classification Performance

• Model A (Full): Achieved near-perfect performance (AUC = 1.0), confirming that distributional features encode the labels directly.
• Model B (Phonological Only): Achieved AUC = 0.763 (± 0.007) after removing the cognacy coverage confound. This confirms that phonological properties alone carry a reliable signal.
• Generalization: The model generalized well across languages in LOLO tests (Mean AUC = 0.715; 5/6 languages > 0.65).
• Robustness: Performance remained stable after approximate IPA conversion ($\Delta$AUC < 0.01) and removal of form length features, confirming the signal is not an orthographic artifact.

The "Phonological Fingerprint"

SHAP analysis identified the top features driving the classification of non-mainstream forms:

1. Longer forms: Average 2.57 syllables vs. 2.29 for inherited forms.
2. Consonant clusters: Higher frequency of CC+ sequences.
3. Glottal stops: Higher presence of glottal stop markers.
4. Semantic Domain: Over-representation of Action Verbs (44% of high-confidence candidates).
5. Lack of Canonical Prefixes: Fewer instances of standard Austronesian prefixes (e.g., ma-, pa-).

Consensus and Clustering

• Consensus: 266 forms were identified as "Consensus Substrate" by both the rule-based method and the ML model (Cohen's $\kappa$ = 0.61).
• Clustering Failure: When these 266 forms were clustered phonologically:
  • Silhouette Score: 0.114 (near random).
  • Cross-linguistic Cognate Test: $p = 0.569$.
  • Conclusion: Forms for the same concept across different languages are not more similar to each other than random vocabulary. This refutes the hypothesis of a single shared pre-Austronesian substrate.

Geographic Expansion

Applied to 16 additional languages, the model confirmed geographic patterning:

• Sulawesi Expansion: Mean predicted substrate rate = 0.606.
• Western Indonesian: Mean predicted substrate rate = 0.393.
• This suggests the "non-mainstream" phenomenon is geographically concentrated in Sulawesi.

5. Significance and Implications

• Reinterpretation of Substrate: The study challenges the traditional view that phonological non-conformity equals a shared substrate. Instead, it suggests these forms are likely parallel independent innovations (e.g., language-specific neologisms, semantic shifts, or contact-induced changes) occurring in similar semantic domains (like action verbs).
• Complement to Comparative Method: The ML approach is not a replacement for the comparative method but a scalable screening tool. It can flag phonologically anomalous forms for specialist etymological investigation, reducing the manual burden of sifting through large vocabularies.
• Historical Script Validation: The finding that the Javanese Hanacaraka script systematically eliminated the exact phonological categories (aspiration, retroflexion, specific sibilants) identified as "non-mainstream" by the ML model provides independent, historical validation of the phonological constraints detected computationally.
• Limitations: The study acknowledges limitations, including reliance on orthography rather than full IPA, small sample sizes, and the inability to distinguish root-internal clusters from morphological clusters without morphological parsing.

Conclusion: The paper demonstrates that machine learning can effectively detect a "phonological fingerprint" of non-mainstream vocabulary in Sulawesi. However, the lack of coherent cross-linguistic word families among these candidates suggests that the "substrate" in this region is not a single, unified pre-Austronesian language, but rather a complex landscape of independent linguistic developments.