The power of context: Random Forest classification of near synonyms. A case study in Modern Hindi

1. Problem Statement

The paper addresses the linguistic phenomenon of synonymy, specifically the "No Synonymy" or "No Equivalence" principle, which posits that absolute synonyms do not exist because language evolution favors efficiency. While it is theoretically accepted that near-synonyms (words with similar meanings but different origins) carry subtle differences in register, cultural association, or perspective, these distinctions are rarely tested quantitatively on a large scale.

The study focuses on Modern Hindi, a language characterized by a unique lexical continuum resulting from centuries of contact between Sanskrit (native) and Perso-Arabic (loanword) layers. The core research question is: Can machine learning models distinguish between near-synonyms of different etymological origins (Sanskrit vs. Perso-Arabic) based solely on their distributional context, even when their semantic meanings are nearly identical?

2. Methodology

Data Collection and Preprocessing

• Dataset: The study utilized 135 pairs of synonyms (270 words total), carefully selected to minimize definitional discrepancies, avoid homonyms, and cover a wide semantic field.
• Corpora: Two large Hindi corpora were used:
  1. IIT Bombay Hindi Monolingual Corpus: ~768 million tokens (non-lemmatized).
  2. hiTenTen13 (SketchEngine): ~350 million words (partially lemmatized, web/newspaper data).
• Normalization: Orthographic normalization was applied to handle Perso-Arabic diacritics (nuqtā) and Unicode variations.

Feature Extraction (Word Embeddings)

• Model: Word2Vec (Skip-gram architecture) was chosen for its interpretability over newer, more complex models.
• Parameters:
  • Vector dimension: 200.
  • Context window: 5 words (left and right).
  • Minimum frequency threshold: 10 occurrences.
• Rationale: A narrow window was selected to capture paradigmatic relations (synonyms) rather than broad syntactic relations.

Classification Algorithm

• Algorithm: Random Forest (a supervised ensemble learning method).
• Task: Binary classification to predict the etymological origin (Sanskrit vs. Perso-Arabic) of a word based on its 200-dimensional vector embedding.
• Validation Strategy: The model was trained and evaluated 100,000 times using random 80:20 train/test splits to ensure statistical robustness.
• Interpretability: SHAP (SHapley Additive exPlanations) values were used to determine which vector dimensions contributed most to the classification.

Control Experiments

• Dimensionality Reduction: Models were tested using subsets of vector dimensions (from 1 to 200) to test robustness.
• Frequency Analysis: Correlation matrices were computed to analyze the relationship between misclassification rates, word frequency, and cosine similarity.
• Control Corpus: Experiments were repeated on the hiTenTen13 corpus to verify generalizability.

3. Key Results

Classification Performance

• High Accuracy: The Random Forest model achieved an average accuracy of 88% across 100,000 iterations.
• Robustness: The model maintained accuracy above 80% even when trained on only the first 50 dimensions of the 200-dimensional vectors, suggesting that etymological signals are distributed throughout the embedding space rather than concentrated in specific dimensions.
• Feature Importance: SHAP analysis revealed that no single vector dimension dominated the prediction; instead, the predictive power was distributed evenly, indicating that the model captures a complex, systemic pattern of usage.

Analysis of Misclassification

• Error Distribution: Misclassifications were not random; they were concentrated in a small subset of words (~20% of the dataset).
• Frequency Effect: Contrary to the expectation that rare words are harder to classify, high-frequency words were more prone to misclassification. This is attributed to the "Mean-frequency law" (Zipf), where frequent words tend to be more polysemous and appear in broader, less specialized contexts.
• Etymological Bias: Sanskrit words were slightly more prone to misclassification (58% of errors in the top 20) compared to Perso-Arabic words, suggesting that Perso-Arabic loanwords retain more distinctive contextual "signatures."
• Semantic Specialization: Words associated with specific social concepts (e.g., nation, religion, army) were classified correctly, while generic, everyday terms (e.g., water, dream) were often misclassified.

Validation

• Cross-Corpus Consistency: The set of most frequently misclassified words showed a high overlap (Jaccard index ~0.43) between the two different corpora, confirming that the findings are not artifacts of a specific dataset.
• Exclusion Test: When the 50 most misclassified words were removed, the model's accuracy rose to 95%, confirming that the remaining words follow more conventional, distinguishable usage patterns.

4. Key Contributions

1. Quantitative Evidence for "No Synonymy": The study provides empirical proof that even centuries after borrowing, words of different origins maintain distinct distributional patterns. This supports the linguistic principle that "form implies meaning," even when the core definition is identical.
2. Etymology in Context: It demonstrates that context encodes etymological signals. Usage patterns preserve traces of a word's cultural and historical origin, allowing a model to infer etymology without explicit access to dictionary definitions or historical data.
3. New Semantic Framework: The findings suggest that etymologically related words form distinct conceptual subspaces. Synonyms are not just interchangeable labels but offer different "perspectives" or "semantic frames" shaped by their cultural background.
4. Methodological Innovation: The paper successfully applies Random Forest classification to word embeddings for the specific task of etymological distinction, a novel approach in corpus linguistics.

5. Significance and Implications

• Linguistic Theory: The results challenge the notion of "absolute synonymy" by showing that synonyms occupy slightly different semantic niches defined by usage frequency, register, and cultural association.
• NLP and Embeddings: The study reveals that word embeddings capture more than just lexical semantics; they encode sociolinguistic and pragmatic information (such as cultural origin and social class distinctions). This suggests that embeddings can serve as a proxy for detecting subtle cultural biases and stylistic variations.
• Future Research: The methodology is applicable to other languages with layered lexical histories (e.g., English with its Germanic/French layers). It opens new avenues for investigating how historical language contact shapes modern cognitive structures and how machine learning can uncover "hidden" linguistic features invisible to traditional analysis.

In conclusion, Bąkowski's study confirms that context is powerful enough to distinguish between near-synonyms based solely on their historical origin, validating the idea that language users intuitively navigate different "perspectives" when choosing between synonymous terms.