Hybrid Neural-LLM Pipeline for Morphological Glossing in Endangered Language Documentation: A Case Study of Jungar Tuvan

1. Problem Statement

Interlinear Glossed Text (IGT) is the standard format for linguistic documentation, aligning transcriptions with morpheme segmentation, grammatical glosses, and translations. However, creating IGT is labor-intensive and remains a bottleneck, particularly for low-resource, morphologically rich languages like endangered tongues.

Existing automated approaches face distinct limitations:

• Structured Prediction Models (e.g., BiLSTM-CRF): Effective at capturing local morphological patterns and global constraints but lack world knowledge, flexibility, and the ability to handle rare morphemes or novel combinations not seen in training data.
• Large Language Models (LLMs): Possess broad linguistic knowledge and few-shot generalization capabilities but struggle with consistency, require extensive in-context examples (increasing inference costs), and lack the inductive biases of structured sequence models.

The paper addresses the challenge of creating a computationally light, accurate, and scalable pipeline for automatic glossing that overcomes the weaknesses of both standalone approaches.

2. Methodology

The authors propose a two-stage hybrid pipeline evaluated on Jungar Tuvan, a low-resource Turkic language with complex agglutinative morphology.

A. Data

• Corpus: 895 manually annotated IGT sentences collected from fieldwork in Xinjiang (1987–1995).
• Statistics: 240 unique grammatical morpheme labels, 1,258 unique content word glosses, averaging 1.69 morphemes per word.
• Split: Document-level split (85% train, 15% test) to prevent information leakage from discourse-level patterns.

B. The Hybrid Architecture

The pipeline consists of:

1. Stage 1: Structured Prediction (BiLSTM-CRF):
   • A baseline model trained from scratch using character-level embeddings.
   • It generates an initial sequence of gloss labels for segmented morphemes.
   • Role: Provides structural guidance and captures frequent grammatical paradigms.
2. Stage 2: LLM Post-Correction:
   • The BiLSTM output is treated as a "rough initial hypothesis" rather than ground truth.
   • An LLM (using Retrieval-Augmented Generation - RAG) reviews the hypothesis, compares it against retrieved similar examples, and corrects errors.
   • Role: Leverages contextual inference to fix rare lexical items and novel combinations.

C. Experimental Design

The authors conducted systematic ablation studies across four LLMs (DeepSeek-v3.2-exp, Qwen3-max, GPT-4o-mini, Gemma-3-27b-it) to test:

1. Retrieval Strategy: Similarity-based retrieval (TF-IDF cosine similarity) vs. random example selection.
2. N-Shot Scaling: Varying the number of in-context examples (1 to 20).
3. Glossary Ablation: Testing the impact of providing morpheme dictionaries (None, Top-100, Grammatical-only, Entire dictionary).
4. Hybrid vs. Pure Generation: Comparing the two-stage pipeline against pure LLM generation and the BiLSTM baseline.

3. Key Contributions

1. Hybrid Architecture: Demonstrates that combining a structured predictor (BiLSTM-CRF) with LLM reasoning yields consistent improvements over either method alone, particularly in low-resource settings.
2. Design Principles for Fieldwork:
   • Retrieval is Critical: Similarity-based example selection significantly outperforms random selection.
   • Dictionary Paradox: Providing morpheme dictionaries (even complete ones) generally hurts performance for most models, likely due to information overload or distraction.
   • Optimal Few-Shot Scaling: Performance scales logarithmically with example count, plateauing or declining after 10–15 examples.
3. Error Analysis Insights: The hybrid approach specifically targets the weaknesses of structured models (lexical gaps) while maintaining their strengths (grammatical consistency).

4. Key Results

A. Retrieval and Scaling

• RAG vs. Random: Retrieval-augmented prompting provided massive gains over random selection (e.g., DeepSeek improved from 0.118 to 0.506 accuracy).
• N-Shot Scaling: Performance follows a logarithmic curve. Most models peaked between n=10 and n=15 examples. Beyond this, performance either plateaued or declined due to context saturation or retrieval quality degradation.

B. The Dictionary Paradox

• Counter-intuitively, providing morpheme dictionaries degraded performance for 3 out of 4 models.
• Partial dictionaries (Top-100 or Grammatical-only) caused the largest drops (0.04–0.10 accuracy loss).
• Only Gemma-3-27b-it showed improvement with the complete dictionary.
• Hypothesis: Models may be distracted by the unstructured key-value lists or fail to balance dictionary lookups with contextual inference.

C. Hybrid Pipeline Performance

The hybrid pipeline (BiLSTM + LLM Correction) achieved the highest results across all models:

• DeepSeek-v3.2-exp: Reached 0.698 accuracy (vs. 0.474 baseline).
• Gemma-3-27b-it: Reached 0.644 accuracy.
• Low-Shot Advantage: The hybrid approach showed the most significant gains in low-shot scenarios (n=1 to n=5), where the BiLSTM provided crucial structural scaffolding that pure LLMs lacked.

D. Error Analysis

• BiLSTM Baseline: High accuracy on frequent grammatical morphemes (0.923) but poor on rare lexical morphemes (0.479).
• Hybrid Improvement: Dramatically improved lexical accuracy (from 0.479 to 0.682) while maintaining high grammatical accuracy. The gains were concentrated in infrequent morphemes (1–5 occurrences in training), where the LLM's contextual inference compensated for the lack of training signal.

5. Significance and Implications

• Practical Fieldwork: The study offers a computationally light solution for endangered language documentation. It suggests that linguists should invest in retrieval infrastructure and hybrid workflows rather than relying solely on massive dictionaries or pure LLM prompting.
• Rethinking Prompt Engineering: The finding that dictionaries often hurt performance challenges the conventional wisdom of "more information is better." It suggests that for morphological tasks, contextual examples are more effective than static lexical lookups in current LLM architectures.
• Scalability: The logarithmic scaling of performance with example count provides a clear guideline for resource-constrained projects: aim for 5–15 high-quality, similar examples rather than maximizing the number of in-context shots.
• Ethical Considerations: The authors emphasize the need for human oversight, treating model outputs as hypotheses rather than authoritative annotations, and respecting data sovereignty in cross-border endangered language contexts.

In conclusion, this paper establishes that a hybrid approach—leveraging the structural rigor of traditional NLP models and the contextual reasoning of LLMs—offers the most promising path forward for automating linguistic annotation in low-resource, morphologically complex environments.