💬 NLP

Dzongkha Next Word Prediction System

This paper presents a Dzongkha next word prediction system designed to streamline typing by reducing keystrokes, which utilizes a dataset of 100,000 sentences to train and compare LSTM, Bi-LSTM, and GRU models, ultimately demonstrating that the GRU model achieves the best performance with 74.03% accuracy while effectively mitigating overfitting.

Original authors: Prerna Chhetri, Tenzin Yoezer, Phuntsho Wangmo, Tshewang Bomjan

Published 2026-07-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Prerna Chhetri, Tenzin Yoezer, Phuntsho Wangmo, Tshewang Bomjan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you're trying to type a story in Dzongkha, the national language of Bhutan. It's a beautiful script, but typing it is like trying to build a house out of tiny, intricate Lego bricks where every single syllable needs a dozen different key presses. It's slow, it's frustrating, and it's easy to make a mistake. The authors of this paper, a team of students from the College of Science and Technology in Bhutan, asked a simple question: "Can we build a digital assistant that guesses the next word you want to type, so you don't have to press as many keys?"

They didn't just guess; they built a brain for a computer to learn this language. They fed it a massive library of 100,000 sentences from the Dzongkha Development Commission. That's over 1.3 million words and nearly 28,000 unique words the computer had to memorize. Before the computer could learn, the team had to clean the data, stripping away numbers and special symbols that didn't belong in the words, and breaking the sentences down into tiny chunks called tokens, like turning a sentence into a string of unique ID numbers.

To teach the computer, they tried three different types of "neural networks," which are like different styles of student brains. They tested an LSTM, a Bi-LSTM, and a GRU. Think of these as three different study groups. The LSTM was like a student who studied hard but got a bit confused by the sheer volume of material. The Bi-LSTM was a bit better, looking at the sentence from both directions, but it still struggled to keep up with the massive dataset.

The real star of the show was the GRU. The authors describe the GRU as "lightweight," which is like saying it's a nimble athlete who can run a marathon without getting winded, unlike the heavier, slower models. When they trained all three models on the full 100,000-sentence dataset, the GRU was the standout performer. While the other models initially showed acceptable accuracy on smaller datasets (where LSTM hit 73.79%), they faced issues with overfitting when the data size increased. After fine-tuning the hyperparameters for the 100k dataset, the LSTM dropped to 60% accuracy, and the Bi-LSTM reached 70.79%. The GRU, however, not only achieved the highest accuracy of 74.03% but also solved the problem of overfitting specifically within the context of training on this large 100k dataset, meaning it could predict new words it hadn't seen before with much greater reliability than its counterparts under those specific conditions.

The results were measured with cold, hard numbers. The LSTM managed an accuracy of 60%, and the Bi-LSTM reached 70.79%. But the GRU model hit a 74.03% accuracy. That might not sound like a perfect score, but in the world of predicting language, it's a solid victory. The paper explicitly notes that while the other models struggled with overfitting on the larger dataset, the GRU solved this problem, meaning it could predict new words it hadn't seen before with much greater reliability.

The team didn't stop at just the numbers. They built a whole system around this winning model. They designed a user-friendly interface (using tools like Figma and Django) so that when a person types a few words, the system can suggest the next one, cutting down the number of keystrokes needed. They even mention that they had to upgrade their computer hardware to a powerful GPU (an Nvidia RTX 3090) just to handle the training, because the smaller computers they started with couldn't hold the massive dataset.

In the end, the paper suggests that this GRU-based system is a promising solution to make typing in Dzongkha faster and less error-prone. It's not a magic wand that fixes everything instantly, but it's a significant step forward. The authors are careful to say this is a "promising result" and that they plan to turn their web app into a downloadable software package in the future to make it even more useful for writers and speakers of the language. They proved that with the right model and enough data, a computer can learn to help humans speak their native tongue more easily.

Technical Summary: Dzonghka Next Word Prediction System

Problem Statement
Dzongkha, the national language of Bhutan, is essential for official documentation, scriptures, and cultural preservation. However, typing in Dzongkha is inherently time-consuming and error-prone due to the script's complexity. Unlike alphabetic scripts, Dzongkha syllables often require multiple keystrokes and complex character combinations, making the typing process "hectic" and inefficient. The authors identify an urgent need for a system that can predict the next syllable or word to reduce manual keystrokes, enhance typing speed, and maintain language authenticity.

Methodology
The project follows a structured pipeline involving data acquisition, preprocessing, model training, and system integration:

Dataset: The primary corpus was acquired from the Dzongkha Development Commission (DCDD). The study utilized two dataset scales: an initial 20,000-sentence subset and a final 100,000-sentence dataset containing 1,331,282 words and 28,344 unique words.
Preprocessing: The text underwent rigorous cleaning, including the removal of alphanumeric characters, special symbols (e.g., '#', '?', '།'), and specific characters like 'Tsha' and 'shay' to align Dzongkha word structures more closely with English equivalents. The process included tokenization (converting words to unique integer IDs), N-gram sequence generation (splitting sequences into features $X$ and labels $Y$ ), and pre-padding to ensure uniform input dimensions (maximum sequence length set to 22).
Model Architecture: Three Recurrent Neural Network (RNN) architectures were selected and trained using the TensorFlow/Keras framework: Long Short-Term Memory (LSTM), Bidirectional LSTM (Bi-LSTM), and Gated Recurrent Unit (GRU).
- The models utilized an Embedding layer, followed by recurrent layers (GRU layers with 512 and 256 units), a Dense layer, and an Output layer with Softmax activation.
- Training employed "sparse categorical cross-entropy" as the loss function and the Adam optimizer.
Hyperparameter Tuning: The team performed fine-tuning on parameters such as Epochs (200), Batch Size (500), Dropout (0.2), and Early Stopping (10) to optimize performance and prevent overfitting.
System Integration: The trained model was integrated into a web application using Django for the backend, SQLite for the database, and HTML/CSS/JavaScript (with Bootstrap) for the frontend. Figma was used for UI prototyping.

Key Results
The study compared the performance of the three models on the 100,000-sentence dataset:

LSTM: Achieved 60% accuracy with a training time of 8 hours.
Bi-LSTM: Achieved 70.79% accuracy with a training time of 13 hours.
GRU: Achieved the highest accuracy of 74.03% with a training time of 15.33 hours.

The authors note that while initial training on smaller datasets resulted in overfitting issues, the GRU model trained on the larger 100K dataset successfully resolved overfitting, demonstrating robust generalization on unseen data. The GRU model was selected for final integration due to its lightweight nature and superior handling of the larger dataset.

Significance and Claims
The paper claims that the proposed system successfully addresses the challenge of Dzongkha typing by significantly reducing the number of keystrokes required. By implementing a next-word prediction system, the project aims to make typing more convenient for Dzongkha writers, thereby aiding in the preservation of language richness and promoting meaningful expression. The authors emphasize that the system is a practical solution to the "cumbersome" nature of Dzongkha spelling and typing, offering a technological remedy to enhance efficiency without compromising the language's authenticity.

Future Work
The authors modestly outline future directions, which include developing a standalone software package (.exe) for the web application and further improving the model to enhance the overall performance of the application.

Technical Summary: Dzonghka Next Word Prediction System

More like this