Sense-Augmented Corpus for Sanskrit-English Machine Translation: Corpus Construction, Model Fine-Tuning, and Evaluation
This paper addresses the challenge of lexical ambiguity in Sanskrit-English machine translation by constructing a sense-augmented parallel corpus using LLMs and a custom sense inventory, demonstrating that explicitly incorporating word sense information significantly improves translation quality across different model architectures.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Language is more than a collection of words; it is a living system where meaning shifts depending on the company a word keeps. In the study of language, a single word can wear many different hats, a concept known as polysemy. For a human reader, context acts as a guide, instantly clarifying whether a word refers to a physical object, an abstract idea, or an action. However, for computers, this task is notoriously difficult. Machines often struggle to distinguish between these different meanings, leading to translations that are grammatically correct but semantically confused. This problem is particularly acute for ancient languages like Sanskrit, where a single term can carry centuries of nuanced meaning, and for which digital resources are scarce. When a machine fails to grasp the intended sense of a word, the resulting translation can lose the very essence of the original text, turning a profound philosophical statement into a nonsensical string of words.
Researchers at IIIT Hyderabad and the Central Sanskrit University have tackled this challenge by teaching machines to pay closer attention to the specific meaning of words before they attempt to translate them. Their work focuses on creating a new kind of training data for Sanskrit-to-English translation. Instead of simply feeding a computer pairs of Sanskrit sentences and their English equivalents, the team enriched these pairs with explicit labels that identify the exact sense of every word in the sentence. They used a powerful artificial intelligence tool to act as a digital lexicographer, reading each Sanskrit sentence and selecting the correct definition for every word from a massive dictionary, much like a human scholar would. This process resulted in a "sense-augmented" corpus, a dataset where the computer is not just shown the words, but is also told exactly what those words mean in that specific context.
The team tested this approach using several different translation models, including specialized translation engines and general-purpose large language models. They began by seeing how these models performed without any extra training, a state known as zero-shot inference. As expected, the models struggled, often producing translations that were fragmented or completely missed the point of the original text. For instance, when translating a sentence about warriors fleeing in terror, a basic model might translate a word for "mountain fortress" simply as "forest," missing the crucial imagery of the original epic. When the researchers then fine-tuned these models using the standard parallel sentences, the translations improved significantly, becoming more coherent and fluent. However, the most dramatic leap in quality occurred when the models were trained on the new, sense-augmented data.
The results showed that providing explicit sense information consistently improved the quality of the translations across all the models tested. The machines became much better at choosing the right word for the right situation. In one specific case, a model trained on the sense-augmented data correctly translated a phrase describing a "large army" as an army, whereas the same model trained only on standard data had incorrectly translated it as "cloth," a mistake that completely altered the meaning of the scene. Another example involved a word that could mean "party" or "cause"; the enhanced model correctly chose "party" to refer to a group of people, while the standard model chose "cause," which did not fit the context. These improvements were not just minor tweaks; they represented a fundamental shift in how the models understood the text, allowing them to resolve ambiguities that had previously caused them to fail.
The study also highlighted that this method works well even for general-purpose artificial intelligence models that were not originally designed for translation. A general model, which typically struggles with specific language pairs, was able to reach performance levels comparable to dedicated translation engines once it was trained on the sense-augmented data. This suggests that the key to better translation lies not just in having more data, but in having data that is semantically rich. The researchers found that the models learned to make more contextually appropriate choices, effectively reducing the number of times they repeated phrases or generated nonsense, a common error in machine translation. While the process of creating this enriched dataset required significant computational power and time, the payoff was a clear demonstration that teaching machines to disambiguate words explicitly leads to translations that are not only more accurate but also more faithful to the spirit of the original language.
The work does not claim to have solved every problem in machine translation, nor does it suggest that the current methods are perfect. The researchers noted that their sense inventory was extremely detailed, which sometimes made it harder for the models to choose between very similar meanings. They also acknowledged that the process is computationally expensive and that errors in the initial sense assignment could potentially affect the final translation. However, the findings offer a promising path forward, particularly for low-resource languages where data is scarce. By showing that explicit sense information can bridge the gap between human understanding and machine processing, the study provides a blueprint for improving how computers interact with the complex, layered meanings of human language. The ultimate goal is not just to translate words, but to convey the precise intent and nuance of the speaker, a task that requires a level of understanding that only careful, sense-aware training can provide.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.