Comparative Analysis of Rule Based Symbolic Logic and a Large Language Model in Sanskrit Morphological Analysis
This exploratory study compares a rule-based symbolic classifier and a Large Language Model on Sanskrit morphological analysis using Bhagavad Gita data, finding that both approaches maintain stable compatibility under word-order scrambling, with the LLM achieving slightly higher accuracy (86.43%) than the symbolic system (81.43%).
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Language is often thought of as a stream of words flowing in a specific order, where the position of a word determines its meaning. In many languages, changing the order of words changes the sentence entirely. However, Sanskrit, an ancient language of India, operates differently. It is built on a system of endings attached to words, much like tags that tell you who is doing what to whom, regardless of where the word sits in the sentence. This flexibility allows speakers to rearrange words for emphasis or poetic effect without losing the grammatical logic. For centuries, scholars have relied on strict, written rules to decode these endings, a method known as rule-based logic. Today, a new kind of tool has emerged: large language models. These are computer systems trained on vast amounts of text that predict the next word or the meaning of a word based on patterns they have learned, rather than following a fixed set of instructions. The question researchers have been asking is how these two very different approaches handle the unique challenge of Sanskrit when the order of words is shuffled.
A recent study set out to test this by pitting a traditional rule-based computer program against a modern artificial intelligence system. The researchers used a collection of one hundred verses from the Bhagavad Gita, a sacred Hindu text. They took the words from these verses and created two versions of the text for each verse. In the first version, the words appeared in their natural, original order. In the second version, the same words were mixed up, or scrambled, so that the sequence was completely different, though the collection of words remained exactly the same. The goal was to see if the computer systems could still correctly identify the grammatical role of each word—such as whether a word was the subject, the object, or a verb—when the context of its position was removed.
The study focused on a specific task: looking at individual words that had already been separated from the sentences and asking the systems to label them. The researchers used a Python program that followed explicit, step-by-step rules to check the endings of the words. If a word ended in a specific way, the program assigned it a label based on a pre-written list of possibilities. This system does not look at the surrounding words or the sentence structure; it only looks at the word itself. On the other side, the researchers used prediction files generated by Gemini, a large language model. These files contained the model's guesses for the grammatical labels of the same words, based on prompts that asked it to focus on word endings rather than translation or context.
When the researchers compared the results, they found something striking. The rule-based program performed exactly the same on the natural order and the scrambled order. This was expected, because the program was designed to ignore word order entirely; it simply checked the word it was given against its list of rules. However, the large language model also showed remarkable stability. Even when the words were scrambled, the model's predictions for the grammatical labels did not change in any meaningful way. Out of 280 specific words that were checked against a reference guide, the model correctly identified the grammatical category in 86.43 percent of the cases, whether the words were in their original order or mixed up. The rule-based system was slightly less accurate, getting 81.43 percent correct, but it was equally consistent across both versions of the text.
The researchers were careful to note that this stability does not mean the large language model is ignoring word order or that it has learned the rules of Sanskrit grammar in the same way a human scholar does. The study did not prove that the model works without context, nor did it show that it could handle a completely new set of words it had never seen before. The results only showed that for this specific set of words, the model's output remained steady even when the input was shuffled. It is possible that the model recognized the words from its training data or that the specific prompts used discouraged it from using context. The study also highlighted that the reference guide used to check the answers was not perfect; it contained some ambiguities and was not reviewed by multiple experts, which means the scores are a measure of compatibility with the provided guide rather than absolute truth.
What this research ultimately reveals is a snapshot of how these two different technologies behave under a specific test. The rule-based system behaved exactly as its design intended, remaining unchanged because it was built to be blind to word order. The large language model, which is often thought to rely heavily on the sequence of words to understand meaning, also produced the same results when the sequence was broken. This suggests that for this particular task, the model's predictions were robust against the scrambling of the input. However, the author cautions that this does not mean the model has mastered Sanskrit or that it will behave this way in all situations. The study serves as a clear record of what happened with these specific files and these specific prompts, showing that in this recorded comparison, the order of the words did not change the outcome. It leaves open the question of whether this stability holds true for more complex sentences, different models, or texts that were not part of the training data, pointing toward a need for further testing with more rigorous standards.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.