Retrieval-Augmented Generation for Taxpayer Advisory in an LMIC Setting: A Proof-of-Concept Evaluation of Statutory Legal Retrieval
This study presents a proof-of-concept evaluation of a retrieval-augmented generation framework for taxpayer advisory in low- and middle-income countries, demonstrating that semantic retrieval of national income-tax legislation can achieve high accuracy in identifying statutory provisions to support grounded language model prompting, while highlighting the need for further validation on generation quality and legal compliance before deployment.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In many parts of the world, understanding the rules that govern how people and businesses pay taxes is a daunting task. The laws are often written in dense, technical language, and finding the specific rule that applies to a unique situation can feel like searching for a needle in a haystack. To help people navigate this complexity, tax authorities are increasingly turning to digital tools, including artificial intelligence systems designed to answer questions. However, a major challenge remains: these computer programs are trained on vast amounts of general data, which means they can sometimes sound confident while providing incorrect or made-up information, a problem known as "hallucination." To solve this, researchers have developed a method called retrieval-augmented generation. This approach works by first having the computer search a trusted, official library of documents to find the exact text of the law, and only then asking the computer to write an answer based strictly on what it found. This ensures the response is grounded in reality rather than guesswork. The question for experts is whether this method works well enough to be useful in countries with fewer resources, where tax systems are being modernized but where access to specialized legal experts is often limited.
A team of researchers from universities in Uganda set out to test this idea with a specific focus on the Ugandan Income Tax Act. Their goal was not to build a fully finished product that could give legal advice, but to create a working prototype to see if the core technology could accurately find the right sections of the tax law when asked a question. They began by taking the entire text of the national tax law and breaking it down into 198 distinct, manageable pieces of information. Each piece was tagged with its location in the law, such as the specific part and section number, and then converted into a digital format that allows a computer to understand the meaning of the words rather than just matching keywords. This process created a searchable digital library where the computer could look for the concept behind a question, not just the exact words.
To test how well this system worked, the researchers generated 900 different questions that a taxpayer might ask, such as inquiries about deductions or specific rates. For each question, they knew exactly which section of the law contained the correct answer. They then asked their computer system to find the three most relevant sections of the law for each question. The results showed that the system was quite effective at its primary job. When asked to find the single best match, the system succeeded 72 percent of the time. More importantly, when looking at the top three results it provided, the correct answer was included 86.78 percent of the time. This suggests that while the computer did not always pick the absolute best answer as its very first choice, it almost always placed the correct information within the first few options it presented.
The researchers noted that when the system missed the top spot, it was often because the wrong answer was very similar in meaning to the right one, rather than being completely unrelated. This indicates that the system understands the concepts well but sometimes struggles to decide which of two very similar legal provisions is the most precise fit. The study successfully demonstrated that it is possible to take a complex legal document, turn it into a machine-readable format, and use it to retrieve specific information with a high degree of accuracy. However, the team was careful to clarify that this was only a test of the search mechanism. They did not test whether the final written answers generated by the computer were legally perfect, safe to use, or easy for real people to understand.
This work is significant because it offers a potential path for tax authorities in developing nations to provide better, more accessible information to their citizens without needing to hire a vast army of legal experts for every query. The study suggests that a relatively small, well-organized collection of laws can be transformed into a powerful tool for finding information. Yet, the authors emphasize that this is just the beginning. Before such a system could be used in the real world to guide taxpayers, it would need to be tested with actual questions from real people, checked by legal professionals to ensure the advice is correct, and adapted to handle multiple languages and different types of tax laws. The current results show that the technology is feasible, but turning that feasibility into a reliable public service requires further work to ensure safety, accuracy, and trust.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.