SoraniSQL: A Novel Synthetic Text-to-SQL Dataset and Benchmark for Low-Resource Central Kurdish (Sorani)
This paper introduces SoraniSQL, the first Text-to-SQL dataset and benchmark for Central Kurdish (Sorani), comprising 7,518 question-SQL pairs that demonstrate the effectiveness of fine-tuned models in achieving high execution accuracy across diverse linguistic challenges while enabling offline application deployment.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written by the author. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a world where you could ask a computer a simple question in your own language and get an answer pulled directly from a complex database, without needing to know the secret code that usually controls it. This is the promise of "text-to-SQL" technology. For decades, these systems have been trained primarily on English, learning to translate human questions into the structured language that databases speak. While this works well for widely spoken languages, it leaves behind hundreds of millions of people who speak low-resource languages, such as the various dialects of Kurdish. In these regions, small shop owners and managers often rely on point-of-sale terminals that are simple, offline machines. They cannot send their data to the cloud for processing, and they lack the tools to ask their computers questions in their native tongue. The gap between what these systems can do and what these communities need is wide, largely because no one has yet built a reliable testing ground to see if computers can truly understand Kurdish.
A researcher at Sulaimani Polytechnic University in Iraq has now filled that gap with a new project called SoraniSQL. This work introduces the first-ever dataset and testing benchmark designed specifically for Central Kurdish, known as Sorani. The goal was not just to translate existing English tests, but to build a resource that understands the unique quirks of the Kurdish language, which mixes Arabic and Persian letter forms, uses different numbers, and often spells words in multiple ways depending on the keyboard used. To create this, the researcher constructed a digital version of a grocery store chain with eight branches, complete with thousands of sales records, product lists, and employee details. From this database, they generated over 7,500 questions in Sorani, ranging from simple lookups like "Which category does this cake belong to?" to complex requests like "Show me the average receipt value by month for card payments." Each question was paired with the exact computer code needed to retrieve the answer, and a panel of native speakers rated the questions for natural flow and accuracy, ensuring they sounded like something a real person would ask.
The study then put a variety of computer systems to the test to see how well they could translate these Sorani questions into working database commands. The researchers split the questions into different categories to see where the systems would struggle. Some questions used new product names but followed familiar sentence structures, while others used completely new ways of phrasing things or combined ideas in ways the computer had never seen before. The results showed a clear divide between simple systems and more advanced ones. A basic system that tried to match questions to answers it had already memorized performed well on simple, new names but failed completely when asked to combine ideas in new ways. However, a more sophisticated system that broke the question down into parts—identifying the topic, the action, and the filters separately—was able to handle these new combinations with high accuracy, getting nearly all of them right.
Perhaps the most surprising finding involved the use of modern, pre-trained language models, which are powerful artificial intelligence tools often used for coding. When these models were simply shown a few examples of how to translate Kurdish to code, they performed poorly, getting less than half of the simple questions right and failing almost entirely on the complex ones. But when the researchers took a smaller version of this model and carefully trained it on the specific rules of this dataset, its performance jumped dramatically. This fine-tuned model, which can run on a standard laptop without needing a supercomputer, achieved accuracy rates comparable to the best specialized systems. It successfully handled new phrasings and complex combinations, proving that with the right training, even small models can master the task.
The research also tackled the messy reality of how people actually type. In Kurdish, users often mix different keyboard layouts, drop specific letters, or swap characters that look similar. The study found that these spelling variations caused significant errors, but a system that could fuzzy-match words—recognizing that "milk" and "mlik" refer to the same thing—recovered most of the lost accuracy. To ensure safety, the team also built a digital guard that checked every generated command to make sure it could not be tricked into revealing private data or damaging the database. Finally, they packaged the best-performing system into a standalone Windows application called DanaText2SQL. This program runs entirely offline, connecting directly to the local database used by shops. In tests, it answered questions in less than half a second, demonstrating that a small shop in the Kurdistan region could soon have a computer that understands their language, respects their privacy, and works without an internet connection.
This work does not claim to have solved every problem in language technology, nor does it suggest that computers are now perfect translators. The study explicitly showed that systems still struggle with certain complex sentence structures they have never seen before, and that confidence in the answer is not always a guarantee of correctness. However, it provides a solid, reproducible foundation for the first time. By creating a dedicated dataset, testing different approaches, and proving that a safe, offline system is possible, the research offers a clear path forward. It suggests that with careful training and a focus on the specific challenges of the language, technology can finally be brought to the small businesses and communities that have been left behind by the digital divide.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.