LLM-Based SQL Generation: Prompting, Self-Refinement, and Adaptive Weighted Majority Voting
This paper proposes a novel framework for LLM-based Text-to-SQL generation that combines a Single-Agent Self-Refinement with Ensemble Voting (SSEV) pipeline and a multi-agent collaborative system (ReCAPAgent-SQL) to achieve competitive execution accuracy on benchmarks like Spider and BIRD while effectively addressing the complexities of real-world enterprise databases without relying on ground-truth data.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you want to ask a question to a massive library of information (a database), but the librarians only speak a very strict, robotic language called SQL. You speak natural English. The goal of this research is to build a translator that can turn your English questions into perfect SQL commands so the library can answer you.
The authors of this paper, Yu-Jie Yang and colleagues, tackled the problem that these translators often get confused, make mistakes, or don't understand the library's layout. They built two main systems to fix this: a "Team of Experts" approach and a "Specialized Agent Team" approach.
Here is how they did it, using simple analogies:
1. The Problem: The "One-Size-Fits-All" Translator Fails
Imagine asking a single person to translate a complex legal document into another language. They might miss a nuance, get a name wrong, or use the wrong grammar. In the world of databases, this happens when a single AI model tries to guess the SQL code. It often gets lost in the "schema" (the map of the library) or misunderstands your question.
2. The First Solution: The "Team of Experts" (SSEV Pipeline)
The authors created a system called SSEV (Single-Agent Self-Refinement with Ensemble Voting). Think of this not as one translator, but as a panel of five different experts sitting around a table.
- The Drafting Phase (PreSQL): The panel writes down their initial guesses for the answer.
- The Map Check (Schema Linking): They realize they are looking at a giant map of the library. Instead of trying to read the whole map, they use their first guesses to highlight only the relevant aisles and shelves. This cuts out the noise.
- The Second Draft (PostSQL): With the map narrowed down, they write a cleaner, more focused second draft.
- The "Self-Correction" Loop: If a draft has a typo or doesn't work when they try to run it, they don't just throw it away. They look at the error message, fix it, and try again. It's like a writer editing their own work until it makes sense.
- The Voting System (WMA): This is the secret sauce. Instead of just taking a vote where everyone gets one equal point, they use a Weighted Majority Algorithm.
- Imagine a game show where the judges have different levels of trust. If "Judge A" has been right 10 times in a row, their vote counts for more. If "Judge B" keeps making mistakes, their vote counts for less.
- The system constantly updates these "trust scores" based on who got the answer right in the past. Over time, the system learns to listen mostly to the smartest expert in the room.
The Result: On standard tests (like Spider 1.0 and BIRD), this "Team of Experts" approach got the right answer about 86% of the time, which is much better than any single expert working alone.
3. The Second Solution: The "Specialized Agent Team" (ReCAPAgent-SQL)
For even harder problems (like the new Spider 2.0 dataset, which mimics messy, real-world corporate databases), a simple panel isn't enough. The authors built ReCAPAgent-SQL, which is like hiring a specialized task force where every member has a specific job.
- The Planner: Breaks your big question into small, logical steps (like a project manager).
- The Retriever: Goes out to find extra manuals or documentation if the team doesn't know a specific rule (like a researcher).
- The Critic: Reviews the work and says, "Wait, that logic doesn't make sense," or "You missed a step."
- The Schema Linker: Specifically handles the library map, making sure they are looking at the right tables and columns.
- The Self-Refiner: If the code breaks, this agent fixes it.
- The Validator: The final quality control inspector who checks, "Does this answer actually solve the user's problem?"
These agents talk to each other in a loop. They plan, act, get criticized, fix the mistake, and try again until they get it right.
The Result: When they tested this on the hardest, most realistic questions (Spider 2.0-lite), they took a baseline system that only got 6% of the answers right and boosted it to 31%. That is a massive leap in a field where progress is usually measured in tiny fractions.
4. Why This Matters
The paper claims that by combining self-correction (fixing your own mistakes), smart voting (listening to the best experts), and specialized agents (having a team with specific roles), we can build systems that handle complex, real-world data questions much better than before.
They didn't just guess; they tested these methods on standard benchmarks and proved that:
- Weighted Voting works better than simple majority voting because it adapts to who is actually good at the task.
- Self-Refinement helps fix errors that happen when the database says "No, that command is wrong."
- Multi-Agent Systems are necessary for the messy, complicated databases found in real companies.
In short, they built a smarter, more resilient translator that learns from its mistakes and listens to its best experts, making it much easier for regular people to ask complex questions to massive databases.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.