💬 NLP

Test-Time Verification for Text-to-SQL via Outcome Reward Models

This paper introduces GradeSQL, a framework that leverages Outcome Reward Models (ORMs) as learned semantic scorers to enhance Text-to-SQL reliability, demonstrating that ORM-based verification significantly outperforms traditional heuristic methods like execution-based Best-of-N and Majority Voting on the BIRD and Spider benchmarks.

Original authors: Mattia Tritto, Giuseppe Farano, Dario Di Palma, Gaetano Rossiello, Fedelucio Narducci, Dharmashankar Subramanian, Tommaso Di Noia

Published 2026-07-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mattia Tritto, Giuseppe Farano, Dario Di Palma, Gaetano Rossiello, Fedelucio Narducci, Dharmashankar Subramanian, Tommaso Di Noia

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are asking a very smart, but sometimes overconfident, chef (the AI) to cook a specific dish based on a recipe you describe in plain English. The chef knows how to cook, but sometimes they get the ingredients wrong or mix up the steps.

In the world of computers, this is called Text-to-SQL: translating a human question into a database query (a set of instructions for a computer database). The problem is that if the chef makes even a tiny mistake, the computer might give you the wrong answer, or nothing at all.

The Old Way: "Guess and Check"

Usually, when the chef isn't sure, the system asks them to cook the dish 32 times (generating 32 different SQL queries). Then, it has to pick the best one.

The old methods for picking the winner are like this:

Majority Voting: "Who cooked the dish most often?" If 20 chefs say "add salt" and 12 say "add sugar," the system assumes "salt" is right. But what if the recipe actually needed sugar, and the majority just made the same mistake?
Execution Success: "Who actually got the pan to work?" If a query runs without crashing, the system picks it. But a query might run perfectly and still give you the wrong data (like serving a cake when you asked for soup).

These methods rely on simple, surface-level clues (heuristics) rather than actually understanding if the dish is correct.

The New Way: The "GradeSQL" Food Critic

This paper introduces a new system called GradeSQL. Instead of just counting votes or checking if the pan works, they train a specialized Food Critic (called an Outcome Reward Model or ORM).

Here is how the GradeSQL system works, step-by-step:

1. The Cooking Class (Training)
First, the system needs to teach the Critic what "good" and "bad" look like.

It takes a question and asks the main chef (the AI) to cook 32 different versions of the dish.
It then runs all 32 dishes against the "Gold Standard" (the correct answer).
If a dish tastes exactly like the Gold Standard, the Critic gives it a High Score. If it tastes different, it gets a Low Score.
The Critic learns from these examples to recognize the flavor of a correct query, not just whether it crashed.

2. The Tasting Session (Inference)
Now, when a real user asks a question:

The chef cooks 32 new versions.
Instead of just checking if they run, the Critic tastes every single one.
The Critic gives each dish a score based on how well it matches the intent of the question.
The system picks the dish with the highest score.

Why is this better?

The paper tested this on two huge databases of questions (called BIRD and Spider). They found that the Critic was much better at spotting the right answer than the old "vote counting" or "did it crash?" methods.

The Analogy: Imagine a multiple-choice test. The old method picks the answer that appears most often or doesn't have a typo. The new method (GradeSQL) actually reads the question and the answer to see if they make sense together.
The Results: On difficult questions, the Critic helped the system get the right answer about 4% more often on the BIRD dataset and 2% more often on the Spider dataset. While 2-4% sounds small, in the world of AI, that is a huge improvement, especially for the hardest questions where the old methods usually give up.

Key Takeaways

It's a "Learned" Judge: The Critic isn't just following rules; it learned what a correct SQL query looks like by practicing on thousands of examples.
No Human Needed: The system taught itself how to be a critic by automatically checking which answers worked, so no humans had to manually grade the homework.
It Scales: The more options (candidates) the chef generates, the better the Critic performs. The old methods get stuck and stop improving, but the Critic keeps getting smarter as it has more choices to pick from.

In short, GradeSQL is like hiring a professional food critic to choose the best dish from a batch of 32, rather than just asking the crowd or checking if the stove is on. It makes the AI more reliable when the questions get tricky.

Technical Summary: Test-Time Verification for Text-to-SQL via Outcome Reward Models

Problem Statement

The reliability of Large Language Models (LLMs) in structured reasoning tasks, specifically Text-to-SQL, remains a critical challenge. While LLMs have advanced significantly, they often struggle with complex queries involving multi-table joins, nested subqueries, and subtle constraints. Current test-time inference strategies, such as Best-of-N (BoN) sampling and Majority Voting, attempt to mitigate this by generating multiple candidate SQL queries and selecting the best one. However, these methods rely on heuristic signals—primarily execution success (does the query run without error?) or output frequency (do multiple queries return the same result?).

The authors argue that these heuristics provide only coarse proxies for semantic correctness. As candidate sets grow, these signals often saturate and fail to discriminate between semantically valid and invalid queries, particularly when multiple incorrect queries execute successfully but return wrong results, or when semantically equivalent queries have different syntactic structures. There is a need for a learned verification mechanism that can score candidates based on task-specific semantic alignment rather than surface-level execution metrics.

Methodology: The GradeSQL Framework

To address the scarcity of labeled data required for training verifiers in structured tasks, the authors introduce GradeSQL, a scalable framework for training Outcome Reward Models (ORMs). Unlike Process Reward Models (PRMs) that score intermediate steps, ORMs assign scalar scores to final outputs. The framework operates in three stages:

Candidate Generation: Given a natural language question and a database schema, a generator LLM produces a diverse set of $N$ candidate SQL queries using stochastic decoding strategies. This diversity is crucial to expose the verifier to a wide range of correct formulations and near-miss errors.
Automated Data Labeling: Instead of manual annotation, the framework labels candidates via execution equivalence. Each candidate is executed against the database. If the result set matches that of the gold-standard query ( $R(c_j) = R(y_{gold})$ ), the candidate is labeled "Correct"; otherwise, it is labeled "Incorrect." Queries causing execution errors are discarded. This creates a large-scale, automatically labeled dataset for verifier training.
Supervised Fine-Tuning (SFT): A verifier LLM is fine-tuned (using LoRA) as a binary autoregressive classifier. The input consists of the schema, question, and a candidate query, and the model is trained to predict the probability of the "Yes" token (indicating correctness).

Inference Pipeline: At test time, the generator produces $N$ candidates. The trained ORM scores each candidate based on the probability of the "Yes" token. The candidate with the highest score is selected, replacing heuristic selection with learned semantic scoring.

Key Contributions

The paper makes three primary contributions:

ORM-based Verification for Text-to-SQL: The study adapts Outcome Reward Models as learned semantic scorers for test-time candidate selection in structured query generation, demonstrating their applicability beyond their traditional use in alignment and reasoning tasks.
Scalable Data Curation Pipeline: The authors propose GradeSQL, an automated framework that generates and labels candidate SQL queries via execution equivalence. This enables the training of task-specific ORMs without the need for expensive manual annotation.
Empirical Analysis of Test-Time Verification: A systematic evaluation across multiple open-source LLM families (including Qwen, Granite, Llama, and OmniSQL) and benchmarks (BIRD and Spider) shows that ORM-based selection consistently outperforms heuristic baselines.

Experimental Results

The approach was evaluated on the BIRD and Spider benchmarks using multiple generator models. Key findings include:

Performance Gains: ORM-based selection consistently achieved higher Execution Accuracy (EX) than Majority Voting and execution-based Best-of-N.
- On BIRD dev, ORM achieved 68.90%, a +5.01% gain over the baseline ( $N=1$ ) and outperforming Majority Voting (+3.06%) and execution-based BoN (+2.15%).
- On Spider test, ORM reached 87.47%, surpassing Majority Voting (85.47%) and execution-based BoN (85.14%).
Scaling with Candidate Sets: Unlike heuristic methods that saturate early, ORM performance continued to improve as the candidate pool size ( $N$ ) increased, approaching the oracle upper bound defined by Pass@N. This indicates ORMs effectively leverage increased test-time compute.
Robustness: The method showed consistent improvements across different generator families and model scales. The benefits of ORM-based selection persisted even when using larger generators (up to 32B parameters).
Training Dynamics:
- Dataset Imbalance: While the generated datasets were imbalanced (e.g., ~69% correct in Spider), training on balanced subsets (50/50) did not degrade performance and, in some cases, improved it while reducing training costs.
- Model Size: Increasing the ORM size beyond 7B parameters yielded only marginal and inconsistent improvements, suggesting lightweight ORMs offer a favorable performance-efficiency balance.
- Prompt Design: Prompts exposing the verifier directly to SQL structure ("SQL-Only") yielded the best results compared to instruction-style or data-only prompts.

Significance and Claims

The paper claims that learned verification provides a simple, effective, and scalable alternative to heuristic test-time selection strategies for Text-to-SQL. The significance of this work lies in demonstrating that:

Execution-based labeling is sufficient to train effective verifiers for structured tasks, removing the barrier of manual annotation.
Semantic discrimination is possible at inference time without modifying the generator or requiring additional environment interaction beyond the initial candidate generation.
ORMs are particularly effective for complex queries where heuristic signals fail, offering a principled way to handle the ambiguity and structural complexity inherent in Text-to-SQL.

The authors maintain a modest tone, acknowledging that absolute gains are moderate (2–5%) and that the approach relies on execution equivalence as a proxy for semantic correctness, which may not distinguish between semantically equivalent but syntactically different queries or reward spurious matches. The work focuses on improving the selection of candidates rather than the generation capability itself.