💬 NLP

Cheaper, Better, Faster, Stronger: Robust Text-to-SQL without Chain-of-Thought or Fine-Tuning

This paper introduces "N-rep," a cost-efficient text-to-SQL method that achieves state-of-the-art performance on the BIRD benchmark at just $0.039 per query by leveraging multiple schema representations to ensure robustness without relying on expensive Chain-of-Thought reasoning or fine-tuning.

Original authors: Yusuf Denizay Dönder, Derek Hommel, Andrea W Wen-Yi, David Mimno, Unso Eun Seo Jo

Published 2026-04-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yusuf Denizay Dönder, Derek Hommel, Andrea W Wen-Yi, David Mimno, Unso Eun Seo Jo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to give a very specific, complex order to a chef who speaks a different language. You want to ask for a dish based on a menu (the database), but the menu is huge, and the chef gets confused easily if you describe the ingredients in a way they don't like.

This paper is about a new, cheaper, and smarter way to talk to these "chef" computers (AI models) to get them to write the right code (SQL) to pull data from a database.

Here is the breakdown of their idea, N-rep, using simple analogies:

The Problem: The "Expensive Genius" vs. The "Confused Chef"

Currently, the best way to get an AI to write database code is to ask a super-smart, expensive AI to "think out loud" (Chain-of-Thought) or to hire a team of experts to train a custom AI (Fine-Tuning).

The Cost: This is like hiring a team of 100 consultants to solve a simple math problem. It costs a lot of money (up to $0.46 per question) and takes a long time.
The Issue: Even the smartest chefs get confused if you describe the menu in a format they don't prefer. If you say "Apples, Bananas, Oranges," they get it. If you say "Fruit: Red, Yellow, Orange," they might get lost.

The Solution: The "N-rep" Strategy (Many Perspectives)

The authors realized that instead of hiring a super-expensive chef or training a new one, you can just ask a cheaper, smaller chef to look at the menu in different languages at the same time.

They call this N-rep (N-representations).

The "Many Angles" Approach:
Imagine you are trying to describe a house to a real estate agent.
- Angle 1: You describe it as a list of rooms (Tables).
- Angle 2: You describe it as a blueprint (Schema).
- Angle 3: You describe it as a list of furniture (Columns).
- Angle 4: You describe it as a story.
The AI might miss a detail in Angle 1, but catch it in Angle 3. By feeding the AI the same database information in four or five different formats at once, you cover all the bases.
The "Group Vote" (Consistency):
The system asks the AI to write the code based on each of these different descriptions.
- If the AI writes the same code 4 times out of 5, no matter how the menu was described, you know that answer is probably right.
- If the answers are all different, the system uses a smart "tie-breaker" (a second, quick check) to pick the best one.

Why This is a Big Deal

Cheaper: Instead of paying for a "super-genius" AI to think deeply, they use a "smart-but-cheap" AI and just ask it to look at the problem from different angles.
- The Paper's Claim: They cut the cost from 46 cents per question down to 3.9 cents. That's like going from ordering a fancy steak dinner to getting a great sandwich for a fraction of the price.
Faster: They don't need to wait for the AI to "think" step-by-step (Chain-of-Thought). They just get the answers and pick the best one.
Better: Surprisingly, this cheap method actually performed better than the expensive methods on their tests (the BIRD benchmark). It got the right answer more often.

The "Secret Sauce"

The paper claims that AI models are actually quite "sensitive" to how information is written. They aren't always "reasoning" deeply; they are often just matching patterns. By giving them the same information in different patterns (formats), the system becomes robust. It's like having a safety net; if the AI trips on one way of looking at the data, it catches itself with another way.

Summary

The authors built a system that asks a cheaper AI to look at a database problem from multiple different "perspectives" (formats) simultaneously. By voting on the best answer among these different perspectives, they get results that are more accurate, faster, and 10 times cheaper than the current state-of-the-art methods that rely on expensive "thinking" or custom training.

In short: Don't hire a super-expensive genius to think hard. Instead, hire a few regular smart people, show them the problem in different ways, and let them vote on the answer. It works better and costs a lot less.

1. Problem Statement

Current State-of-the-Art (SOTA) Text-to-SQL approaches rely heavily on computationally expensive techniques to achieve high accuracy:

Chain-of-Thought (CoT) Self-Consistency: Generates multiple candidate queries via reasoning paths (often requiring 20–100+ LLM calls per query) to select the most consistent answer.
Fine-Tuning: Requires extensive data labeling and model training, incurring costs in the thousands of dollars.
Cost vs. Performance: These methods can cost up to $0.46 per query (e.g., CHASE, CHESS) and often require large models. The authors argue that these approaches are over-engineered for specific tasks and fail to address the fundamental sensitivity of Large Language Models (LLMs) to schema representation formats.

2. Methodology: The N-rep Framework

The authors propose "N-rep" (N-representation) Consistency, a framework that generates robust SQL queries by leveraging multiple text representations of the database schema rather than multiple reasoning paths. The framework operates in three stages:

A. Schema Linker

Instead of a single schema input, the database schema is converted into $n$ different text representations (e.g., DDL, MAC-Schema, M-Schema, JSON, Python/SQLAlchemy).

Process: For each representation, the system predicts relevant tables and columns.
Filtering Levels: To increase diversity, three filtering strategies are applied to the schema before generation:
1. No Filtering: Full schema.
2. Table-Only: Selected tables with all their columns.
3. Full Filtering: Selected tables with only the predicted relevant columns.
Goal: This creates a diverse set of "schema linking candidates" without relying on temperature sampling or CoT reasoning.

B. Candidate Generator

An LLM generates SQL candidates using the diverse schema representations produced in the previous stage.

Input: A specific schema representation (filtered or unfiltered), the Natural Language Question (NLQ), and 3 few-shot examples.
Efficiency: Unlike CoT methods that generate long reasoning chains, N-rep generates direct SQL queries, significantly reducing token usage per candidate.

C. Candidate Selector

A confidence-aware two-stage selection strategy is used to choose the final SQL:

Regular Voting: Candidates are voted on based on execution results or string matching.
Confidence Thresholding: If the voting distribution indicates high confidence (e.g., a clear majority), the top candidate is selected immediately.
LLM-Based Tie-Breaking: If confidence is low (e.g., a tie between top candidates), a secondary LLM-based pairwise comparison (similar to CHASE-SQL) is triggered only for those specific low-confidence cases. This minimizes unnecessary LLM calls.

3. Key Contributions

Cost Efficiency: N-rep achieves SOTA-level performance at a fraction of the cost. It reduces the cost per query from ~$0.46 to $0.039 (an ~8.7x reduction).
No Fine-Tuning or CoT: The method achieves robustness without model fine-tuning or expensive Chain-of-Thought reasoning, relying instead on schema representation diversity.
Model Agnosticism: The approach works effectively with smaller, cheaper open-source models (e.g., Qwen3-8B), narrowing the performance gap between small and large models.
Robustness: By varying schema formats, the method mitigates the "input sensitivity" of LLMs, where a single schema format might cause hallucinations or linking errors.

4. Experimental Results

The framework was evaluated on the BIRD and SPIDER benchmarks.

Performance on BIRD (Dev Set):

Execution Accuracy (EX): N-rep achieved 69.25%, outperforming the Gemini 2.5 Pro baseline (66.82%) and the CHESS method (68.31%).
Cost Comparison:
- N-rep: $0.039/query (8 LLM calls on average).
- Gemini 2.5 Pro: $0.047/query.
- CHASE: $0.44/query (100+ calls).
- CHESS: $0.46/query.
Small Model Gains: When applied to Qwen3-8B, N-rep improved EX from 44.98% to 57.04% (+12 points), demonstrating that schema diversity compensates for model size limitations.

Performance on SPIDER (Test Set):

N-rep achieved 87.0% EX, outperforming DAIL-SQL (86.6%) and CHESS (87.2%), and coming very close to the top-performing MiniSeek (91.2%).

5. Significance and Conclusion

The paper challenges the prevailing trend of using massive CoT reasoning and fine-tuning for Text-to-SQL. The authors demonstrate that domain-specific knowledge (understanding how schema representation affects LLM performance) is more effective than generic reasoning strategies.

Key Insight: LLMs are highly sensitive to how database schemas are formatted. By providing multiple representations, the system creates a "consistency" effect similar to self-consistency but through input diversity rather than reasoning diversity.
Impact: N-rep offers a practical, scalable, and cost-effective solution for enterprise Text-to-SQL applications, proving that "cheaper" models with smart engineering can outperform expensive, reasoning-heavy approaches.

Limitations: The current implementation relies on manually selected schema representations. Future work aims to automate the generation of diverse schema formats and optimize the confidence-aware voting policy for arbitrary candidate counts.