SQaLe: A Large Text-to-SQL Corpus Grounded in Real Schemas

This paper introduces SQaLe, a large-scale semi-synthetic text-to-SQL dataset containing over 517,000 high-quality triples derived from 135,875 real-world database schemas, designed to overcome current data limitations and advance model generalization through realistic schema variability and diverse query patterns.

The Gist

Imagine you have a massive library of books, but instead of reading them, you want to ask a librarian specific questions about the data inside them, like "How many books were published in 1990?" or "Which author has the most bestsellers?"

In the world of computers, this is called Text-to-SQL. You type a question in plain English, and the computer translates it into a special code (SQL) to dig up the answer from a database.

The problem? The computers (AI models) we have today are like brilliant students who have only read a few small storybooks. They struggle when asked to navigate a massive, messy, real-world library with thousands of interconnected shelves and confusing labels. They need more practice, but the "practice books" available so far are too simple, too small, or too fake.

Enter SQALE.

What is SQALE?

Think of SQALE as a massive, super-realistic training gym for these AI librarians.

The researchers built a dataset containing over 517,000 practice questions paired with 135,000 different database blueprints (schemas). But here's the magic: they didn't just make up fake libraries. They started with a pile of real database blueprints from the real world (called "SchemaPile") and used a smart AI assistant to expand them, creating new, complex, and realistic scenarios.

How did they build it? (The Construction Site Analogy)

Imagine you are an architect who wants to train a robot to build houses. You don't just give the robot a drawing of a cardboard box. You give it a pile of real blueprints for garages, then ask the robot to imagine adding a second floor, a garage, and a pool, while keeping the style consistent.

The SQALE team did exactly this in three steps:

1. The Foundation (Real Blueprints): They took real database structures from the wild. These weren't perfect, clean diagrams; they were messy, like real life, with weird naming conventions and missing connections.
2. The Expansion (The "What If" Game): They used a powerful AI (Qwen3) to act as an "Architect." They told the AI: "Take this small garage blueprint and expand it into a massive mansion with 100 rooms, but keep the style the same. Add some tricky connections between rooms." This created huge, complex databases that didn't exist before but felt exactly like real ones.
3. The Questions (The Tour Guide): Once the "mansion" was built, they asked the AI to act as a confused tourist. "How do I get from the kitchen to the attic?" or "Show me all the rooms with blue doors." The AI then generated the correct "map" (SQL code) to answer that question.

Why is this a big deal?

Previous training sets were like toy train sets. They had a few tracks and a few trains. They were clean and predictable.

• Spider 2.0 (a previous benchmark) was like a model village. Better, but still small.
• SQALE is like a real, sprawling metropolis.

Here is why SQALE is special:

• It's Messy (in a good way): Real databases have missing links and weird names. SQALE includes these "imperfections" so the AI learns to handle real-world confusion, not just perfect textbook examples.
• It's Huge: It has over 13 million "connections" (foreign keys) between data points. This forces the AI to learn how to navigate complex relationships, not just simple lookups.
• It's Diverse: Some questions are easy ("What's the weather?"), and some are incredibly hard ("Find the average salary of employees who worked on projects that failed, grouped by department, excluding those hired before 2010").

The Goal

The researchers believe that if you feed an AI more data, it gets smarter (this is known as "scaling laws"). By giving these AI models a "university education" based on SQALE instead of a "high school education" based on older datasets, they hope to create models that can finally handle the complex, messy databases used by actual businesses, hospitals, and governments.

In short: SQALE is the ultimate "flight simulator" for AI. Instead of practicing in a calm, empty sky, it's training in a stormy, crowded airspace with real traffic, so when it finally flies a real plane (answers a real business question), it won't crash.

Technical Summary

1. Problem Statement

The field of Text-to-SQL (converting natural language queries into SQL) has seen rapid progress due to Large Language Models (LLMs). However, the development of generalizable, robust models is currently bottlenecked by the lack of large-scale, high-quality datasets that possess sufficient:

• Schema Complexity: Existing benchmarks (e.g., Spider 1.0, BIRD, Spider 2.0) often feature small schemas with limited table counts and column density compared to real-world enterprise databases.
• Diversity: Current datasets lack the variety in naming conventions, normalization levels, and foreign key integrity found in production environments.
• Scale: The volume of available data is orders of magnitude smaller than standard NLP corpora, preventing the application of established scaling laws for training large foundation models.
• Realism: Fully synthetic datasets often rely on artificial schemas that fail to capture the ambiguity and structural idiosyncrasies of real-world data.

2. Methodology: The SQALE Generation Pipeline

The authors introduce SQALE, a semi-synthetic dataset generation pipeline designed to bridge the gap between synthetic scale and real-world realism. The pipeline consists of three main stages:

A. Foundation: SchemaPile

The process begins with SchemaPile, a collection of 22,989 real-world database schemas. These serve as the "ground truth" for structural realism.

B. Schema Extension (Scaling)

Using an LLM (specifically Qwen3 30B), the pipeline iteratively extends existing real schemas to increase complexity while preserving their original style.

• Target Distribution: Schemas are expanded to include 5 to 25 synthetic tables, targeting a median of 91 tables and 435 columns per schema.
• Constraints: The extension process maintains the original naming conventions, normalization levels, and foreign key (FK) logic.
• FK Handling: For schemas originally lacking FKs, new FKs are added during extension; however, for schemas inferred to have no FKs, added FKs are removed to maintain data integrity consistency (C3).
• Distribution: The final schema sizes follow a smooth gamma-like probability distribution, with a mode of 100 tables and a maximum of 350.

C. Question and Query Synthesis

Once schemas are extended, the pipeline generates (Question, Schema, Query) triples:

1. Question Generation: Natural language questions are synthesized using examples from BIRD and Spider 2.0 as few-shot prompts. The LLM is instructed to generate questions requiring a specific number of joins (targeting a median of 4 joins per query) to ensure diverse intent (lookup, aggregation, filtering, ranking).
2. SQL Generation: The ReFoRCE framework is used to generate multiple SQL candidates for each question.
   • Voting: A voting mechanism selects the best candidate.
   • Validation: The selected SQL is executed against the generated schema to ensure executability.
   • Filtering: Outliers based on prompt/query length and execution errors are removed.
3. Quality Control: A manual random check of 300 samples confirmed the dataset's quality and correctness.

3. Key Contributions

1. A Principled Generation Pipeline: A novel workflow that combines real-world schema sampling, LLM-driven schema extension, and guided question/SQL synthesis to ensure both structural realism and linguistic diversity.
2. The SQALE Dataset: A massive corpus containing 517,676 validated triples derived from 135,875 unique relational database schemas.
3. Comprehensive Characterization: An in-depth analysis demonstrating that SQALE surpasses existing benchmarks in schema complexity, foreign key relations, and query diversity, establishing a new standard for Text-to-SQL research.

4. Results and Dataset Characteristics

The SQALE dataset significantly outperforms existing benchmarks (Spider 2.0, BIRD, SynSQL, EHRSQL) in several key metrics:

Metric	SQALE	Spider 2.0	BIRD	SynSQL
# Schemas	135,875	236	80	16,575
Median Tables/Schema	91.0	7.0	5.0	10.0
Median Columns/Schema	435.0	89	39	72
Total Foreign Keys	13.2 Million	0	526	159,547
Join Complexity	76.1% of queries use joins	72.0%	76.2%	89.4%
Query Diversity	High (Aggregations, Nested, Set Ops)	High	High	Moderate

• Realism: Unlike fully synthetic datasets, SQALE schemas retain the "messiness" of real data, including inconsistent naming, mixed casing, and varying normalization depths.
• Query Complexity: The dataset features a balanced distribution of query types, with 76.1% involving joins and a high prevalence of complex operators (e.g., WITH, UNION, nested subqueries).
• Scale: While SynSQL has more total queries, SQALE offers vastly more complex and realistic schemas, which is the primary bottleneck for model generalization.

5. Significance and Impact

• Enabling Scaling Laws: SQALE provides the necessary data volume and diversity to test and apply scaling laws to Text-to-SQL models, potentially allowing for the training of larger, more capable foundation models.
• Bridging the Gap: It addresses the "reality gap" where models trained on clean, small benchmarks fail when deployed on complex, enterprise-grade databases.
• Research Utility: The dataset is designed to support both the training of small-scale specialized models and the fine-tuning of large LLMs.
• Availability: The dataset is publicly available on Hugging Face, fostering reproducibility and further advancement in the field.

In conclusion, SQALE represents a paradigm shift from "small, clean" benchmarks to "large, realistic" corpora, providing the infrastructure needed to solve the Text-to-SQL problem in real-world enterprise settings.