💬 NLP

Schema on the Inside: A Two-Phase Fine-Tuning Method for High-Efficiency Text-to-SQL at Scale

This paper presents a specialized 8B-parameter model for the Dream11 ecosystem that achieves high-precision, low-latency Text-to-SQL performance by using a novel two-phase fine-tuning method to internalize the database schema, thereby reducing input tokens by over 99% and outperforming state-of-the-art proprietary API-based models in a large-scale production environment.

Original authors: Chinmay Soni, Shivam Chourasia, Gaurav Kumar, Hitesh Kapoor

Published 2026-03-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Chinmay Soni, Shivam Chourasia, Gaurav Kumar, Hitesh Kapoor

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Heavy Backpack" Approach

Imagine you have a brilliant but very expensive librarian (a Large Language Model, or LLM) who lives in a different country. Every time you ask them a question about a specific library, you have to mail them a giant, 17,000-page encyclopedia describing every single book, shelf, and rule in that library, just so they know where to look.

This is how most companies currently build "Text-to-SQL" tools (where you ask a question in English, and the computer writes a database query).

The Cost: Since the librarian charges by the page they read, mailing them a 17,000-page book for every single question is incredibly expensive.
The Speed: It takes a long time to mail, read, and reply. The user has to wait seconds (or even minutes) for an answer.
The Result: It's slow, costly, and frustrating.

The Solution: "Baking" the Knowledge In

The team at Dream11 (a massive fantasy sports app in India) wanted to build a chatbot that could answer complex cricket questions instantly. They realized that mailing the encyclopedia every time was a bad idea.

Instead, they decided to teach the librarian the entire library by heart before they ever started answering questions. They wanted the librarian to carry the "encyclopedia" inside their own brain, so they wouldn't need to be sent the book every time.

They called this "Schema Internalization." Think of it like moving from a tourist with a map to a local who knows every shortcut.

How They Did It: The Two-Phase Training Camp

To turn a general AI into a cricket expert who knows the database by heart, they used a clever two-step training method (Fine-Tuning).

Phase 1: The "Classroom" (Learning the Rules)

First, they put the AI in a classroom with a massive textbook.

The Lesson: They showed the AI thousands of examples where the question, the full database map (schema), and the correct answer were all visible at once.
The Homework: They also gave the AI a specific test: "Here is a secret code (a database ID). Just repeat the entire database map back to me."
The Goal: This forced the AI to memorize the structure of the database, not just the answers. It was like teaching a student to memorize the periodic table and how to use it, rather than just looking it up.

Phase 2: The "Exam" (Using the Memory)

Once the AI had studied hard in Phase 1, they moved to Phase 2.

The Change: They stopped sending the 17,000-page encyclopedia. Now, they only sent a tiny note with the user's question and the "secret code" (the database ID).
The Magic: Because the AI had memorized the map in Phase 1, seeing the code triggered its memory. It instantly recalled the database structure from its own "brain" and wrote the correct answer.
The Result: The input size dropped from 17,000 tokens to less than 100. That's like shrinking a 17,000-page book down to a single sticky note.

The Results: Fast, Cheap, and Accurate

When they switched to this new method, the results were amazing:

Speed: Because they stopped mailing the giant encyclopedia, the answers came back almost instantly.
Cost: They stopped paying the expensive "per-page" fees to the third-party API. Now, they run the AI on their own servers, which is a fixed, predictable cost.
Accuracy: Surprisingly, the AI got better at answering questions.
- Old Way (The Tourist): 95.6% accuracy.
- New Way (The Local): 98.4% accuracy.

Why This Matters (The "So What?")

Think of it like this:

Before: You were hiring a taxi driver who didn't know the city, so you had to give them a GPS map for every single turn. It was slow and expensive.
After: You hired a local driver who grew up in that city. You just say "Go to the stadium," and they know exactly how to get there without needing a map.

The Takeaway:
For companies that have a specific, unchanging set of data (like a sports database, a legal code, or a product manual), it is much smarter to train a specialized AI to know that data inside out than to keep feeding a generic AI huge chunks of text every time. It saves money, speeds things up, and actually makes the AI smarter.

This paper proves that you don't always need the biggest, most expensive AI model to get the best results; sometimes, you just need the right teacher and the right training method.

1. Problem Statement

The paper addresses the critical challenges of deploying Text-to-SQL systems in large-scale, production environments, specifically for a sports intelligence application (CriQ) serving Dream11's 250+ million users.

The Bottleneck: Traditional approaches rely on Large Language Models (LLMs) via third-party APIs (e.g., Google Gemini). These require massive prompts containing the full database schema, user queries, and few-shot examples.
Consequences:
- Cost: Per-token API pricing makes processing 17,000+ token prompts prohibitively expensive at scale.
- Latency: Large context windows result in high inference latency (often >5 seconds), leading to poor user experience and query abandonment.
- Inconsistency: Initial prototypes using prompt-heavy methods achieved only ~82% execution accuracy.
Goal: Develop a self-hosted, low-latency, and cost-effective solution that maintains or exceeds the accuracy of large-context API models without the associated overhead.

2. Methodology: Two-Phase Fine-Tuning

The authors propose a novel Two-Phase Supervised Fine-Tuning (SFT) strategy designed to "internalize" the database schema into the model's parameters, eliminating the need to include the schema in the inference prompt.

Phase 1: Schema Learning and Association

The goal is to teach the model the complex logic of Text-to-SQL and explicitly associate the database structure with a unique identifier.

Dataset Construction:
- Full-Context Training (80k examples): The model is trained on standard Text-to-SQL pairs where the input prompt includes the entire database schema (17k tokens) + user query. This teaches the model the reasoning required to join tables and filter data.
- Explicit Schema Memorization (15k examples): A secondary task where the input is a prompt like "Give me full schema details for database id = 'cricket stats db v1'", and the output is the full schema text. This forces the model to encode the schema structure and vocabulary directly into its weights, linking it to a specific trigger ID.
Model Base: Qwen 3 8B (selected after benchmarking against LLaMA 3 and CodeLlama).

Phase 2: Schema Recall and Application

The goal is to force the model to rely only on its internalized knowledge.

Mechanism: The LoRA adapter trained in Phase 1 is continued on a new dataset of 150k examples.
Prompt Structure: The input prompt is drastically reduced to a minimal prompt containing only:
1. The user's natural language query.
2. The unique database ID trigger (e.g., "cricket stats db v1").
Outcome: The model recognizes the ID, retrieves the memorized schema from its internal weights, and generates the SQL query without ever seeing the schema in the prompt.

Technical Implementation

Parameter-Efficient Fine-Tuning (PEFT): Used LoRA (Low-Rank Adaptation) with rank $r=64$ and $\alpha=128$ , targeting query, key, and value projection layers.
Infrastructure: Trained on a cluster of 8x NVIDIA H100 GPUs using Distributed Data Parallel (DDP).
Deployment: Self-hosted on GPU machines using vLLM for high-throughput inference (utilizing PagedAttention).

3. Key Contributions

Novel Two-Phase Recipe: A structured curriculum that first teaches the model to understand the schema in context, then explicitly memorizes it, and finally forces recall via a minimal trigger. This bridges the gap between theoretical schema internalization and practical production deployment.
Schema Internalization at Scale: Demonstrated that an 8B parameter model can achieve state-of-the-art performance by "baking in" a complex database schema, reducing input tokens by >99%.
Production Case Study: Provided a comprehensive blueprint for transitioning from a high-cost API prototype to a self-hosted, optimized system, including data generation pipelines (using QwQ 32B for ground-truth generation) and ablation studies.

4. Results and Performance

The system was evaluated on a holdout set of 30,000 real-world user queries against a strong baseline (Gemini 2.0 Flash with a 17k-token prompt).

Metric	Proposed Model (Self-Hosted)	Baseline (Gemini 2.0 Flash + 17k Prompt)	Improvement
Execution Accuracy	98.4%	95.6%	+2.8%
Semantic Accuracy	92.5%	89.4%	+3.1%
Prompt Tokens	< 100	~17,000	> 99% Reduction
Latency/Cost	Low (Fixed infra cost)	High (Variable per-token cost)	Drastic reduction

Ablation Studies:
- Single-Phase Training: Removing Phase 1 (full context) dropped accuracy to 74.96%, proving the necessity of initial context training.
- No Explicit Memorization: Removing the 15k schema memorization examples dropped execution accuracy to 87.2%, proving that implicit learning is insufficient for reliable recall.
- Model Scale: The 8B parameter model offered the optimal trade-off; smaller models failed to reach the required accuracy threshold.

5. Significance and Impact

Economic Viability: Shifts the cost model from variable, high per-query API fees to a fixed, predictable infrastructure cost, making high-volume Text-to-SQL economically feasible for consumer apps.
User Experience: Reduces latency from seconds to sub-second levels, preventing user abandonment.
Reliability: Higher semantic accuracy builds user trust in AI-driven analytics.
Generalizability: The approach is applicable to any domain with a large but static context (e.g., API documentation, legal policy manuals, game rules) where schema internalization can replace massive context windows.
Limitation: The method assumes a static schema; significant database changes would require re-fine-tuning.

In conclusion, the paper demonstrates that for specialized, high-throughput applications, fine-tuning smaller models to internalize domain knowledge is a superior strategy to relying on massive, prompt-heavy general-purpose LLMs.