Schema-First Retrieval: Embedding Catalogs for Natural Language Analytics
This paper introduces Schema-First Retrieval, a novel retrieval layer that embeds and indexes diverse catalog metadata (tables, columns, metrics, relationships, and query history) to significantly improve schema selection accuracy and reduce SQL execution errors in enterprise text-to-SQL systems by treating catalog context as a first-class retrieval problem rather than a prompt formatting detail.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to ask a very smart, but slightly confused, librarian for a specific book.
The Problem: The Wrong Bookshelf
In the old way of doing things (what the paper calls "raw schema prompting"), you walk up to the librarian and say, "I want to know about revenue." But the librarian doesn't have a bookshelf labeled "Revenue." Instead, they have thousands of shelves with confusing names like tbl_fin_2024_q3 or amt_net_rev_lcl.
Because the librarian doesn't know which shelf you mean, they might grab a book from the wrong shelf. They might write a perfect sentence (SQL code) based on that wrong book, but the answer will be useless. In big companies, the "library" (the database) is so huge and messy that the librarian often grabs the wrong context before they even start writing the answer.
The Solution: Schema-First Retrieval
This paper introduces a new system called Schema-First Retrieval. Instead of asking the librarian to guess from the whole library, this system acts like a super-smart catalog card system that finds the exact right pieces of information before the librarian even sees the question.
Here is how it works, using simple analogies:
1. The Five Types of "Catalog Cards"
Instead of just looking at book titles, this system creates five specific types of "cards" for every piece of information in the library:
- Table Cards: These are like the "Section" signs (e.g., "This section is about Schools").
- Column Cards: These are like the specific "Chapters" or "Rows" inside the book (e.g., "This column lists the Grade Levels").
- Metric Cards: These are like the "Glossary" or "Business Dictionary." If you ask for "Churn," this card explains that it actually means "people who stopped using the service," even if the database calls it something else.
- Relationship Cards: These are like the "Cross-References" that tell you how two different books connect (e.g., "To find the total sales, you must join the 'Orders' book with the 'Customers' book").
- History Cards: These are like a "Log of Past Questions." If someone asked a similar question last week, this card reminds the system, "Hey, we found the answer to this last time using these specific pages."
2. The Search Process (The "Retrieval Layer")
When you ask a question, the system doesn't just dump the whole library on the librarian's desk. It does a three-step search:
- The Broad Net: It uses a "semantic net" (AI embeddings) to catch anything that sounds like your question, even if the words are different.
- The Expert Review (Reranking): A second, more careful AI looks at the caught items. It realizes, "Wait, 'Revenue' usually lives in this specific table, not that one," and re-orders the list to put the best matches at the top.
- The Memory Check: It checks the "History Cards." If the company usually asks about "Revenue" in a specific way, it prioritizes that path.
3. The "Security Guard" (Access Control)
The paper emphasizes that the librarian (the AI) shouldn't be trusted to remember security rules. Instead, there is a Security Guard standing at the door.
- Before the librarian sees the books, the guard checks: "Does this user have permission to see the 'Salary' section?"
- If not, the guard physically removes those pages.
- If the librarian tries to write a sentence using a forbidden page, the guard stops the sentence before it's sent. This is done by strict rules, not by asking the librarian to "please be careful."
4. The Results: Fewer Mistakes
The paper tested this system on three different "libraries" (datasets):
- Finding the Right Pages: The system found the correct tables and columns much more often than the old methods. For example, on one test, it found the right table 96% of the time, compared to much lower rates for older methods.
- Fewer Broken Sentences: When the system used this "catalog-first" approach, the number of errors in the final code dropped by 2.5 times.
- Why? Often, the old system failed because it didn't know how to write a column name correctly (like forgetting to put quotes around a name with a space in it). Because the new system pulls the "Column Card" which already has the correct formatting, the librarian just copies it perfectly.
The Big Takeaway
The paper argues that Natural Language Analytics (asking questions in plain English) shouldn't be treated as a "writing" problem. It is actually a searching problem.
If you give a smart AI the wrong map, it will write a perfect sentence leading to the wrong place. By building a better map (the catalog) and finding the right path before the AI starts writing, you get reliable, safe, and accurate answers. The AI isn't the one discovering the warehouse; it's just writing the report based on the curated, safe, and correct documents the system found for it.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.