💬 NLP

FD-NL2SQL: Feedback-Driven Clinical NL2SQL that Improves with Use

FD-NL2SQL is a feedback-driven clinical NL2SQL assistant for oncology databases that decomposes natural language queries into sub-questions, retrieves expert exemplars, and synthesizes executable SQL while continuously improving its performance through clinician edits and automated logic-based SQL augmentation.

Original authors: Suparno Roy Chowdhury, Tejas Anvekar, Manan Roy Choudhury, Muhammad Ali Khan, Kaneez Zahra Rubab Khakwani, Mohamad Bassam Sonbol, Irbaz Bin Riaz, Vivek Gupta

Published 2026-04-20

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Suparno Roy Chowdhury, Tejas Anvekar, Manan Roy Choudhury, Muhammad Ali Khan, Kaneez Zahra Rubab Khakwani, Mohamad Bassam Sonbol, Irbaz Bin Riaz, Vivek Gupta

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a doctor trying to find the perfect clinical trial for a patient. You know exactly what you need: "Show me trials for lung cancer that started after 2018, using a specific drug, with a follow-up period longer than two years."

In the old days, to get this answer, you'd have to speak a secret, complex language called SQL (Structured Query Language). It's like trying to ask a librarian for a book, but instead of saying "I want a mystery novel," you have to know the exact shelf number, the book's barcode, and the specific code for the "mystery" genre. If you get one digit wrong, the librarian (the database) gives you nothing or, worse, the wrong book.

This is where FD-NL2SQL comes in. Think of it as a super-smart, self-improving translator that sits between you and the database.

Here is how it works, broken down into simple steps:

1. The "Deconstruction" Chef

When you ask your question in plain English, the system doesn't just guess the answer. Instead, it acts like a head chef breaking down a complex recipe.

Your Question: "Show me lung cancer trials from 2018+ with Drug X."
The System's Thought Process: It breaks this down into tiny, simple ingredients:
1. Filter by: Cancer Type = Lung.
2. Filter by: Year >= 2018.
3. Filter by: Drug = Drug X.
  This ensures the system doesn't get overwhelmed and misses a crucial detail.

2. The "Memory Bank" Librarian

Before it writes the final code, the system checks its Expert Memory Bank. It looks for past questions that are similar to yours.

Analogy: Imagine you ask, "How do I bake a chocolate cake?" The system doesn't start from scratch; it looks at a card in its library that says, "How to bake a vanilla cake," and another that says, "How to bake a strawberry cake." It uses these proven recipes as a guide to ensure it doesn't invent a weird, inedible cake.
In this system, these "recipes" are SQL queries that experts have already verified are correct.

3. The "Translator" and the "Safety Check"

The system combines your broken-down question with the expert recipes to write the final SQL code.

The Safety Check: Before showing you the result, it runs a quick "test drive." It checks: "Does this code actually work? Does it return any results? Did I accidentally try to delete the whole database?" (It's programmed to only read data, never delete it).

4. The "Magic Loop" (How it gets smarter)

This is the coolest part. The system is alive.

Scenario: You ask a question, the system gives you a SQL query, but you notice it missed a detail. You fix the code yourself.
The Upgrade: The system says, "Thank you! I learned something new." It saves your corrected version into its Expert Memory Bank.
The "Mutation" Trick: To get even smarter without needing humans to write more examples, the system takes your approved code and makes tiny, safe changes (like changing "2018" to "2019" or "Lung" to "Breast"). If that new version works, it asks a second AI to write a new question for it. Now, the system has learned two new things from just one human correction!

Why does this matter?

For Doctors: You don't need to be a computer programmer to explore millions of medical trials. You just talk to the system like a human.
For Science: It speeds up research. Instead of waiting days for a data analyst to write a query, a doctor can get answers in seconds.
For Safety: Because it shows you how it got the answer (the breakdown, the examples it used, the final code), you can trust it. It's not a "black box" guessing; it's a transparent assistant.

In short: FD-NL2SQL is like having a personal research assistant who speaks both "Doctor" and "Computer," remembers every successful search you've ever done, and gets better at its job every time you interact with it.

1. Problem Statement

Clinical researchers and oncologists frequently need to perform ad-hoc, multi-constraint queries on structured clinical trial databases (e.g., filtering by biomarkers, trial phases, endpoints, and timeframes). However, accessing this data is hindered by:

High Barrier to Entry: Querying requires SQL expertise and deep knowledge of complex, multi-table schemas (involving eligibility, interventions, and disease ontologies).
Limitations of Existing Tools: Keyword-based registry searches lack structured filtering capabilities, while Business Intelligence dashboards are too rigid for exploratory research.
Fragility of General NL2SQL: Standard Large Language Model (LLM) approaches often fail in specialized domains like oncology. They tend to generate "hallucinated" SQL (incorrect columns, joins, or values) because they lack schema-aligned grounding and domain-specific constraints, leading to clinically implausible or empty results.

2. Methodology: FD-NL2SQL Architecture

The authors propose FD-NL2SQL, a feedback-driven system designed specifically for SQLite-based oncology databases. It operates on a modular pipeline consisting of three core stages, followed by a self-evolving feedback loop.

A. Core Pipeline

Schema Grounding: Before generation, the system introspects the database to build a schema dictionary (tables, columns, types, join keys). This context is injected into prompts to ensure structural validity.
Decomposition-Retrieval:
- Decomposition: An LLM breaks the user's natural language question into atomic, schema-aligned sub-questions (predicates), where each sub-question targets a specific column, operator, or value.
- Retrieval: For each sub-question, the system retrieves semantically similar, expert-verified NL2SQL exemplars from a bank using Sentence-BERT embeddings. It extracts "WHERE-pattern hints" from these exemplars to guide the synthesis.
Retrieval-Guided Synthesis: A second LLM synthesizes the final executable SQL. It is conditioned on the original question, the decomposed predicates, and the retrieved exemplar bundles. The model is instructed to use retrieved SQL as structural templates while avoiding irrelevant literal copying.
- Post-Processing: The output undergoes validity checks (read-only policy, schema existence, timeout) before execution.

B. Feedback Loop & Self-Evolution

The system is designed to improve continuously through two update signals:

Expert Approval: If a clinician edits and approves a generated SQL query, the corrected (Question, SQL) pair is added to the exemplar bank, improving future retrieval neighborhoods.
Logic-Based SQL Augmentation (Self-Training):
- Mutation: The system takes an approved SQL query and applies a single atomic mutation (e.g., changing an operator from = to >=, substituting a column, or adjusting a value threshold).
- Validation: The mutated query is executed. It is retained only if it runs successfully and returns a non-empty result.
- Back-Translation: A separate LLM generates a natural language question and predicate decomposition corresponding to the valid mutated SQL.
- Expansion: This new (Question, SQL) pair is added to the exemplar bank, automatically expanding coverage without additional manual annotation.

3. Key Contributions

Schema-Aware Decomposition: A strategy that breaks complex clinical questions into atomic predicates aligned with the database schema, significantly reducing omission errors and schema drift.
Exemplar-Guided Synthesis: A retrieval-augmented pipeline that grounds LLM generation in expert-verified, domain-specific query patterns rather than relying solely on generic in-context learning.
Feedback-Driven Self-Improvement: A novel mechanism where clinician interactions and logic-based SQL mutations automatically expand the training data (exemplar bank), creating a "living" assistant that adapts to real-world usage patterns.

4. Experimental Results

The system was evaluated on a dataset of 500 expert-authored seed questions (expanded to 1,500 via controlled variants) from the IOTOX1 oncology trial database.

Baselines: Compared against Zero-Shot, Few-Shot, Chain-of-Thought (CoT), and a reasoning-finetuned model (SQL-R1) using various LLM backbones (Gemma3, Qwen3, GPT-5 variants).
Performance: FD-NL2SQL consistently outperformed all baselines across execution-based metrics:
- eEM (Exact Match): Significant improvements in returning the exact gold result set.
- eF1 & AST: Higher scores indicate better partial correctness and structural adherence to the schema.
- Robustness: The system narrowed the performance gap between smaller/cheaper models and larger models, suggesting that decomposition and retrieval guidance can compensate for model size limitations.
Failure Modes: Baseline models frequently dropped predicates or selected incompatible columns. FD-NL2SQL mitigated this by enforcing schema alignment and using retrieved templates.

5. Significance and Impact

Clinical Utility: The system bridges the gap between oncology expertise and structured data access, allowing clinicians to perform complex, multi-constraint exploratory queries without SQL training.
Safety and Trust: By outputting executable, auditable SQL with a transparent trace (decomposition $\to$ retrieval $\to$ synthesis), the system supports verification and reduces the risk of silent failures common in medical AI.
Scalability: The self-evolving feedback loop allows the system to adapt to schema changes and new clinical queries over time without requiring continuous manual re-annotation, making it a sustainable tool for dynamic research environments.

In conclusion, FD-NL2SQL demonstrates that combining schema-aligned decomposition, retrieval augmentation, and interactive feedback loops creates a robust, domain-specific NL2SQL system that is superior to general-purpose prompting for high-stakes biomedical applications.