AmbiSQL: Interactive Ambiguity Detection and Resolution for Text-to-SQL

1. Problem Statement

Text-to-SQL systems, particularly those powered by Large Language Models (LLMs), have shown significant promise but remain error-prone due to query ambiguity.

• The Core Issue: Natural language queries often contain ambiguous phrases that lead to misalignment between user intent and the generated SQL. This results in syntactically valid but semantically incorrect queries.
• Limitations of Current Approaches:
  • Fully Automated Methods: Often resolve ambiguities without user feedback, leading to divergence from true user intent.
  • Scope Gaps: Existing methods primarily focus on schema-linking errors (mapping words to tables/columns) but neglect ambiguities arising from LLM reasoning (e.g., temporal granularity, external knowledge requirements).
  • Taxonomy Deficiencies: Without a structured taxonomy, systems generate excessive candidate interpretations, causing computational overhead and poor generalization.

2. Methodology

AmbiSQL introduces an interactive, human-in-the-loop system that detects ambiguities and guides users to clarify their intent through multiple-choice questions.

A. Fine-Grained Ambiguity Taxonomy

The authors propose a comprehensive taxonomy classifying ambiguities into two dimensions:

1. Database-sourced Ambiguity: Arising from unclear references to the database schema or content.
   • AmbiSchema: Unclear table/column selection (e.g., "oldest user" = age vs. registration date).
   • AmbiValue: Values not matching stored literals (e.g., "New York City" vs. "NYC").
   • AmbiIntent: Missing operators (e.g., "by date" = sort vs. filter vs. group).
2. LLM-sourced Ambiguity: Arising from unclear requirements for reasoning or external knowledge.
   • AmbiSource: Unclear data origin (DB vs. LLM inference).
   • AmbiContext: Missing context for reasoning (e.g., unspecified currency for exchange rates).
   • AmbiFallacy: Contradictory knowledge assumptions (e.g., events that never happened).
   • AmbiRef: Underspecified spatial/temporal constraints (e.g., "end of Vietnam War" = specific date vs. year).

B. System Architecture

AmbiSQL operates via a two-stage pipeline:

1. Stage 1: Ambiguity Identification

   • Detection: Uses pre-trained LLMs with in-context learning. Prompts include explicit definitions, the taxonomy, and representative examples to identify ambiguous phrases and classify them.
   • Generation: Creates targeted Clarification Questions (CQs). For database issues, it offers schema snippets; for reasoning issues, it asks for context. Options include "Abstain" (skip) and "Others" (free text).

2. Stage 2: Iterative Refinement

   • Preference Update: User selections are stored in a tree-like structure based on the taxonomy. Conflicting preferences are automatically updated to reflect the latest intent.
   • Query Refinement: Integrates user clarifications and additional constraints into the original query.
   • Re-verification: The refined query is re-scanned for new ambiguities introduced during rewriting. Once clear, it is passed to the backend Text-to-SQL engine.

3. Key Contributions

• Novel Taxonomy: The first comprehensive classification distinguishing between database-mapping errors and LLM-reasoning errors, covering a broader spectrum of real-world ambiguities.
• Interactive Framework: A system that moves beyond "one-shot" generation to an iterative clarification loop, ensuring the final SQL aligns with user intent.
• Integration with Commercial Backend: AmbiSQL is integrated with XiYan-SQL, a commercial Text-to-SQL system, demonstrating practical applicability.
• Benchmark Dataset: A curated set of 40 ambiguous queries from the BIRD and TAG benchmarks, annotated with ambiguity types and ground-truth intents, available for public evaluation.

4. Results and Demonstration

The paper presents a demonstration of AmbiSQL at SIGMOD '26:

• Interactive UI: Features three panels: User Input, Ambiguity Resolution (with dropdown CQs), and SQL Generation.
• Comparative Analysis: The system compares SQL generated with and without ambiguity resolution.
  • Example: For the query "drivers born after the end of the Vietnam War," the system clarifies the specific date (e.g., 1973-03-29 vs. 1975-04-30) and the column mapping for "ranked 2" (position vs. rank).
• Traceability: The system maintains a persistent log of interactions, showing exactly how each ambiguity was resolved step-by-step.
• Outcome: The clarified queries result in SQL that more faithfully reflects user intent, correcting errors in filtering conditions, column mappings, and temporal logic that the raw LLM would have missed.

5. Significance

• Robustness: AmbiSQL addresses a critical failure mode in LLM-based Text-to-SQL systems, significantly improving reliability in real-world deployments where queries are rarely perfectly precise.
• Evaluation Standard: By providing a dataset of annotated ambiguous queries, it offers a new benchmark for evaluating the robustness of Text-to-SQL systems against ambiguity.
• User-Centric Design: It empowers non-expert users to correct system misunderstandings through intuitive multiple-choice interactions, bridging the gap between natural language vagueness and strict SQL syntax.
• Future Direction: The work suggests that future Text-to-SQL research must account for both schema linking and complex reasoning ambiguities, moving toward more interactive and adaptive systems.