Technical Summary: A Semantic-Layer-Mediated Agent for NL2SQL over Heterogeneous Enterprise Databases

1. Problem Statement

Translating natural language questions into executable SQL (NL2SQL) remains significantly more challenging in enterprise environments than on academic benchmarks. While models perform well on datasets like Spider and BIRD, where schemas are small and questions map directly to single queries, enterprise databases present distinct hurdles:

• Scale and Opacity: Schemas contain hundreds of physical tables with cryptic column names and lack self-descriptive metadata.
• Complexity: Analytical questions often require nested aggregations, time-windowed logic, multi-table joins, and dialect-specific functions.
• Brittleness: Directly prompting Large Language Models (LLMs) with raw schema text forces the model to simultaneously solve grounding (identifying relevant tables/columns) and composition (assembling valid SQL). A single error in column naming or join logic causes the entire query to fail, with no structured mechanism for recovery.

Existing approaches that rely on schema pruning or multi-agent decomposition still operate directly on physical schemas, leaving the model to hallucinate physical identifiers or recall names from flat dumps.

2. Methodology

The paper proposes spider2-daquv-quvi, an architecture that decouples schema grounding from SQL composition by interposing a curated semantic layer between the LLM and the database.

Core Architecture

The system operates through four tiers:

1. Orchestration Tier: Manages benchmark instances, handles parallel execution, and routes queries based on database prefixes (SQLite, BigQuery, Snowflake).
2. QUVI NL2SQL Service: Hosts the LLM agent and the semantic layer.
3. SMQ-to-SQL Engine: A deterministic compiler that translates intermediate queries into dialect-correct SQL.
4. Multi-Backend Executor: Executes final queries against the target database.

The Semantic Layer and SMQ

Instead of exposing raw tables, the system wraps physical tables in Semantic Models. These models expose:

• Dimensions: Grouping/filtering keys with human-readable descriptions.
• Measures: Aggregation targets.
• Metrics: Predefined aggregations or derived calculations.
• Join Graph: Typed edges defining how models connect, including necessary transformations (e.g., TRIM).

The agent does not write SQL directly. Instead, it generates a Semantic Model Query (SMQ), a compact JSON intermediate representation containing lists of metrics, filters, and group-by dimensions.

• SMQ Limitations: The SMQ intentionally covers only the "common analytical core" (selection, filtering, grouping, declared joins). It excludes complex constructs like recursive CTEs, arbitrary subqueries, or advanced window functions.
• Compilation: The deterministic engine compiles the SMQ into SQL, resolving abstract names to physical column expressions and injecting join predicates.

Agent Workflow: Think–Act Loop

The agent follows a strict single-tool think–act loop:

1. Exploration: The agent uses the convertSmqToSql tool to explore the semantic layer. This tool compiles an SMQ and returns the resulting SQL and a data preview.
2. Grounding: The agent extracts verified physical table names and column expressions from the compiler's output. It is strictly forbidden from referencing semantic model names as physical tables in the final query; it must use names lifted from the compiler.
3. Composition: Once the agent has gathered sufficient "building blocks" (verified fragments), it composes the final, complex SQL query. This step allows the agent to add constructs the SMQ compiler cannot handle (e.g., window functions, recursive CTEs).
4. Execution: The agent issues a single terminal execute call to run the final query.

3. Key Contributions

• Decoupled Architecture: A system design that separates schema grounding (handled via the semantic layer) from SQL composition (handled by the agent), preventing the conflation of these two difficult problems.
• Structured Intermediate Representation (SMQ): A compact, structured query format that forces the agent to reason over business-named elements rather than raw schema dumps, while relying on a deterministic engine for physical mapping.
• Constrained Agent Loop: A single-tool paradigm where SMQ compilation serves strictly for exploration and verification, and SQL execution is the sole terminal action. This reduces error propagation and makes the agent's trajectory auditable.
• Multi-Backend Harness: An end-to-end evaluation system capable of routing and scoring queries across SQLite, BigQuery, and Snowflake backends under the Spider2 protocol.

4. Results

The system was evaluated on Spider2-snow, a benchmark of 547 enterprise NL2SQL tasks executed against Snowflake.

• Performance: The system achieved 94.15% execution accuracy (515/547 correct instances).
• Ranking: This places the system as the third-highest entry on the official Spider2 leaderboard.
• Comparison: The result significantly outperforms schema-only baselines. For context, DAIL-SQL with GPT-4o achieved 2.20%, and other agent-based methods (Spider-Agent, ReFoRCE) ranged between 23% and 31%.
• Per-Backend Breakdown: Accuracy was high across most origins (97.0% for SQLite-origin, 96.0% for GA4-origin, 94.9% for BigQuery-origin). The native Snowflake class showed lower accuracy (55.6%), attributed to the small sample size (18 instances) and higher variance, as well as dialect-specific handling challenges.

5. Significance and Claims

The paper argues that mediating LLM reasoning through a curated semantic layer is decisive for enterprise NL2SQL. By treating compiled SMQ-to-SQL fragments as "verified building blocks," the system ensures that physical identifiers are grounded in truth rather than hallucination.

The authors emphasize that the high performance is not solely due to the LLM's reasoning capabilities but is a result of the system architecture (semantic layer + agent). They acknowledge a critical tension: while the semantic layer provides the necessary grounding, curating it against an evaluation set risks overfitting (Goodhart's law), where descriptions might drift to encode specific answers rather than generalizable schema knowledge. The paper concludes that the primary challenge moving forward is maintaining the quality and generalizability of the semantic layer without compromising it to the evaluation set.