Technical Summary: ACE-SQL

Problem Statement

Text-to-SQL systems face a critical bottleneck in schema linking, particularly when dealing with modern databases containing large and complex schemas. Existing approaches generally fall into two categories, both of which suffer from a generator-conditioned supervision mismatch:

1. Full-Schema Generation: These methods implicitly learn schema selection within end-to-end SQL generation. However, this leaves schema linking within a massive search space, often resulting in lower execution accuracy compared to using ground-truth (gold) columns.
2. Retriever-Generator Pipelines: These systems use a separate retriever trained on static gold-column annotations to explicitly select schema components before generation. The limitation here is that the retriever is anchored to fixed "gold" targets. As demonstrated in the paper, a significant portion (19.1% to 42.3%) of execution-correct SQL predictions rely on non-gold column sets (alternative executable routes). A retriever trained strictly on static gold labels penalizes these valid, generator-preferred routes, creating a suboptimal policy for the current generator.

The core challenge is that schema linking and SQL generation are coupled: the retriever defines the generator's input distribution, while the generator's successful outputs determine which schema configurations are valid. Optimizing them jointly introduces a circular dependency and gradient interference, yet decoupling them leads to the supervision mismatch described above.

Methodology: ACE-SQL

The authors propose ACE-SQL (Adaptive Co-Optimization via Empirical Credit Assignment), a Reinforcement Learning (RL) framework that jointly optimizes schema retrieval and SQL generation under a shared policy ($\pi_\theta$). The framework operates in two stages:

1. Supervised Fine-Tuning (Cold Start)

To ensure the retriever-generator pipeline is functional before RL, the authors perform a self-distillation SFT stage using the Qwen3-8B model.

• Source Filtering: Samples are filtered from the SynSQL-2.5M dataset based on complexity and schema diversity.
• Paired Self-Distillation: The model generates retriever outputs (column sets) via self-consistency voting. These define pruned schemas for the generator. Only pairs where the generator's execution matches the gold SQL are retained.
• Mixed SFT: The retriever and generator samples are trained together to initialize the shared policy.

2. Joint GRPO Training (Reinforcement Learning)

The core innovation lies in the Joint Group Relative Policy Optimization (GRPO) training, which replaces static gold targets with adaptive, on-policy targets derived from execution feedback.

• Bidirectional Adaptation:
  • Retriever $\to$ Generator: The retriever samples $N$ column sets, which are aggregated via majority voting to create a pruned schema ($\hat{S}$). This schema conditions the generator's input.
  • Generator $\to$ Retriever: The generator produces $N$ SQL queries. Successful executions (matching the gold result) are used to update an Empirical Column-Set Pool.
• Empirical Credit Assignment:
  • Instead of targeting a fixed gold column set, the retriever's target is dynamically updated to the most frequent column set ($S^\star(q)$) observed in the empirical pool of execution-correct rollouts.
  • This pool is initialized with high-recall unions of column sets and updated online with a decay factor ($\gamma=0.5$) to prioritize newer evidence while retaining successful routes.
• Reward Construction:
  • Both roles receive sparse rewards gated by execution correctness (for the generator) or exact match to the empirical target $S^\star(q)$ (for the retriever).
  • A clipped length penalty is applied to discourage overly long outputs, but only if the output satisfies the sparse reward condition.
• Stabilization Mechanisms:
  • PCGrad (Gradient Surgery): To mitigate gradient interference between the retriever and generator losses, the authors apply PCGrad to project conflicting gradients.
  • Generator-Weight Schedule: The generator's contribution to the joint loss is linearly increased from 0 to 1 over the first 25% of RL steps. This allows the empirical retriever targets to stabilize before the generator exerts full influence.

Key Contributions

1. On-Policy Credit Assignment: The paper formulates schema retrieval as an on-policy credit assignment problem. Instead of relying on static gold annotations, ACE-SQL derives adaptive retrieval targets from the generator's own execution-correct rollouts.
2. Bidirectionally Adaptive Framework: ACE-SQL creates a closed loop where the retriever adapts to the generator's executable preferences, and the generator adapts to the retriever's evolving schema selections, both grounded by execution accuracy.
3. Stabilized Joint Optimization: The authors introduce a combination of an empirical column-set pool, PCGrad, and a generator-weight schedule to stabilize the non-stationary and coupled optimization process.
4. Efficiency and Performance: The method achieves high performance with significantly fewer output tokens compared to existing RL-based baselines.

Experimental Results

Evaluated on the BIRD Dev and Spider benchmarks using the Qwen3-8B backbone:

• BIRD Dev: ACE-SQL achieves 65.3% greedy execution accuracy (EX), outperforming all listed open-source baselines (e.g., SQL-R1-7B at 63.7%, MTIR-SQL-8B at 63.6%).
• Spider: Achieves 87.2% EX on the test set and 79.5% on the challenging Spider-Realistic variant.
• Efficiency: ACE-SQL uses only 0.93k output tokens per query on BIRD Dev. This is approximately 3.3$\times$ fewer tokens than SQL-R1-7B (3.10k) and 2.2$\times$ fewer than MTIR-SQL-8B (2.00k), despite achieving higher accuracy.
• Ablation Studies:
  • Removing PCGrad or the generator-weight schedule leads to performance drops, confirming the necessity of these stabilizers for joint RL.
  • Using static gold targets for the retriever results in lower performance (64.7% vs 65.3%) and higher instability, validating the benefit of empirical credit assignment.
  • Sparse rewards outperform dense rewards in this joint setting, as dense rewards can lead to gradient dominance by the retriever and reward hacking.

Significance and Claims

The paper claims that ACE-SQL provides a practical direction for building efficient and robust Text-to-SQL systems for complex, real-world databases. By moving away from static supervision and embracing bidirectional adaptation, the framework resolves the tension between schema retrieval and SQL generation.

The authors emphasize that their approach offers a new perspective on upstream-downstream credit assignment in multi-stage LLM pipelines. They argue that execution-aligned signals are superior to static annotations for tracking on-policy executable routes, allowing the model to leverage its own learned reasoning patterns rather than being constrained by potentially suboptimal gold labels. The work suggests that explicit schema retrieval, when co-optimized with generation, can improve both inference quality and efficiency without relying on massive token generation.

Limitations: The authors note that experiments are limited to a single 8B model (Qwen3-8B) and rely on synthetic training data (SynSQL-2.5M). They acknowledge that scaling to larger models or using real-world query-database pairs might reveal different dynamics, and that the empirical pool initialization from an SFT checkpoint may under-explore the base model's initial preferences.