Technical Summary: CAPER – Clause-Aligned Process Supervision for Text-to-SQL

Problem Statement

Text-to-SQL systems aim to translate natural language questions into executable SQL queries. While Large Language Models (LLMs) have advanced this field through schema linking and retrieval, the task remains a high-precision structured generation problem where a single semantic error (e.g., an incorrect join key or aggregation) can render the entire query invalid.

Current training paradigms face two primary limitations regarding supervision:

1. Query-Level Execution Correctness: Standard evaluation relies on terminal execution signals (Execution Accuracy, EX). While scalable, this provides a sparse, binary reward (success/failure) that offers no guidance on which intermediate decision caused the failure. This creates a credit assignment problem where "near-miss" queries (mostly correct but failing on one clause) are treated identically to completely wrong queries.
2. Token-Level Dense Supervision: Moving to token-level feedback is ill-suited for SQL semantics. Individual tokens (aliases, commas, operators) rarely constitute complete semantic decisions, while critical SQL decisions are expressed via multi-token structures (e.g., join paths, predicates). Token-level supervision can diffuse credit across low-level symbols and requires unreliable fine-grained labeling.

The core challenge is to derive clause-aligned process supervision from coarse query-level outcomes without costly manual annotation, preserving execution correctness as the ultimate objective while enabling root-cause error localization.

Methodology: The CAPER Framework

The authors propose CAPER (Clause-Aligned Process Supervision), an automated framework that constructs clause-level preference data and trains a lightweight Clause-Level Process Reward Model (Clause-PRM). The framework operates in three stages:

1. Automatic Reward Annotation

CAPER converts coarse trajectory outcomes into fine-grained clause-level rewards using two complementary strategies on an Abstract Syntax Tree (AST) representation of SQL:

• Fault Injection (for Successful Trajectories): For trajectories that execute correctly, the system samples editable key nodes in the AST and applies type-preserving mutations (e.g., swapping columns, changing operators). A mutation is retained only if it alters the execution result, thereby inducing a genuine semantic error. This creates "negative" examples where the root cause is known.
• Counterfactual Intervention (for Failed Trajectories): For trajectories that fail, the system compares the predicted SQL against the gold standard. It identifies the earliest divergent clause and constructs a counterfactual corrected query by repairing the AST under the same prefix. The system then computes the minimum-cost edit script to transform the predicted clause to the corrected one, identifying the root-cause node ($v^*$).

Reward Propagation:
Once the root-cause node is identified, a continuous penalty is assigned to AST nodes based on their structural distance to $v^*$.

• The root-cause node receives the strongest penalty ($-\alpha$).
• Ancestor nodes on the same dependency path receive Gaussian-decayed negative rewards.
• Unrelated branches are masked (zero reward).
  These node-level penalties are aggregated to the clause level (e.g., SELECT, FROM-JOIN, WHERE) using a minimum operator, ensuring that a clause containing the decisive error inherits a strong non-positive label. This results in preference tuples $(u^+, u^-)$ with a continuous margin $\delta_k$ reflecting the severity of the structural error.

2. Clause-Level Process Reward Model (Clause-PRM)

Using the annotated dataset $\mathcal{D}$, the authors fine-tune a lightweight model (CAPER-9B, based on Qwen3.5-9B) to act as a Clause-PRM.

• Objective: The model learns to score the preference between a correct clause unit and a dispreferred (error-containing) clause unit given the input and preceding context.
• Training: It optimizes a margin-weighted preference objective, where pairs with larger structural margins (more decisive errors) contribute stronger supervision signals.

3. Policy Optimization

The trained Clause-PRM is deployed within a Reinforcement Learning (RL) loop (specifically GRPO) for Text-to-SQL policy optimization.

• Multi-Granular Reward: The reward signal at each clause boundary combines:
  1. Clause Reward: The score from the Clause-PRM for the current clause.
  2. Execution Reward: A terminal signal added only at the final clause boundary.
• Credit Assignment: Instead of collapsing rewards into a single trajectory-level advantage, the method computes a clause-level return-to-go. Tokens within a clause are optimized based on the advantage of the remaining trajectory after that clause boundary, preserving temporal credit assignment.

Key Contributions

1. Automated Pipeline: A counterfactual SQL-AST pipeline that localizes root-cause errors and constructs clause-level preference supervision without manual annotation. This pipeline generated over 90,000 clause-annotated preference tuples across three datasets (BIRD, Spider, SYNSQL).
2. Clause-PRM (CAPER-9B): A lightweight model trained to provide dense, clause-boundary feedback. It bridges the gap between sparse execution outcomes and brittle token-level supervision.
3. Empirical Validation: Demonstrated that clause-aligned supervision significantly improves both end-to-end execution accuracy and the model's ability to localize failures.

Experimental Results

Experiments were conducted on the BIRD and Spider benchmarks using Qwen3.5-9B as the backbone.

• Execution Accuracy (EX):
  • The GRPO policy trained with CAPER-9B (Clause-level reward) achieved a 15.3% relative improvement in Execution Accuracy over GPT-5.4 on the BIRD dataset.
  • It outperformed both sparse-reward GRPO and token-level reward GRPO baselines, particularly under Majority Vote@8 settings, indicating improved semantic reliability in sampled candidates.
• Failure Localization:
  • On held-out failed trajectories, CAPER-9B achieved 84.53% Top-1 localization accuracy and 90.60% Mean Reciprocal Rank (MRR).
  • This significantly outperformed heuristic selectors, self-debugging prompts, and execution-only reward models. Ablation studies confirmed that both topology-aware propagation and counterfactual error construction were critical for this performance.
• Candidate Verification:
  • When used as a plug-and-play verifier to rerank candidates from closed-source LLMs, CAPER-9B consistently improved execution accuracy over Majority Vote@8 by +0.7 to +1.6 points, demonstrating its ability to identify candidates with reliable intermediate structural decisions even when the generator is fixed.

Significance and Claims

The paper claims that clause-aligned process supervision is a scalable and effective direction for structured generation tasks like Text-to-SQL. By attributing execution success and failure to specific SQL clause decisions, CAPER provides a practical credit assignment mechanism that avoids the sparsity of terminal labels and the semantic misalignment of token-level feedback.

The authors emphasize that their approach does not require new parser architectures or complex prompting pipelines but rather leverages counterfactual AST perturbations to derive semantically grounded supervision. The results suggest that intermediate process feedback, when aligned with the semantic units of the domain (SQL clauses), can significantly enhance both the optimization of generation policies and the diagnostic capability of reward models.

Limitations Acknowledged:
The authors note that the current implementation decomposes SQL only at the top-level clause granularity, which may be coarse for errors inside complex predicates or subqueries. Additionally, the automated annotation pipeline introduces computational overhead compared to sparse rewards, though the resulting data is reusable for PRM training. The method is currently focused on SQL generation rather than Chain-of-Thought reasoning.