🤖 AI

Synthetic Contrastive Reasoning for Multi-Table Q&A

This paper introduces a synthetic contrastive reasoning-trace dataset for multi-table question answering and demonstrates that fine-tuning open-weight LLMs with Contrastive Preference Optimization (CPO) on these preference pairs significantly outperforms standard supervised fine-tuning by enhancing compositional reasoning and schema linking.

Original authors: Ankit Pratap Singh, Xin Su, Phillip Howard

Published 2026-06-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ankit Pratap Singh, Xin Su, Phillip Howard

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Lost in the Library" Effect

Imagine you are a librarian trying to answer a question like, "Which engineer from Chicago worked on the 'Alpha' project?"

To answer this, you can't just look at one book. You have to:

Find the Engineer book to see who lives in Chicago.
Find the Project Lead book to see who leads which project.
Find the Project book to see the start dates.
Connect the dots between all three books.

Current AI models (Large Language Models) are great at reading one book, but when they have to jump between multiple "books" (tables in a database) to find an answer, they often get lost. They might mix up names, forget a step, or guess the wrong connection.

The Missing Piece: Showing the "How," Not Just the "What"

Most training data for these AI models is like a flashcard:

Question: "Who led the Alpha project?"
Answer: "Tom Smith."

The AI learns the answer, but it doesn't learn how to find it. It's like a student memorizing the answer key without learning the math. The paper argues that to get better, the AI needs to see the step-by-step reasoning (the "trace") of how to get from the question to the answer.

The Solution: The "Good Cop / Bad Cop" Training Method

The authors created a new way to train AI called Synthetic Contrastive Reasoning. Think of it as a training camp with two types of instructors:

The Perfect Instructor (Positive Trace): This instructor shows the AI exactly how to solve the puzzle correctly, step-by-step, using only the information in the books.
The Trickster Instructor (Negative Trace): This instructor also tries to solve the puzzle, but they make subtle, believable mistakes. They might swap two names, mix up the order of steps, or pick the wrong column. The answer they give is wrong, but the logic looks convincing at first glance.

The Magic Ingredient: The paper found that if you use the same instructor to make both the perfect and the trickster examples, the AI gets confused because the "style" is too similar. Instead, they used two different AI models (GPT-4o for the good examples and Gemini 2.0 for the bad ones). This created a stronger contrast, like having a strict teacher and a playful trickster, making it much easier for the student AI to tell the difference between right and wrong.

The Training Technique: CPO (Contrastive Preference Optimization)

Once they had these pairs of "Right Way" and "Wrong Way" examples, they used a special training method called CPO.

Old Way (SFT): "Here is the right answer. Memorize it."
Old Way (DPO): "Here is a right answer and a wrong answer. Pick the right one." (This method was found to be unstable and memory-heavy).
New Way (CPO): "Here is a right answer and a wrong answer. Learn to strongly prefer the right one and actively reject the wrong one."

CPO teaches the AI not just to know the right path, but to recognize and avoid the "traps" (the plausible but wrong reasoning).

The Results: A Big Leap Forward

The researchers tested this on three different AI models (Qwen, Mistral, and Llama) using four different test sets.

The Score: Compared to standard training, this new method improved the AI's accuracy by 9.7% to 16.3% on average.
The Highlight: On the hardest tests (MMQA), the improvement was as high as 21 percentage points.
The Surprise: Even though they only trained the AI on puzzles involving two tables, the AI got significantly better at solving puzzles with three tables without any extra training. It learned the skill of connecting dots, not just the specific dots.

Why This Matters

This paper proves that to make AI smarter at complex tasks, we shouldn't just feed them more questions and answers. We need to feed them examples of how to think, including examples of how to think wrong so they can learn to spot the errors.

By using synthetic data (AI-generated examples) and a "contrastive" approach (comparing right vs. wrong side-by-side), they created a much more efficient and effective way to teach AI how to navigate complex, multi-step problems.

Technical Summary: Synthetic Contrastive Reasoning for Multi-Table Q&A

Problem Statement
Multi-table Question Answering (Q&A) presents a significant challenge for Large Language Models (LLMs) because the necessary knowledge resides in external relational databases rather than model parameters. Unlike single-table reasoning, multi-table tasks require models to retrieve evidence from multiple tables, link schemas (aligning primary and foreign keys), and perform compositional reasoning involving joins, filters, and aggregations. As the number of tables increases, contexts lengthen, and the risk of missing relevant evidence or making small schema-linking errors grows, often leading to incorrect final answers.

While recent advancements have improved single-table reasoning, multi-table Q&A remains underdeveloped. A critical gap exists in existing resources: most datasets provide only questions and final answers, lacking explicit reasoning supervision (traces) that explain how an answer is derived. Furthermore, they do not provide paired examples of plausible but incorrect reasoning, which are essential for preference-based optimization techniques.

Methodology
To address these gaps, the authors propose a framework for generating synthetic contrastive reasoning traces and utilizing Contrastive Preference Optimization (CPO) for fine-tuning.

Synthetic Trace Generation:
- Positive Traces ( $+y$ ): The authors use strong foundation models (primarily GPT-4o) to generate step-by-step reasoning traces that derive the gold-standard answer from the provided tables. These traces are designed to be explicit, referencing specific tables and using concise summaries rather than copying full data rows.
- Negative Traces ( $-y$ ): For every validated positive trace, a corresponding negative trace is synthesized using a different heterogeneous LLM (Gemini 2.0 Flash). These traces are fluent and plausible but contain deliberate factual or logical errors (e.g., entity swaps, reversed join orders, or incorrect operations) that lead to an incorrect answer.
- Heterogeneity: A key design choice is using different models for positive and negative generation. Ablation studies suggest this creates a stronger contrastive signal than using a single model for both, as it prevents the reasoning "styles" from being too similar, making it easier for the student model to distinguish correct logic from failure modes.
- Validation: A rigorous filtering pipeline ensures quality. Positive traces are validated using an LLM-as-a-judge (Gemini 2.0 Flash) to ensure semantic equivalence with the ground truth. Negative traces are constrained to be structurally consistent with positive ones but factually flawed.
Contrastive Preference Optimization (CPO):
- The authors fine-tune open-weight LLMs (Qwen3-14B, Mistral-8B, Llama-3.1-8B) using CPO.
- Unlike Direct Preference Optimization (DPO), which requires loading both a policy and a reference model simultaneously (increasing memory and slowing training) and treats all preference pairs equally, CPO removes the reference model (setting it to a uniform prior) and adds a behavior cloning regularizer.
- This approach stabilizes training on long reasoning chains and ensures the model learns the magnitude of the quality difference between correct and incorrect traces, rather than just the direction of preference.
Dataset Construction:
- The primary training data is derived from the MMQA benchmark (two-table subset), expanded with synthetic traces.
- A new evaluation set is constructed from the BIRD benchmark. This involves a three-stage filtering process: multi-table filtering (selecting 2-3 table queries), full-table evidence extraction (ensuring tables are not truncated to guarantee answer derivability), and automated semantic consistency verification (using an LLM judge to filter question-SQL misalignments).

Key Contributions

First Contrastive Reasoning-Trace Dataset: The paper introduces the first dataset for multi-table Q&A that pairs validated positive reasoning traces with plausible negative traces generated by heterogeneous LLMs.
Demonstration of CPO Efficacy: Through comprehensive experiments, the authors show that CPO training on these preference pairs significantly improves multi-table reasoning across three different open-weight LLMs and four evaluation datasets.
Validation of Synthetic Generation: The study includes ablations and human evaluations confirming that heterogeneous trace generators strengthen the contrastive signal and that the generated pairs are faithful, coherent, and meaningfully contrastive.
New Evaluation Benchmark: The construction of a rigorous BIRD-derived evaluation set that guarantees answer derivability through full-table evidence extraction and automated semantic verification.

Results

Performance Gains: Across Qwen3-14B, Mistral-8B, and Llama-3.1-8B, CPO achieved absolute average improvements over standard Q&A supervised fine-tuning (SFT) ranging from 9.7% to 16.3%. On the MMQA dataset, gains reached up to 21 percentage points.
Comparison to Baselines: CPO consistently outperformed both Q&A-only SFT and Trace-only SFT (using only positive traces). It also outperformed DPO on the same preference pairs in most evaluations, particularly avoiding the performance degradation on structured reasoning tasks that DPO sometimes exhibits.
Generalization: Models trained on two-table data using CPO successfully generalized to harder three-table queries in both MMQA and the BIRD evaluation set, demonstrating that the learned reasoning patterns transfer to increased complexity.
Ablation Insights: Using different models for positive and negative trace generation yielded higher performance than using the same model. Additionally, while open-source models (e.g., Qwen-3 30B) could generate useful traces, proprietary models (GPT-4o and Gemini 2.0) provided the strongest supervision.

Significance and Claims
The paper claims that synthetic contrastive reasoning traces provide an effective supervision signal for complex reasoning tasks. The authors argue that models benefit not only from observing correct reasoning paths but also from learning to reject plausible failure modes (such as missed constraints or incorrect joins) through contrastive learning.

The work highlights that CPO is a more stable and memory-efficient alternative to DPO for long reasoning chains. Furthermore, the study suggests that the approach is data-efficient; despite a relatively small training set of 1,920 contrastive pairs, the consistent gains across diverse models and datasets indicate that high-quality synthetic traces can elicit strong contrastive learning signals without requiring prohibitively large-scale data generation. The authors conclude that generating synthetic reasoning traces is a promising direction for advancing LLM capabilities in multi-table Q&A and other complex reasoning domains.