💬 NLP

QAQ: Bidirectional Semantic Coherence for Selecting High-Quality Synthetic Code Instructions

The paper proposes QAQ, a novel data selection framework that leverages Reverse Mutual Information to evaluate bidirectional semantic coherence between queries and answers, effectively filtering noisy synthetic code data to achieve high model performance with significantly reduced training sets.

Original authors: Jiayin Lei, Ming Ma, Yunxi Duan, Chenxi Li, Tianming Yang

Published 2026-03-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiayin Lei, Ming Ma, Yunxi Duan, Chenxi Li, Tianming Yang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to write computer code. You have a massive library of "practice problems" (questions) and "solutions" (answers) generated by other robots. This is called synthetic data.

The problem? Not all of these practice problems are good. Some are nonsense, some are too easy, and some are just copy-pasted garbage. If you feed your robot bad practice problems, it will learn bad habits.

The paper you shared introduces a new method called QAQ (which stands for Question, Answer-Question) to filter out the trash and keep only the gold.

Here is how it works, explained with simple analogies:

1. The Old Way: "How hard is this?" (The Student's Perspective)

Previously, researchers used a method called IFD. Imagine a teacher looking at a math problem and asking, "How hard is it for a student to solve this?"

If the answer is obvious, the problem is too easy.
If the answer is impossible, the problem is too hard.
They kept the "just right" problems.

The Flaw: This only looks at the problem from the front. It doesn't check if the problem itself makes sense. A robot might generate a question like, "How do I turn a 'gritty sulla matrix' into code?" The robot might write a perfect answer that repeats the nonsense words. The "difficulty" score looks fine, but the whole thing is garbage.

2. The New Way: "Does the answer explain the question?" (The Detective's Perspective)

The authors, QAQ, flip the script. Instead of asking "How hard is the question?", they ask: "If I only saw the answer, could I guess what the question was?"

They call this Reverse Mutual Information (RMI). Think of it like a detective looking at a crime scene (the answer) and trying to reconstruct the crime (the question).

Good Data (The Sweet Spot): You see a complex code solution. You can look at it and say, "Ah, this must have been the answer to a question about sorting a list." The answer gives you a clear clue about the question. This is high quality.
Bad Data Type A (The Echo Chamber): The question is nonsense, and the answer just repeats the nonsense words back.
- Question: "How to fix a 'flibber-flabber'?"
- Answer: "To fix a 'flibber-flabber', you must..."
- The Detective's View: Because the answer just parrots the question, it's too easy to guess the question. The "clue" is obvious because it's a cheat. This is low quality.
Bad Data Type B (The Lost Connection): The question is "Hello!" and the answer is a complex Python script.
- The Detective's View: Looking at the code, you have no idea why someone asked "Hello!" The answer gives you zero clues about the question. This is low quality.

3. The "Cognitive Gap" Strategy: Using Two Teachers

The authors realized that one robot (model) might be too smart or too dumb to spot the difference between "good" and "tricky" data. So, they used two teachers:

The Expert Teacher (Strong Model): Very smart, knows deep patterns.
The Junior Teacher (Weak Model): Less experienced, sees things more simply.

They looked for a specific type of disagreement, which they call the "Cognitive Gap":

Scenario: The Expert Teacher looks at a sample and says, "Wow, this is a brilliant, complex problem with a perfect solution!" (High Score).
But the Junior Teacher looks at the same sample and says, "I have no idea what's going on here. This is confusing." (Low Score).

Why keep this?
If both teachers agree it's easy, it's probably too simple (boring).
If both agree it's broken, it's trash.
But if the Expert sees value that the Junior misses, that sample is a hidden gem. It's challenging enough to teach the robot new things, but valid enough to be real.

The Results: Doing More with Less

The team tested this on a massive dataset of 300,000 code examples.

The Magic Number: They found that by using this "QAQ" filter, they only needed to keep 25% of the data (the top 1/4th).
The Outcome: Training the robot on just this small, filtered 25% worked just as well as training on the full 100% of the messy data.
The Benefit: This saves a huge amount of time, money, and electricity (computational cost) without making the robot any dumber.

Summary Analogy

Imagine you are building a gym for a weightlifter.

Old Method: You grab a pile of random weights. Some are too light (boring), some are too heavy (impossible), and some are made of cardboard (fake). You try to guess which are real by how heavy they feel when you lift them.
QAQ Method: You look at the weightlifter's form after they lift the weight.
- If their form is perfect and you can tell exactly what exercise they were doing, it's a good weight.
- If they are just swinging a cardboard box and the form looks fake, it's trash.
- If you have a Coach (Expert) who says, "That was a great lift!" and a Trainee (Junior) who says, "I don't get it," you keep that weight. It's the perfect challenge.

The Bottom Line: By checking if the answer makes sense backwards to the question, and by listening to the disagreement between smart and less-smart robots, we can clean up synthetic data and train AI much faster and cheaper.

1. Problem Statement

Synthetic data has become the dominant paradigm for training code generation models (e.g., via seedless methods like Magpie and WarriorCoder). However, these methods introduce significant noise and hallucinations that are difficult to detect with current metrics.

Limitations of Existing Metrics: Traditional selection methods like Instruction-Following Difficulty (IFD) measure how hard it is for a model to generate an answer given a query ( $P(A|Q)$ ). The authors argue this is ambiguous on noisy synthetic data because low probability can stem from either intrinsic task complexity or model-generated hallucinations.
The Core Issue: Synthetic pipelines often overlook query quality. A poorly specified or semantically incoherent query (e.g., containing fabricated terminology or off-topic greetings) paired with a technically correct answer provides little learning signal. Existing metrics fail to detect "semantic misalignment" or "shortcut patterns" (e.g., keyword echoing) where the answer trivially predicts the query.

2. Methodology: The QAQ Framework

The authors propose QAQ (Query, Answer-Query), a data selection framework that evaluates data quality from the reverse direction: how well the answer predicts the query ( $P(Q|A)$ ).

A. Reverse Mutual Information (RMI)

The core metric is Reverse Mutual Information (RMI), defined as the information gain about the query conditioned on the answer:
$\text{RMI}(Q, A) = \log \text{PPL}(Q) - \log \text{PPL}(Q|A)$

$\text{PPL}(Q)$ : Perplexity of the question alone.
$\text{PPL}(Q|A)$ : Perplexity of the question conditioned on the answer (computed via a "reverse generation" prompt where the model infers the question from the answer).
Interpretation:
- Low RMI: Indicates semantic misalignment (the answer provides no information to reconstruct the query).
- Excessively High RMI: Indicates defect patterns where the answer makes the query trivially predictable (e.g., paraphrasing, keyword echoing, or repetition), offering no genuine learning signal.
- Optimal Range: A "sweet spot" in the middle range indicates strong semantic coherence without trivial shortcuts.

B. Stratified RMI

Direct global ranking of RMI is biased by question complexity. Simple questions naturally have low perplexity, leading to lower RMI scores regardless of answer quality.

Solution: The dataset is partitioned into $K=10$ bins based on $\text{PPL}(Q)$ (question complexity). RMI rankings are computed independently within each bin to ensure fair comparison among samples of similar difficulty.

C. Model Disagreement (Cognitive Gap)

The authors leverage the disagreement between a Strong Model ( $M_s$ ) and a Weak Model ( $M_w$ ) to identify high-value samples.

Hypothesis: Samples where the strong model assigns a high RMI rank (validating quality) but the weak model assigns a low rank (indicating difficulty) represent the "Cognitive Gap." These samples are valid yet challenging, providing the strongest learning signal.
Selection Strategy: Calculate the difference in normalized ranks: $\text{Diff} = r_s - r_w$ . Samples with $\text{Diff} > \tau$ (where $\tau \approx 0.1$ ) are retained. This filters out samples where both models agree on low quality (corrupted) or both agree on high quality (trivial/repetitive).

3. Key Contributions

Reverse Direction Analysis: Introduced QAQ, shifting focus from "generation difficulty" ( $A|Q$ ) to "answer explanatory power" ( $Q|A$ ) to better detect semantic misalignment and synthetic defects.
RMI as a Quality Indicator: Empirically demonstrated that RMI extremes (too low or too high) signal data quality issues, while the mid-range represents optimal training data.
Model Disagreement Signal: Proposed using the "cognitive gap" between strong and weak models as a selection signal, identifying a unique subset of high-value samples that consensus-based methods miss.
Efficiency: Demonstrated that selecting only 25% of the data using this method achieves performance comparable to training on the full dataset.

4. Experimental Results

Experiments were conducted on the WarriorCoder dataset (310K synthetic code samples) using DeepSeek-Coder-6.7B-Base for fine-tuning.

Performance vs. Full Data: The RMI 50-75% subset (25% of data) achieved performance comparable to full-data training on HumanEval+ (72.56 vs. 72.56) and MBPP+ (58.47 vs. 59.52).
Comparison with Baselines:
- Outperformed IFD (Instruction-Following Difficulty) significantly (e.g., 72.56 vs. 66.46 on HumanEval+).
- Outperformed Random sampling and other selection methods like SCAR and RDS+.
Analysis of Extremes:
- Top 25% RMI: Underperformed, confirming that extremely high RMI correlates with defect patterns (e.g., answers that merely echo query keywords).
- Bottom 25% RMI: Underperformed, confirming low RMI indicates semantic misalignment.
Disagreement vs. Consensus:
- Diff-High (Disagreement strategy) achieved 71.95 on HumanEval+.
- Sum-High (Consensus strategy, where both models agree on high quality) achieved 68.90.
- The overlap between Diff-High and Sum-High selections was only 13.85%, proving that the disagreement signal captures a unique, valuable subset of data that consensus methods filter out.

5. Significance and Impact

Computational Efficiency: The method allows training on only 25% of synthetic data while maintaining full-data performance, drastically reducing computational costs (training time and energy).
Data Curation Paradigm: Highlights the importance of bidirectional semantic coherence. It suggests that high-quality synthetic data must not only be hard to generate ( $A|Q$ ) but also logically explanatory ( $Q|A$ ).
Scalability: Provides a scalable pathway to curate massive synthetic datasets without human intervention, addressing the "noise" problem inherent in seedless data generation.
Generalizability: While tested on code, the framework of using reverse perplexity and model disagreement is applicable to other domains of instruction tuning.

Conclusion

The paper establishes that QAQ is a superior method for filtering synthetic code data. By measuring how well an answer explains a question and leveraging the "cognitive gap" between models, it effectively removes noise and trivial patterns, enabling efficient training of high-performance code generation models.