💻 computer science

An Iterative Test-and-Repair Framework for Competitive Code Generation

This paper introduces FixAudit, an iterative test-and-repair framework that employs a shared model with specialized Fixer and Auditor roles to progressively debug and improve code candidates, significantly outperforming existing methods and larger baselines on competitive programming benchmarks.

Original authors: Lingxiao Tang, Muyang Ye, Zhaoyang Chu, Xiaoxue Ren, Zhongxin Liu, Lingfeng Bao, He Ye

Published 2026-04-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Lingxiao Tang, Muyang Ye, Zhaoyang Chu, Xiaoxue Ren, Zhongxin Liu, Lingfeng Bao, He Ye

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a very smart, but slightly clumsy, robot to solve complex math puzzles. This robot is a Large Language Model (LLM). It's great at writing code, but when it comes to competitive programming (where you have to write code that solves tricky logic problems perfectly), it often makes subtle mistakes.

Here is the story of how the authors of this paper, FixAudit, taught their robot to become a master problem-solver, using a method that is very different from what everyone else was doing.

The Old Way: The "Spray and Pray" Approach

Before FixAudit, the best method (called CURE) worked like a lottery machine.

The robot would generate 100 different versions of a solution (like buying 100 lottery tickets).
A separate "Tester" robot would generate 100 random test questions based only on the written instructions of the puzzle.
They would run all 100 solutions against all 100 questions.
Whoever passed the most tests won.

The Problem:

The Blind Tester: The Tester didn't look at the code. It was like a teacher grading a test without reading the student's answers, just guessing what questions might be hard. It missed the specific, sneaky bugs in the code.
The "Do-Over" Policy: If a solution was 99% correct but had one tiny error, the system threw it away and started from scratch with a new random guess. It never learned to fix the mistake; it just hoped the next random guess would be perfect.

The New Way: The "FixAudit" Approach

The authors realized that instead of guessing 100 times, you should fix one solution until it's perfect. They created a framework called FixAudit, which acts like a Master Detective and a Sharp Editor working together in a loop.

They trained one AI model to play two roles:

1. The Auditor (The Sharp Detective)

What it does: Instead of guessing random questions, the Auditor reads the code the robot wrote.
The Analogy: Imagine a detective looking at a suspect's alibi. The detective doesn't just ask random questions; they look for the specific lie. If the code forgets to check the left side of the street, the Auditor says, "Aha! I see you didn't check the left side. Let me create a test case where all the lanterns are on the right side to prove you're wrong."
Why it's better: It finds the exact hidden bugs that the robot missed.

2. The Fixer (The Sharp Editor)

What it does: When the Auditor finds a bug, the Fixer doesn't throw the code away. It patches the hole.
The Analogy: Imagine you are editing a manuscript. If you find a typo, you don't burn the whole book and rewrite it from page one. You just fix that one sentence. The Fixer looks at the broken part, understands why it failed, and fixes only that part, keeping all the good work the robot did previously.

The Four-Stage Training Camp

To make this team work, the authors put the AI through a four-step training camp:

Stage A (The Brain Gym): Before teaching them to fix or test, they taught the AI to understand how code runs. They made it predict what a program would do before actually running it. This gave the AI "execution reasoning"—the ability to think through the logic in its head.
Stage B (The First Repair): They gave the AI a broken program and a failing test. The Fixer learned to patch it up without breaking the parts that were already working.
Stage C (The Bug Hunter): Now that the program was "mostly" fixed, the Auditor learned to read the code and invent new, tricky tests specifically designed to break the remaining hidden bugs.
Stage D (The Final Polish): The Fixer took the new, tricky tests from the Auditor and made the final, tiny adjustments to make the code perfect.

The Results: Why It Matters

The paper tested this on three famous coding competitions. Here is the magic:

Small vs. Big: They used a small AI model (7 Billion parameters). Usually, you need a giant model (32 Billion parameters) to get good results. FixAudit's small model beat the giant model's zero-shot performance.
Efficiency: The old "lottery" method needed to generate dozens of solutions to find a good one. FixAudit found the right answer in fewer steps because it was learning and fixing, not just guessing.
The "Aha!" Moment: In one example, the robot forgot to check if a number could be zero. The Auditor noticed this specific gap in the logic and created a test that forced the robot to realize its mistake. The Fixer then made a tiny change (changing > to >=), and the problem was solved.

The Takeaway

FixAudit changed the game by stopping the "throw it away and start over" mentality. Instead, it treats coding like debugging:

Read the code carefully.
Find the specific flaw.
Fix just that flaw.
Repeat until it's perfect.

It's the difference between trying to win a game by rolling the dice 100 times versus learning the rules, spotting your opponent's weakness, and making a precise move to win.

1. Problem Statement

While Large Language Models (LLMs) have advanced significantly in general code generation, competitive programming remains a persistent challenge. These problems involve strict logical constraints where a model might generate code that passes public test cases but fails hidden test sets due to subtle, implementation-specific bugs.

Existing approaches face two main limitations:

Prompt-based/Agent-based methods: Rely on prompting without training the model's underlying repair capabilities. They often regenerate code from scratch rather than fixing specific errors.
Training-based methods (e.g., CURE): Use Reinforcement Learning (RL) to jointly train a "Coder" and a "Tester." However, CURE suffers from two critical flaws:
1. Blind Testing: The Tester generates tests based only on the problem description, never seeing the candidate code. Consequently, it fails to generate tests that expose specific implementation bugs (e.g., missing boundary checks).
2. No Repair Mechanism: The Coder generates every candidate from scratch. It cannot learn to iteratively fix a buggy program based on a failed test, leading to wasted correct logic and inefficient search.

2. Methodology: The FixAudit Framework

The authors propose FixAudit, a framework that reframes competitive code generation as a continuous, targeted test-and-repair debugging cycle. Instead of generating many candidates from scratch, FixAudit starts with a single initial candidate and iteratively improves it.

The framework trains a single shared LLM to perform two specialized roles:

The Fixer: Repairs the current candidate code based on a failing test while preserving correct logic.
The Auditor: Reads the candidate code and the problem specification to generate new, targeted tests that expose remaining hidden bugs.

The Four-Stage Training Pipeline

FixAudit employs a progressive four-stage training pipeline to build the necessary execution reasoning and repair capabilities:

Stage A: Execution-Aligned Supervised Fine-Tuning (SFT)
- Goal: Equip the model with strong execution reasoning (the ability to trace program state and predict outcomes).
- Tasks:
  1. Program Output Prediction: Predict the output of a given program on a specific input.
  2. Specification-to-Output Derivation: Derive the correct output from the problem description alone.
- Outcome: Creates a foundational model ( $\pi_0$ ) capable of understanding code behavior and specifications.
Stage B: Fixer RL (Round-1 Repair)
- Goal: Train the model to repair an initial buggy candidate using a failing public test.
- Mechanism: The Fixer receives the problem spec, the buggy code, and a failing test. It must generate a repaired version that passes the failing test without breaking previously passing tests (Regression Penalty).
- Reward: 1 if the fix passes the failing test and causes no regressions; 0 otherwise.
Stage C: Auditor RL (Bug-Revealing Test Generation)
- Goal: Train the model to act as an "Auditor" that reads the repaired code to find latent bugs.
- Mechanism: Unlike CURE's Tester, the Auditor sees the candidate code. It generates a test case (input + expected output) that causes the repaired code to fail.
- Reward Shaping: To provide dense feedback, the Auditor is rewarded not just for breaking the target candidate, but also for breaking a pool of other incorrect repairs (auxiliary shaping), encouraging the generation of broadly useful tests.
Stage D: Fixer RL (Round-2 Refinement)
- Goal: Close the loop by training the Fixer to address the specific bugs exposed by the Auditor.
- Mechanism: The Fixer takes the Stage-B repaired code and the new "Auditor-generated" test. It produces a further refined solution.
- Reward: Based on the number of hidden tests passed. The Auditor's test is treated as a "hint" rather than a hard constraint to prevent overfitting to potentially incorrect expected outputs generated by the Auditor.

Optimization: All RL stages utilize DAPO (Decoupled Clip and Dynamic Sampling Policy Optimization), which allows for larger policy updates on rare successful events (like finding a bug fix) while preventing premature search narrowing.

3. Key Contributions

New Perspective: Shifts the paradigm from "generate-and-select" (generating many candidates) to "targeted test-and-repair" (iteratively fixing a single candidate).
FixAudit Framework: Introduces a four-stage training pipeline that jointly trains a Fixer and an Auditor within a single model, enabling the Auditor to read code for targeted test generation and the Fixer to perform localized repairs.
Superior Performance: Demonstrates that a 7B model trained with this framework outperforms significantly larger zero-shot models (32B) and strong baselines on the same 7B backbone.

4. Experimental Results

The framework was evaluated on three competitive programming benchmarks: APPS, CodeContests, and xCodeEval.

Performance vs. Larger Models:
- FixAudit (7B) achieved an average Pass@1 of 30.40% and AvgPassRatio of 56.77%.
- It surpassed the Qwen2.5-Coder-32B (zero-shot) by 24.9% in Pass@1 and 40.5% in AvgPassRatio.
- It outperformed GPT-4o-mini by 46.6% in Pass@1.
Performance vs. Baselines (Same 7B Backbone):
- Compared to Specine (strong agent-based baseline): +36.8% Pass@1.
- Compared to CURE (strong RL-based baseline): +35.1% Pass@1.
- The improvement was most pronounced on CodeContests (complex logic), where FixAudit nearly doubled Specine's Pass@1 (19.80% vs. 10.30%).
Efficiency:
- FixAudit reaches its peak performance in fewer iterations. One full test-and-repair cycle of FixAudit outperforms the final results of Specine and CURE across all benchmarks.
- Ablation studies confirmed that the Auditor loop (Stages C & D) and Execution-Aligned SFT (Stage A) are the most critical components. Removing the Auditor's ability to read code ("Blind Auditor") caused a significant performance drop, validating the importance of code-aware testing.

5. Significance

Efficiency in Reasoning: FixAudit demonstrates that targeted, iterative repair is more efficient than brute-force generation. By preserving correct logic and only fixing specific faults, the model converges to correct solutions faster.
Code-Aware Testing: The paper proves that test generation is significantly more effective when the generator analyzes the candidate code to find specific logical gaps, rather than relying solely on the problem description.
Scalability: The framework allows smaller models (7B) to outperform much larger foundation models (32B) and commercial APIs, suggesting that specialized training pipelines are more valuable than simply increasing model size for complex reasoning tasks.
Closed-Loop Debugging: The integration of a "Fixer" and "Auditor" in a single model creates a self-correcting system that mimics human debugging processes, addressing the "hallucination" of correctness common in current LLMs.