💻 computer science

ZeroCoder: Can LLMs Improve Code Generation Without Ground-Truth Supervision?

ZeroCoder is a fully label-free, co-evolutionary framework that jointly trains a code generator and a test generator using execution-based feedback and a dynamic Bayesian selector to significantly improve code generation performance without relying on ground-truth supervision.

Original authors: Lishui Fan, Mouxiang Chen, Tingwei Zhu, Kui Liu, Xin Xia, Shanping Li, Zhongxin Liu

Published 2026-04-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Lishui Fan, Mouxiang Chen, Tingwei Zhu, Kui Liu, Xin Xia, Shanping Li, Zhongxin Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to write computer code. Usually, to teach a robot, you need a human teacher to look at the robot's work, find the mistakes, and give it a grade. This is expensive, slow, and hard to scale because there aren't enough human teachers for every problem.

ZeroCoder is a new method that asks a bold question: Can the robot teach itself without a human teacher?

The answer, according to this paper, is yes. But it doesn't just "guess" its own grades. Instead, it uses a clever partnership between two roles played by the same robot.

The Two Characters: The Architect and The Inspector

Think of the robot as having two hats it wears, switching back and forth:

The Architect (The Coder): This hat tries to build a solution (write code) to solve a problem.
The Inspector (The Tester): This hat tries to build a test (a set of rules) to see if the solution works.

In traditional methods, the Inspector is static. It's like a rigid checklist that never changes. If the Architect gets better, the old checklist might become too easy, letting bad code slip through.

ZeroCoder's Secret Sauce: It makes the Architect and the Inspector co-evolve. They grow together, pushing each other to get better.

How It Works: The "Pass/Fail" Dance

Here is the step-by-step process, using a simple analogy:

The Dance Floor: The Architect writes 8 different versions of a solution. The Inspector writes 8 different versions of a test.
The Grid: They mix and match. Every solution is run against every test. This creates a giant grid (a matrix) of Pass/Fail results.
- Analogy: Imagine a dance floor where 8 dancers (solutions) try to dance with 8 different music tracks (tests). Some dances work perfectly; others crash.
The Filter (Ranking): The system looks at the grid. If every solution passes every test, the test is too easy (useless). If every solution fails every test, the test is too hard or the code is broken.
- ZeroCoder throws away these "boring" problems. It only keeps the ones where the results are mixed and interesting. This ensures the robot is learning from real challenges, not easy wins or impossible losses.
The Reward:
- For the Architect: If a solution passes the "best" tests, it gets a reward. If it fails, it gets a penalty.
- For the Inspector: This is the tricky part. How do you reward the Inspector?
  - Bad Inspector: Writes a test that says "True" for everything. (Easy to pass, but useless).
  - Good Inspector: Writes a test that catches subtle errors.
  - ZeroCoder uses a "Mutation" trick. It takes a working solution and intentionally breaks it in small ways (like changing a number or a word). If the Inspector's test can still catch the broken version, the Inspector gets a high reward! This forces the Inspector to write smart tests, not just easy ones.

The Problem of "Drifting" (Selector Drift)

As the robot gets better, the rules it used to decide "what is a good solution" start to get out of sync.

Analogy: Imagine a referee in a soccer game who was calibrated for 10-year-old kids. As the players grow up and become pros, the referee's old rules (like "don't run too fast") no longer make sense. The referee starts calling fouls on things that aren't fouls anymore. This is called Selector Drift.

To fix this, the authors created DyB4 (Dynamic B4).

Instead of using a rigid rulebook, DyB4 uses a tiny "calibration set" (just 10 examples with known answers).
Before every training session, it checks these 10 examples to see if its "referee rules" are still accurate. If the rules are off, it tweaks them instantly. This keeps the grading fair even as the robot gets smarter.

The Results: Self-Taught Mastery

The paper tested this on three different AI models. Here is what happened:

Without Human Help: ZeroCoder improved the code-writing ability by about 14.5% and the test-writing ability by 24.3% compared to the base model.
With a Tiny Nudge: When they added the "calibration" (DyB4) using just 10 human-labeled examples, the improvement jumped to 21.6% for coding and 24.3% for testing.
The "Oracle" Comparison: They compared ZeroCoder to a system that had access to perfect, human-written answers (the "Oracle"). Surprisingly, the self-taught ZeroCoder performed almost as well as the one with the perfect human teacher!

The Big Picture

ZeroCoder proves that you don't need a massive army of human teachers to improve AI code generation. By creating a system where the AI writes code, writes tests for that code, and then critiques both, it creates a self-sustaining loop of improvement.

It's like a student who writes an essay, then writes a quiz for themselves, takes the quiz, and uses the results to study harder. Eventually, they become their own best teacher.

1. Problem Statement

Code generation is a critical task in software engineering, typically improved via Reinforcement Learning with Verifiable Rewards (RLVR). However, standard RLVR pipelines rely heavily on ground-truth supervision (e.g., human-curated unit tests or tests derived from reference solutions) to verify code correctness. This reliance creates a bottleneck due to the high cost and scarcity of such labeled data.

Existing attempts to bypass this using self-rewarding mechanisms (where models generate their own tests) often fail because:

Low-Quality Tests: Self-generated tests often lack discriminative power, allowing incorrect ("spurious") solutions to pass.
Static Selection: Fixed selection rules (heuristics) used to identify "correct" solutions become miscalibrated as the model's capabilities evolve during training, leading to noisy reward signals.

Core Question: Can Large Language Models (LLMs) improve code generation without relying on any ground-truth supervision?

2. Methodology: ZeroCoder Framework

The authors propose ZeroCoder, a fully label-free, co-evolutionary framework that jointly trains a Coder (generates solutions) and a Tester (generates tests) using only execution feedback from their interactions.

Key Components:

A. Co-Evolutionary RL Loop
Instead of fixing the tester while training the coder, ZeroCoder alternates between roles.

Interaction: For a given problem, the Coder samples $N$ solutions ( $S$ ) and the Tester samples $M$ tests ( $T$ ).
Passing Matrix ( $E$ ): The system executes every pair $(s_i, t_j)$ to form a binary matrix where $e_{ij} = 1$ if solution $s_i$ passes test $t_j$ .
Selector: A selection algorithm identifies a "consensus subset" of likely-correct solutions ( $C_S$ ) and consistent tests ( $C_T$ ) based on $E$ .
Rewards:
- Coder Reward: Binary reward ($1$) if the solution is in the consensus set $C_S$ .
- Tester Reward: A composite reward balancing three terms:
  - Proxy-Agreement: Does the test pass a consensus solution?
  - Discriminativeness (Mutation): Does the test fail mutated versions of the solution? (Crucial for preventing trivial tests like assert True).
  - Format: Penalty for unparsable code.
- Curriculum Learning: The weight of the mutation term is gradually increased (via a cosine schedule) to shift focus from test validity to test discriminativeness as training progresses.

B. Offline Rank-Based Pre-filtering
To prevent training on degenerate problems (too easy or too hard), ZeroCoder filters the dataset before RL.

It computes the rank of the passing matrix $E$ generated by the initial policy.
Low-rank matrices indicate redundant interactions (e.g., all pass or all fail).
Only problems with a matrix rank above a threshold $\tau$ are retained, ensuring high interaction diversity and informative reward signals.

C. Dynamic B4 ($DyB4$) Selector
The paper identifies "Selector Drift": static selection rules (like MaxPass or fixed Bayesian priors) become miscalibrated as the Coder and Tester improve, leading to noisy rewards.

Solution: $DyB4$ is a dynamically calibrated Bayesian selector.
Mechanism: It uses a tiny labeled set ( $|H| = 10$ ) to periodically recalibrate the prior hyperparameters of the Bayesian selector during training. This aligns the selector's assumptions with the current policy's reliability without requiring ground-truth for the main training loop.

3. Key Contributions

ZeroCoder Framework: A fully label-free RLVR framework that co-evolves code and test generation. It proves that self-generated code-test interactions can serve as effective supervision, improving both tasks simultaneously.
Selector Drift Analysis & $DyB4$: The authors identify that fixed selection rules degrade over time in co-evolutionary settings. They introduce $DyB4$, a dynamic selector that uses a minimal labeled set (as few as 10 instances) to maintain selection quality, outperforming static selectors.
Empirical Validation: Comprehensive evaluation across three model families (Qwen2.5-1.5B, Qwen3-4B, Qwen2.5-Coder-7B) and six benchmarks (MBPP, LiveCodeBench, APPS, CodeForces).

4. Experimental Results

The experiments demonstrate significant improvements over base models and baselines (Offline/Online Test-Driven RL without co-evolution).

Fully Label-Free Performance:
- On Qwen2.5-Coder-7B-Instruct, ZeroCoder with static selectors improved code generation by 14.5% and test generation by 15.6% over the base model.
- With $DyB4$, code generation improved by 21.6% and test generation by 24.3%.
Comparison to Oracle:
- ZeroCoder with $DyB4$ achieved performance competitive with oracle-supervised training (which uses ground-truth solutions and tests). For Qwen2.5-Coder-7B, the label-free approach reached 42.2% Pass@1 vs. 43.2% for the oracle.
Ablation Studies:
- Removing the mutation-based reward caused a significant drop in code generation performance, confirming that discriminative tests are essential for the co-evolution loop.
- Rank-based filtering was shown to be critical; training on unfiltered data or data with low interaction diversity yielded poor results.
- Calibration Set Size: $DyB4$ is highly label-efficient; a set of just 10 labeled instances was sufficient to achieve performance comparable to larger calibration sets.

5. Significance

Breaking the Supervision Bottleneck: ZeroCoder demonstrates that high-quality code generation can be achieved without expensive human-curated unit tests, relying instead on the mutual improvement of code and test generation.
Addressing Non-Stationarity: The identification of "Selector Drift" and the proposal of $DyB4$ provide a novel solution to the instability often found in self-improving RL systems.
Practical Efficiency: The framework is computationally feasible, with the dynamic calibration step adding only ~2% to the wall-clock time per training step.
Generalizability: The approach works across different model scales and architectures, suggesting a scalable path for future self-improving code generation systems.

In conclusion, ZeroCoder establishes that co-evolutionary learning between code and test generation, guided by execution feedback and dynamic selection, is a viable and powerful paradigm for advancing LLMs in software engineering without ground-truth supervision.