Compete Then Collaborate: Frontier AI Teachers Build a Verifiable Curriculum to Improve a Coding Student Beyond Imitation

Technical Summary: Compete then Collaborate

Problem Statement
The paper addresses two under-explored questions in the distillation of Large Language Models (LLMs) into smaller student models: (1) Which commercial frontier model serves as the best "teacher" when measured by objective downstream student performance rather than biased LLM-as-judge evaluations? and (2) Is the optimal method for combining multiple teachers to merge their answers for imitation, or is there a superior mechanism? Prior work often relies on LLM judges (which suffer from self-preference bias) or merges teacher outputs naively, potentially causing knowledge conflicts that degrade student performance.

Methodology
The authors propose a "Compete-then-Collaborate" framework centered on Python coding tasks. The methodology consists of three distinct phases:

1. Execution-Verified Competition:

   • Teachers: Four frontier models (Claude/Anthropic, Codex-GPT/OpenAI, Grok/xAI, and Gemini/Google) act as teachers.
   • Judge: An execution-based judge (unit tests and stdin–stdout checks) replaces LLM judges to eliminate self-preference bias.
   • Fairness Controls:
     • Shared Task Bank: All teachers solve identical problems.
     • Self-Correction: Teachers are allowed two retries on failed tasks using error feedback to model revision capabilities.
     • Intersection Control: The final training curriculum is restricted to problems solved by all teachers, ensuring the student learns from a unified set of tasks where only the solution style varies.

2. Collaboration Modes:
   The verified solutions from all teachers form a single curriculum, utilized in two distinct ways:

   • Imitation (SFT): The student model (Qwen2.5-Coder) is fine-tuned via Supervised Fine-Tuning to imitate the pooled teacher solutions.
   • Verifiable-Reward Reinforcement Learning (RLVR): The same curriculum serves as a reward environment for GRPO (Group Relative Policy Optimization). The reward is defined strictly by the fraction of execution tests passed, not by mimicking the teacher's output.

3. Experimental Setup:

   • Student: Qwen2.5-Coder (7B and 32B variants) trained with LoRA.
   • Hardware: NVIDIA GB10 (128 GB unified memory).
   • Datasets: MBPP (teaching split, with MBPP-test held out), bug-fix tasks via mutation, and hard competition problems from deepmind/code_contests (difficulty 6–9).
   • Metrics: Execution pass@1 on held-out sets.

Key Results

1. Teacher Ranking via Execution:

   • Easy Tasks (MBPP): All four teachers achieved near-perfect scores (99–100%) after self-correction. The authors attribute this to benchmark saturation (training data exposure) rather than skill differentiation.
   • Hard Tasks (Code Contests): A clear hierarchy emerged: Gemini (77%) > Claude (69%) ≈ Codex (69%) > Grok (50%).
   • Fairness: The ranking was derived solely from execution, avoiding family bias. The authors transparently reported and corrected infrastructure artifacts (e.g., API timeouts, billing errors) that initially skewed results.

2. Failure of Imitation (SFT):

   • Applying SFT to the union of verified teacher solutions degraded the performance of already-competent student models.
   • 7B Model: MBPP-test scores dropped from 76.7% (base) to 72.7% (SFT union).
   • 32B Model: Similar degradation observed.
   • Competition Problems: SFT reduced performance from 5.9% to 2.9%.
   • Interpretation: Imitation transfers "style," which competent coders already possess; it does not transfer capability and introduces knowledge conflicts.

3. Success of Verifiable-Reward RL (RLVR):

   • Using the same curriculum as an RLVR environment improved student performance, reversing the negative trend of SFT.
   • Competition Problems: The 7B student improved from a 5.9% base to a peak of 8.8% (a +49% relative improvement) over a 1000-step GRPO run, before a mild regression to 7.4%.
   • Mechanism: The student learned by "doing" (optimizing for test passes) rather than by copying, effectively utilizing the curriculum as a reward landscape.

Significance and Claims
The paper's central claim is that the value of AI-teacher collaboration lies not in pooling answers for imitation, but in jointly constructing a verifiable environment where the student learns through reinforcement.

• Paradigm Shift: The work challenges the standard distillation approach (merging outputs for SFT), demonstrating that for competent students, this approach is harmful. Instead, it positions verified synthetic problems as a reward signal for policy optimization.
• Lineage: The authors frame this as a modern realization of 1999 ensemble learning concepts (Jang [2]), where "virtual data" (execution-verified problems) defines a reward landscape for policy optimization rather than simply augmenting diversity for a single predictor.
• Reproducibility: The paper provides a fully reproducible on-prem pipeline (including framework patches for GRPO on bleeding-edge stacks) and releases all code, data, and the final RLVR adapter, allowing independent verification of all reported numbers.

Limitations
The authors note that the teacher ranking on hard problems is subject to sample size noise (Gemini leads by ~8 points, but the gap between others is small). They also acknowledge that absolute RLVR gains are modest due to sparse rewards in the held-out set (68 problems) and that the results are currently specific to Python and the Qwen2.5-Coder family. The work is explicitly positioned as academic research and benchmarking, not the development of a commercial competing product.