Technical Summary: Recursive Self-Training Collapse in Code LLMs

1. Problem Statement

The paper investigates the risk of recursive self-training collapse in Code Large Language Models (LLMs). As AI coding tools (e.g., GitHub Copilot, Cursor) increasingly generate code that is merged into public repositories, future training corpora may consist largely of AI-generated code. If a model is fine-tuned on data it previously generated, and that data is subsequently used to train the next iteration without external quality control, the system enters a "self-consuming" loop.

The core problem is that without fresh human data or exogenous quality signals, this loop can lead to model collapse:

• Distributional Drift: The model learns to imitate its own previous outputs rather than the original data distribution.
• Error Amplification: Bugs, incomplete logic, and stylistic narrowness in generated code are copied and reinforced in subsequent generations.
• Loss of Diversity: The model converges to low-variance, low-quality outputs, failing to solve programming tasks effectively.

The paper specifically contrasts three regimes of recursive training:

1. No Review (Ungated): All generated code is reused.
2. Human-Gate Review: Code is filtered by model-independent signals (e.g., compilation, static analysis, execution tests).
3. AI-Self-Gate Review: The code LLM evaluates and filters its own outputs using internal signals (e.g., perplexity, binary self-scoring).

2. Methodology

Theoretical Framework

The authors formalize recursive training as gated distributional reweighting.

• Ungated Recursion: The training distribution at iteration $t$ is simply the model's output distribution $p_{\theta_t}(c|x)$. The next model minimizes the KL divergence to this distribution, enforcing only self-consistency.
• Gated Recursion: A sample is accepted with probability $r(x, c)$. The training distribution becomes $q_{\theta_t}(c|x) \propto p_{\theta_t}(c|x)r(x, c)$.
• Theoretical Analysis:
  • Spectral Concentration: The authors prove (Proposition 2.1) that recursive retraining amplifies dominant representation directions while suppressing others, leading to a collapse in the covariance matrix of the representation space (reduced rank/diversity).
  • Degeneracy of AI Self-Gating: The paper proves (Theorem 2.3) that if an AI reviewer is coupled to the generator (sharing parameters or distribution), and the reviewer becomes "self-confirming" (assigning high scores to the generator's likely outputs regardless of quality), the gating mechanism mathematically degenerates to ungated self-training. In this "rubber-stamp" regime, the filter loses its ability to reject degraded outputs.
  • Stability Condition: AI-gated training only avoids collapse if the AI reviewer remains calibrated to an exogenous human standard (Theorem 2.6).

Experimental Setup

• Models: Four code LLM families were tested: SantaCoder (1.1B), StarCoder2-3B, Qwen2.5-Coder-1.5B, and Code Llama-7B.
• Benchmarks: HumanEval, MBPP, LiveCodeBench, and their augmented versions (HumanEval+, MBPP+).
• Protocol: Recursive fine-tuning over 5 rounds. In each round, the model generates code, a filtering strategy selects 5,000 samples, and the model is fine-tuned on this set for 3,000 steps.
• Filtering Strategies:
  • Vanilla: No filter.
  • Human Gates: Compile (syntax check), Quality (static metrics like repetition/length), Compile+Quality.
  • AI Self-Gates: Perplexity (lowest 25% by model likelihood), Binary Classifier (top 25% by logit-difference score).

3. Key Results

Universal Degradation

Across all model families and filtering strategies, recursive self-training leads to performance degradation compared to the pre-training baseline.

• No Review: Collapses the fastest and most severely. For example, Qwen2.5-Coder dropped from a HumanEval+ baseline of 0.372 to 0.043 (−88.4%) by Round 5.
• Human Gates: Slow the rate of collapse but do not stop it. While Compile and Quality filters preserve executable validity and delay semantic drift, they cannot prevent long-horizon degradation.
• AI Self-Gates: Exhibit a "rubber-stamp" failure mode. Initially, they may select high-performing samples (e.g., Binary Classifier achieved the highest Round 5 HumanEval+ for three models), but their filtering efficacy degrades over time.
  • Divergence of Metrics: A critical finding is the divergence between the filter's internal signal and actual correctness. For the Binary Classifier, acceptance scores (the model's confidence in its own code) rose while benchmark correctness fell. This confirms the theoretical prediction that the self-gate becomes self-confirming.

Cross-Model Consistency

The collapse phenomenon is not specific to a single architecture.

• LiveCodeBench: All models approached near-zero pass@1 on this out-of-distribution benchmark by Round 5 under vanilla training, indicating that recursive self-training disproportionately degrades complex reasoning capabilities.
• Magnitude of Collapse: Models with stronger initial distributions (e.g., Qwen2.5-Coder) showed sharper distributional shifts when contaminated with self-generated outputs, though all models eventually converged toward a low-performance "collapse attractor."

Filter Pass Rates

• Human Gates: Maintained stable pass rates (e.g., Compile remained high because it relies on external execution).
• AI Self-Gates: Showed increasing pass rates over rounds (e.g., Perplexity pass rate rose from ~0.17 to ~0.24), indicating a loss of discriminative power as the model's internal scoring distribution drifted with its degraded generation distribution.

4. Contributions

1. Empirical Study of Code LLM Collapse: Provided a systematic evaluation of recursive self-training across five filtering regimes and four distinct code LLM families, demonstrating that collapse is a cross-model phenomenon.
2. Formulation of Review as Gated Reweighting: Theoretically framed code review as a sampling reweighting mechanism, explaining why exogenous gates can slow collapse while endogenous gates fail.
3. Identification of Exogeneity as Key: Established that the critical difference between successful and failed recursive training is whether the acceptance rule is exogenous (fixed, model-independent) or endogenous (coupled to the generator).
4. Theoretical Proof of Degeneracy: Proved that AI self-gating degenerates to ungated self-training under self-confirming conditions and provided a spectral analysis showing how recursive retraining concentrates representation variance into a low-dimensional subspace.

5. Significance and Claims

The paper argues that stable recursive code LLM training requires exogenous verification rather than model-coupled self-review.

• Limitation of AI Review: The study suggests that relying on AI systems to review their own code for future training is inherently unstable. As the generator degrades, the self-reviewer's criteria drift, eventually accepting degraded outputs as high quality (the "rubber-stamp" regime).
• Role of Human Gates: While Human gates (compilation, static analysis) are more robust and preserve executable validity, they are insufficient to stop long-horizon semantic drift entirely. They act as a brake, not a cure.
• Implication for Future Systems: The findings warn against the "self-consuming" loop where AI-generated code enters repositories and is immediately used to train the next generation of models without fresh human data or strong external signals. The paper concludes that to avoid collapse, the verification signal must remain outside the model's own preference distribution.

The authors remain modest, noting that their experiments focused on Python and models up to 7B parameters, and that longer horizons, larger models, and stronger execution-based gates remain open areas for future investigation.