Technical Summary: The Verifier is the Curriculum

Problem Statement

Post-training code generators using learned judges (e.g., LLMs or multimodal models prompted to score outputs) introduces a structural vulnerability: these judges often rely on surface features that can be manipulated without improving the underlying artifact. This leads to "reward hacking," where models optimize for the judge's blind spots rather than functional correctness. In the context of game generation, a learned judge might reward the presence of visual assets even if the game code is broken or frozen.

The paper addresses the question of what happens when the training signal is instead a deterministic, judge-free, and ungameable filter. Specifically, it investigates whether a strict execution check—verifying if a generated project launches cleanly under a headless engine—can drive effective self-distillation and cross-family generalization.

Methodology

The study utilizes GameCraft-Bench, a benchmark requiring models to generate complete, runnable Godot 4 projects from natural language briefs. The task is inherently difficult because correctness is conjunctive; a single parse error, missing resource, or runtime crash renders the entire project non-functional.

The Verification Ladder

The authors define a hierarchy of verification signals (Figure 1):

1. BUILD (Lenient): Checks if the project opens and quits without immediate parse errors. This is "near-vacuous" as it accepts many broken projects (99.9% pass rate).
2. Strict-Launch (Training Gate): A deterministic check requiring the project to launch cleanly under a headless engine (exit code 0, no parse/load/runtime errors). This is ungameable and precise.
3. Execution Grounding (Dynamic Audit): A secondary signal that drives the SceneTree and measures runtime state changes to confirm functional substance.
4. Visual Judge (Gameable): The benchmark's official metric, shown to be manipulatable (e.g., by swapping assets while freezing code).

Iterative Self-Distillation

The core method is rejection-sampling self-distillation filtered by the strict-launch gate:

1. Initialization: Start with a supervised model ($M_0$) trained on 111 gold projects from 10 game families.
2. Generation: For each training-family brief, generate $K=8$ candidate projects.
3. Filtering: Materialize candidates and run the strict-launch check. Only candidates that launch cleanly (and have $\ge$3 files) are kept.
4. Fuel Creation: The kept candidates form the "fuel" ($F_t$) for the next round.
5. Retraining: A new LoRA adapter is trained on the base model using the union of the original gold pool and the new fuel ($D_{t+1} = D_t \cup F_t$).
6. Iteration: This process repeats for three rounds ($RFT-r1, r2, r3$).

Controls

To isolate the cause of performance gains, the authors employ three matched controls:

• Gold-Duplication (CTRL): Adds exact duplicates of gold training projects instead of self-generated fuel.
• Count-Matched: Swaps the generator (using $M_1$ instead of $M_0$) to isolate the effect of fuel quality vs. quantity.
• BUILD-Gated: Replaces the strict-launch filter with the lenient BUILD check while keeping the generator and budget identical.

Key Results

The study focuses on cross-family generalization, evaluating performance on four unseen game families (horror, rhythm, puzzle, shooter) comprising 25 held-out tasks.

• Compounding Gains: Under the strict-launch gate, self-distillation compounds performance significantly:

  • Per-candidate clean-launch rate: Increases from 8.8% (SFT) to 13.6% (r1), 30.9% (r2), and 42.2% (r3).
  • Best-of-K Coverage: Increases from 18/25 (SFT) to 25/25 (r3), reaching the "gold ceiling" where every held-out task has at least one clean candidate.
  • All improvements are statistically significant ($p < 10^{-4}$ for later rounds).

• Attribution of Gains:

  • Not Data Duplication: The Gold-Duplication control (CTRL) actually regressed performance to 5.6%, proving that mere data volume or repetition is insufficient and potentially harmful.
  • Filter Precision is Causal: The BUILD-gated control (using the lenient filter) erased all gains, returning performance to the baseline (8.6%). This isolates verifier precision as the causal factor, not the optimizer or the data volume.
  • Quality vs. Quantity: Decomposing the jump from round 1 to 2 reveals both a quality channel (+8.8 percentage points) and a quantity channel (+8.5 percentage points), with quality being the dominant driver.

• Functional Validity:

  • Static Audit: Clean projects from later rounds are richer (median 473 lines of GDScript vs. 331 for SFT) and contain more files.
  • Dynamic Grounding: An independent execution-grounding signal confirms that the gains are functional. The number of tasks with "grounded" (functionally active) candidates rises monotonically (10 $\to$ 15/25), and self-distillation produces far more grounded candidates than gold duplication (16 vs. 5 at matched budget).

Significance and Claims

The paper's central thesis is that "The verifier is the curriculum."

1. Deterministic Filters Enable Compounding: Unlike learned judges, which can be gamed and lead to superficial optimization, a deterministic, ungameable execution check defines a training distribution that forces the model to learn functional correctness.
2. Precision Drives Transfer: The ability of self-distillation to compound capabilities across unseen families is directly tied to the precision of the acceptance filter. A high-false-positive filter (like BUILD) causes the loop to compound broken projects, while a precise filter compounds working ones.
3. Functional Substance: The improvements are not "stub-inflated" (projects that launch but do nothing); they represent genuine increases in functional complexity and runtime behavior.

The authors conclude that in self-improvement loops, the ceiling of capability is set by the integrity of the verifier, not merely by data volume. While the study is specific to game generation as a verifiable testbed, the principle suggests that for any domain where a deterministic, ungameable check exists, iterative self-distillation can drive robust, cross-domain generalization.