🤖 AI

The Order Is the Guarantee: Verifier-Budgeted Code Deletion with Static-First Learned Proposals

This paper introduces DELSCOUT, a framework that prioritizes the ordering of code deletion candidates over model confidence to safely remove redundant code under finite verification budgets, demonstrating that a hybrid schedule of static and learned proposals maximizes verified deletions while ensuring behavioral preservation through execution authority.

Original authors: Ruitong Li, Binjie Guo, Aisheng Mo, Guowei Su, Han Wang, Jie Li, Ru Zhang

Published 2026-08-06

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Ruitong Li, Binjie Guo, Aisheng Mo, Guowei Su, Han Wang, Jie Li, Ru Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Background: When AI Writes Too Much

Imagine you are building a massive, intricate castle out of LEGO bricks. In the past, you had to place every single brick by hand, which was slow and hard work. Now, imagine a super-fast robot that can snap together entire towers in seconds. This is what modern AI coding models do: they can write working computer programs incredibly fast, often matching or beating human experts at solving puzzles.

But here is the catch: just because a robot can build a castle quickly doesn't mean the castle is tidy. If you ask a robot to "fix this wall" or "add a new door," it often just stacks new bricks on top of the old ones without taking the broken or useless pieces away. Over time, your castle becomes a bloated mess of extra doors, duplicate walls, and hidden traps that no one needs. In the world of software, this is called "technical debt." It makes the code harder to read, harder to fix, and harder to trust.

The big question this paper tackles is: How do we teach an AI to be a good editor, not just a good builder? We know how to ask an AI to write code, but asking it to delete code is dangerous. If the AI deletes the wrong piece, the whole castle might collapse. The challenge is finding a way to let the AI suggest what to throw away, while having a strict safety system that only allows the deletion if it's 100% sure the castle still stands.

The Paper: The "Order of Operations" for Deleting Code

The researchers behind this paper, who call their system DelScout, realized that the secret to safe code deletion isn't just about how "smart" the AI is. Instead, it's about the order in which the AI checks its ideas.

Think of it like a security guard at a museum who has a limited amount of time to check paintings for forgery. The guard has a list of paintings to inspect. If they check the most likely fakes first, they might catch a forgery quickly. But if they check a boring, obviously real painting first, they might run out of time before they ever get to the suspicious one. The paper argues that for deleting code, the schedule (the order of checking) is more important than the AI's confidence score.

The Two Strategies: The "Mixture" and the "Safety Net"

The team tested two different ways to organize the AI's suggestions, using a "budget" of five checks (like having five tickets to inspect five paintings).

The "Validated Mixture" (The Smart Swap):
If the team has data from the specific type of project they are working on, they use a mixed strategy. They keep the first three checks for "safe bets"—simple, obvious things like removing unused imports or short lines of code that a basic computer program can prove are useless. Then, they use the remaining two slots for the AI's "learned" suggestions. These are the AI's best guesses about complex, tricky code that a basic checker can't understand.
- The Result: In their tests on a standard coding benchmark called MBPP, this mix allowed them to successfully delete 9.5% more code than just using the basic safe checks alone. They found 6.7 more successful deletions on average, without needing to run any extra safety checks.
The "Prefix-Preserving Augmentation" (The Safety Net):
What if the AI is working on a totally new type of project where they don't have past data to trust? The researchers realized that swapping out the "safe bets" for AI guesses is risky. If the AI guesses wrong, it might miss a deletion that the basic checker would have found.
So, they designed a "safety net" rule: Never skip the safe bets. They force the system to check all five "safe" suggestions first. Only if all five of those fail does the system get to use its extra slots to check the AI's fancy guesses.
- The Guarantee: This ensures that the system never deletes less code than the basic method would have. It might find more deletions, but it will never miss one that the basic method caught.
- The Cost: The downside is that this safety net sometimes costs more time. Depending on the test, it required 4.8% to 62.5% more safety checks (verifier calls) because the system had to run through the whole list of safe bets before trying the AI's ideas.

What the Paper Rules Out

The authors were very careful to show what doesn't work. They proved that you can't just trust the AI's "confidence score" to decide what to delete. Even if the AI says, "I'm 99% sure this line is useless," it might still be wrong if the testing environment changes.

They also showed that simply training the AI to be "better" at deleting code doesn't solve the problem. If you swap the "safe" checks for "AI" checks without a safety net, the system can actually perform worse when faced with new, unfamiliar code. The paper explicitly rejects the idea that a smarter AI model alone is the solution; the solution is the structure of how the AI and the safety checks work together.

The Bottom Line

The paper concludes that the future of AI coding isn't just about writing more code; it's about keeping code clean. The best approach is a division of labor:

The AI acts as a creative explorer, suggesting tricky, context-heavy deletions that humans might miss.
The Order acts as a traffic cop, making sure the AI doesn't block the path for the boring but reliable safety checks.
The Tests act as the final judge, only allowing a deletion if the code actually runs without crashing.

In their experiments, this method successfully removed redundant code while keeping the software safe. However, the authors warn that this is a tool for maintenance, not a magic wand. If the tests themselves are weak or the code does something critical that isn't tested (like a security check), the AI shouldn't delete it. The goal is to help software stay lean and understandable, ensuring that as AI writes more, our digital castles don't become unmanageable jungles of unused bricks.

Technical Summary: The Order Is the Guarantee: Verifier-Budgeted Code Deletion with Static-First Learned Proposals

1. Problem Formulation

The paper addresses the "inverse problem" of AI coding: rather than generating new code, how should an AI system remove redundant code when execution-verification capacity is finite? While frontier models can generate code that passes benchmarks, "vibe coding" (iterative prompting and patching) tends to accumulate redundant branches, guards, and fallbacks faster than obsolete logic is removed. This creates technical debt that obscures invariants and expands the review surface.

The authors formulate redundant-code reduction as a proposal scheduling problem under a finite budget.

Input: A program $x$ and a set of syntactically valid single-statement AST deletion candidates $C(x)$ .
Constraint: A deterministic verifier $V$ (an execution suite) can only evaluate a limited number of candidates (a budget $m$ ).
Goal: Order the candidates such that the first candidate $c_j$ that passes $V(x \setminus c_j)$ is committed, maximizing the number of accepted deletions (coverage) and the amount of code removed (character reduction).
Core Insight: The order of candidates is the control surface a deployment can reason about, not the model's confidence score. A wrong deletion silently removes required behavior; thus, the execution suite must retain veto power.

2. Methodology: DelScout

The system, DelScout, separates the search (proposing candidates) from the acceptance (verifying candidates). It employs two heterogeneous rankers:

Static Orders: Deterministic, parameter-free strategies.
- Shortest-first: Sorts by span length, character length, then position. Effective for syntactic remnants (unused imports, assignments).
- Zero-reference: Prefers statements with identifiers occurring least frequently in the file.
Learned Ranker: A sequence classifier trained to predict if a deletion is safe based on verifier outcomes. It uses a prompt containing the candidate's node type, text, and context lines. The training objective is listwise, optimizing the probability mass placed on verified deletions near the front of the list, rather than independent classification.

Scheduling Strategies

The paper proposes two distinct schedules to manage the trade-off between static guarantees and learned search, depending on the availability of target-domain validation data:

Validated Mixture (When target validation exists):
- Allocates a fixed budget (e.g., 5 slots).
- Spends the first 3 slots on the deterministic static core ( $S_3$ ).
- Spends the remaining 2 slots on the highest-ranked learned candidates not in the static core.
- Rationale: Empirical validation licenses the exchange of static slots for learned ones to increase coverage.
Prefix-Preserving Augmentation (When target validation is absent):
- Evaluates the complete static ranking first (e.g., all 5 static slots).
- Appends learned candidates only if all static candidates fail.
- Rationale: This ensures that a learned ranking can never displace a static success. It provides a non-decreasing coverage guarantee by construction, regardless of distribution shift, though it may incur higher verifier costs.

3. Key Contributions

Formulation: Defined redundant-code reduction as proposal scheduling under a finite execution-verification budget, cleanly separating learned search from the acceptance boundary.
Scheduling Policies: Introduced a target-validated static–learned mixture and a prefix-preserving augmentation policy. The latter guarantees non-decreasing coverage and character reduction solely through candidate ordering, without requiring calibrated model scores.
Empirical Validation: Reported nine replications across model scales (0.5B, 0.6B, 8B) on MBPP. The study includes objective controls to rule out privileged training losses, budget accounting, cross-benchmark shift analysis, and tests against stronger verifiers (MBPP+, HumanEval+).

4. Results

Fixed-Budget Gains: Across nine runs on MBPP, the validated mixture improved verified-deletion coverage by 9.5% relative (an increase of 6.7 accepted tasks on average) compared to a static baseline, while consuming slightly fewer verifier calls.
Complementarity: The gain is architectural, not due to a specific training objective. Static proposals cover "easy" syntactic candidates (e.g., unused imports), while learned proposals contribute a different order, focusing on contextual control-flow candidates (e.g., conditionals, expressions) that static analysis cannot certify.
Efficiency: Verified deletions per thousand verifier calls increased by 19.5% (from 43.9 to 52.5) because successful proposals often appear earlier in the mixed schedule, stopping the search sooner.
Distribution Shift: Learned-only rankings are unreliable under shift (e.g., performance drops significantly on DS-1000 and BigCodeBench-Hard when trained on MBPP). However, prefix-preserving augmentation restored non-decreasing coverage on these shifted benchmarks, albeit with a quantifiable increase in verifier calls (ranging from 4.8% to 62.5% overhead).
Stronger Verifiers: When tested against MBPP+ (a stricter in-domain specification), the advantage of the mixtures vanished; all policies converged to the same low coverage (6.47%). This demonstrates that scheduling governs the search, but the test suite alone governs what "preserving behavior" means.

5. Significance and Claims

The paper argues that the next frontier of AI coding is not just generating working programs, but maintaining comprehensible and minimal software.

Auditable Division of Labor: The system establishes a clear separation where models widen the search for removable code, order bounds the damage a mis-ranked proposal can do, and execution retains authority over every committed deletion.
Order as the Guarantee: The central claim is that candidate order, not model confidence, is the controllable variable. By preserving a static prefix, the system guarantees that learned models can add value without subtracting established safety.
Limitations: The authors explicitly state that the method relies on benchmark programs, not real-world maintenance histories, and that the absolute coverage gains are modest (moving from ~14% to ~15% on MBPP). They emphasize that "verified" only means verified by the stated oracle; the system does not provide semantic proofs and should not replace human review or safety-critical checks.

In summary, DelScout demonstrates that a disciplined scheduling strategy—combining deterministic static analysis with context-sensitive learned proposals under a verifier budget—can effectively reduce technical debt in AI-generated code while maintaining rigorous safety guarantees.