Scrouting: Cost-Aware Routing of Coding Agents by Scouting the Repository First

Imagine a world where computers can write code to fix broken software, but they are incredibly expensive to hire. Think of these "AI coders" like a team of elite detectives: some are superstars who can solve almost any mystery, but they charge a fortune per hour. Others are smart, capable detectives who cost a fraction of the price but might miss the trickier clues. For a long time, the big question in this field was: "How do we know which detective to hire for a specific case?" Most people tried to guess just by reading the description of the problem (the "issue text"), hoping to predict if the cheap detective would be enough or if they needed to call in the expensive superstar.

But here's the catch: reading the problem description is like trying to solve a crime by only looking at the police report without ever visiting the crime scene. The paper you are about to read explores a smarter idea: what if we send a small, cheap scout to the crime scene first? This scout doesn't try to solve the whole mystery; they just look around, find the broken parts, and write a quick, verified report. Then, a manager uses that report to decide which detective to hire. This approach, called "scrouting," suggests that knowing the context of the problem is far more valuable than just guessing based on the problem's title.

---

The Story of SuperScout: The Detective Who Scouts First

Meet SuperScout, a new system designed to fix software bugs without breaking the bank. The team behind it realized that the old way of hiring AI coders was inefficient. Usually, a router (a decision-maker) looks at a bug report and immediately picks a model. But the authors found something surprising: if you look at the results of past software fixes, the "superstar" models solve almost everything the cheaper models can, plus a few more. This means that trying to route for accuracy (picking the perfect model to get the most fixes) is a dead end; you can't really beat just always hiring the most expensive model.

So, the team flipped the script. Instead of trying to be perfect, they decided to be cost-aware. Their goal wasn't to solve more problems than the best model, but to solve the same number of problems for a tiny fraction of the price.

The Three-Act Play: Scout, Verify, and Dispatch

The SuperScout system works like a three-step assembly line:

1. The Scout (SuperScout-7B): First, a small, 7-billion-parameter AI model (think of it as a junior intern) is sent into the software repository. Its job isn't to fix the bug, but to explore the code, find the specific files involved, and try to write a test that proves the bug exists. It produces a "handoff"—a structured note containing the suspect files, a reproduction test, and some detective notes.
2. The Security Gate (Verify-then-Strip): Here's the magic trick. The intern's notes aren't trusted blindly. Before the main detective sees them, a "sandbox" (a safe, isolated testing room) runs the intern's reproduction test. If the test fails to actually break the code (meaning the intern was hallucinating or wrong), that claim is stripped out and thrown away. Only verified, true facts make it to the next step.
3. The Résumé Router: Finally, a manager looks at the task and the intern's "hidden thoughts" (the internal data states of the scout model) to decide which "fixer" to hire. The pool of fixers includes four different frontier AI models, ranging from cheap to expensive. The router picks the cheapest one that it thinks can handle the job. If the intern didn't find anything, the fixer just works from the original bug report.

The Big Reveal: The Handoff Does the Heavy Lifting

When the team tested SuperScout on 266 real-world Python software bugs (a massive challenge known as SWE-bench Pro), the results were shocking.

• The Score: SuperScout solved 159 out of 266 tasks.
• The Competition: The single best, most expensive model (Claude Opus 4.6) solved 158 out of 266 tasks.
• The Cost: SuperScout achieved this result for about $0.23 per solve, while the expensive solo model cost $1.27 per solve. That's roughly one-fifth of the cost!

The most fascinating discovery, however, wasn't about the router's decision-making. When the researchers removed the router entirely and just sent every task to the cheapest model (Kimi K2.5) with the intern's verified handoff, it solved 159 tasks—the exact same number as the full SuperScout system.

This suggests that the verified handoff is the real hero. The act of scouting the repository and providing a verified report lifted the performance of the cheaper models so high that they could match the expensive ones. The routing decision itself didn't add much magic; it just ensured the cheap models got the handoff.

Why This Matters (and What It's Not)

The paper explicitly rules out a few things:

• Routing for accuracy is a bust: You can't really improve the number of solved tasks by switching models based on the problem text alone; the best model just wins anyway.
• The handoff text isn't for the router: Interestingly, feeding the text of the intern's report to the router actually made it worse. The router works better using the scout's "hidden states" (its internal brain activity) rather than the written notes. The notes are for the fixer, not the manager.
• It's not a magic bullet for everything: The system was tested on a specific set of 266 Python tasks. While the scout model showed it could work on languages it was never trained on, the full system's performance on other benchmarks is still untested.

The authors are careful to note that this isn't a "solved" problem for all time. The results are specific to this set of tasks and the current models. However, the mechanism is robust: by sending a small scout to verify the facts before hiring a fixer, you can redistribute the solving power. The handoff lifts the cheaper models up, making them nearly as good as the expensive ones, while the cost of the scout is negligible (less than half a cent per task).

In short, SuperScout teaches us that in the world of AI coding, context is king. Don't just guess who to hire based on the job description; send a scout to the scene first, verify what they find, and then let the cheapest capable worker do the job. It's a smarter, cheaper way to build software, proving that sometimes, the best way to save money is to do a little homework first.