Imagine a world where computer programs don't just follow a rigid instruction manual written by a human, but can actually learn from their mistakes and rewrite their own rules to get better. This is the dream of "self-evolving agents." Think of them like a video game character that, after losing a boss battle, doesn't just restart; it pauses, figures out why it lost, changes its own strategy guide, and tries again. For a long time, scientists believed that for this to work, a human had to build a very specific, step-by-step recipe for the computer to follow. This recipe was like a strict coach telling the player exactly when to look at the scoreboard, when to change their weapon, and when to stop practicing. But as computers have gotten smarter—becoming "frontier models" capable of complex reasoning—researchers started to wonder: Do we still need that strict coach? Or can the computer figure out its own practice routine if we just give it a goal and a budget?

This paper, titled "Rethinking Self-Evolving Agents," dives right into that question. The researchers, Hui Xue and Fan Yang from Microsoft Research, set up a fascinating experiment to see if a super-smart AI can organize its own learning process without a pre-written script. They compared the old-school method, where the framework (the "coach") dictates every step of the improvement process, against a new method called "Open-Ended Optimization" (OEO). In OEO, the AI is given a clear goal, a list of allowed moves, and a limit on how much "energy" (or tokens) it can spend, but it gets to decide how to use that energy to get better. It's like giving a chef a list of ingredients and a time limit, but letting them decide whether to bake a cake or stir-fry a meal, rather than forcing them to follow a specific recipe.

The results were surprisingly clear. When using a top-tier AI (GPT-5.5) as the "chef," the open-ended approach was just as good, and often much better, than the strict, pre-written recipes. In fact, across 14 different tests, the open-ended AI won 12 times, tied once, and only lost by a tiny margin (0.21 percentage points) once. Even more impressively, it did this while using only about 34.3% of the "energy" budget that the strict method required. It turns out that if the AI is smart enough, it doesn't need a coach to tell it how to practice; it can figure out the best way to learn on the fly.

However, the paper also draws a very important line in the sand. This freedom only works if the AI is truly capable. When the researchers tested the system with a "medium" or "weak" AI, the open-ended approach failed. The weaker AIs got confused, couldn't figure out what to do next, and produced broken results. In those cases, the strict, pre-written recipe (the "coach") was essential to keep things moving. The study suggests that the strict rules aren't always necessary, but they act as a helpful scaffold for less capable brains. The key takeaway isn't that we should throw away all rules, but that we should let the smartest AIs drive their own learning, while keeping the strict training wheels on for those still learning to walk.

The Core Discovery: Who Should Drive the Bus?

The main finding of this paper is that for highly capable AI models, the "prescribed optimization pipeline"—that rigid, step-by-step plan humans used to write for the computer—is no longer a requirement for success. In the past, systems like SKILLOPT (which uses a staged, step-by-step training pipeline) and GEPA (which uses a reflective evolutionary search) were the gold standard. They worked like a factory assembly line: the framework decided exactly how to gather evidence, how to edit the code, and when to stop.

The researchers introduced Open-Ended Optimization (OEO) to test if the AI could take the wheel. In OEO, the framework still sets the rules of the road (the goal, the budget, and what data is allowed), but the AI gets to decide the route. It can choose to gather evidence, rewrite its own instructions, or stop whenever it thinks it's done.

When they pitted the GPT-5.5-driven OEO against the strict methods, the results were striking. In 8 different benchmark settings, OEO beat or tied the strict SKILLOPT method in every single case. Against the GEPA method, OEO won 5 out of 6 times, with the only loss being a microscopic 0.21 percentage points. Perhaps the most exciting part is the efficiency: OEO used a median of only 34.3% of the token budget that SKILLOPT was configured to use. This means the AI didn't just get better; it got better while using less than half the "fuel" of the old method.

Ruling Out the "Easy Way Out"

Before celebrating, the researchers had to make sure they weren't being fooled. They asked: "Is the AI just getting lucky because it already knew the answer?" or "Did it just rewrite the code once and call it a day?"

To test this, they ran a "zero-interaction" control. They gave the AI the initial skill and asked it to rewrite it once without any practice or feedback loops. The results showed that this "one-shot" rewrite was not enough. On some tasks, the one-shot rewrite actually made the AI worse. On others, it improved the score, but still fell short of the OEO system by significant margins (for example, on the LiveMath task, the one-shot rewrite was over 30 percentage points behind the interactive OEO). This proves that the gains didn't come from the AI's pre-existing knowledge alone; they came from the interactive loop where the AI could learn, adapt, and refine its strategy over time.

The Capability Boundary: When Do We Need a Coach?

The paper also discovered a crucial limit to this freedom. The "open-ended" approach is not a magic bullet that works for every AI. The researchers tested the system with "medium" and "weak" optimizers (less capable models).

Here, the story flipped. When the optimizer was only "medium" capable, the strict SKILLOPT pipeline actually performed better than OEO. When the optimizer was "weak," the OEO system completely broke down; the weak AI couldn't even produce a valid action through the open interface, and no progress was made. In contrast, the strict pipeline kept the weak AI moving, even if it wasn't the most efficient.

This suggests that the strict, pre-written pipeline acts as a "scaffold." For a genius-level AI, the scaffold is unnecessary and might even get in the way. But for a learner who is still finding their footing, that scaffold is essential to prevent them from falling off the cliff.

The Path vs. The Destination

Finally, the researchers looked under the hood to see how the AI was learning. They compared the "journey" (the specific edits and changes the AI made) with the "destination" (the final performance score).

They found that the two methods took very different paths. The OEO system made bigger, bolder changes and rewrote its own history more often than the strict SKILLOPT system. However, despite these wildly different routes, they often ended up solving the same problems. In many cases, the final skills produced by both methods were correct on the same set of test questions, even though the text of the skills looked very different.

This tells us that there isn't just one "right" way to solve a problem. The strict pipeline forces the AI down a narrow, safe path, while the open-ended approach lets it explore a wider, more chaotic landscape. Both can lead to the same destination, but the open-ended approach gets there with more flexibility and less fuel, provided the driver is skilled enough to handle the open road.

The Bottom Line

The paper concludes that we should rethink how we build self-evolving agents. We don't need to force every AI into a rigid, pre-written box. Instead, we should adopt a "capability-adaptive" approach. If the AI is smart enough (a "frontier" model), we can hand it the keys and let it compose its own learning process online. We just need to keep the guardrails in place: the goals, the budget, and the rules of engagement. But for less capable models, the old-school, strict coaching is still the best way to ensure they learn effectively. It's a shift from "one size fits all" to "right tool for the right brain."