EvolveTool-Bench: Evaluating the Quality of LLM-Generated Tool Libraries as Software Artifacts
This paper introduces EvolveTool-Bench, a diagnostic benchmark that evaluates the software quality of LLM-generated tool libraries through metrics like reuse, redundancy, and safety, revealing significant health disparities between systems that appear similar based solely on task completion rates.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you hire a team of AI software engineers to build a toolbox for a complex job. Every time they encounter a problem they can't solve, they write a new tool (a Python function) to fix it and add it to their shared toolbox.
Most current tests only ask: "Did the job get done?" If the AI successfully finished the task, we give it a gold star.
This paper argues that's a terrible way to judge software.
It's like hiring a construction crew and only asking, "Did the house stand up?" You wouldn't care if they used rotten wood, built three identical doors in the same room, or left the wiring exposed, as long as the roof didn't leak. Eventually, that house will collapse under its own weight.
Here is the paper broken down into simple concepts:
1. The Problem: The "Black Box" Trap
Current AI agents are getting better at making their own tools on the fly. But we are evaluating them like magic boxes: Input a problem, get a solution. We ignore what's happening inside.
- The Risk: The AI might solve the problem today by writing 500 lines of messy, duplicate, or dangerous code. It works now, but tomorrow, the code is a nightmare to maintain, full of bugs, and breaks things that used to work.
2. The Solution: EvolveTool-Bench
The authors created a new "report card" called EvolveTool-Bench. Instead of just checking if the task was completed, they inspect the quality of the toolbox itself.
They look at the code like a senior software architect would:
- Reuse: Did the AI write a new tool, or did it grab an existing one and use it again? (Good engineers reuse; bad ones rewrite everything).
- Redundancy: Did the AI accidentally build two tools that do the exact same thing? (Waste of space).
- Safety: Did the new tool break an old tool? (Regression).
- Robustness: Does the tool crash if you give it weird data, or does it handle it gracefully?
3. The Experiment: The Race
They tested three different types of AI "engineers" on 99 difficult tasks (like decoding secret binary files or managing complex APIs):
- The "No-Evolution" Team: They only use the tools they started with. They can't learn new tricks.
- The "EvoSkill" Team: They try to learn, but they only change their words (prompts), not the actual code. They are like a chef who changes the recipe instructions but never actually learns to chop a new vegetable.
- The "ARISE" Team: This is the advanced team. When they fail, they write actual new code, test it in a safe sandbox, and if it works, they add it to the toolbox. They also check if the new code breaks old things.
4. The Shocking Results
Here is where the analogy gets interesting.
- The "Task Completion" Score: All teams finished about the same number of tasks (around 63–68%). If you only looked at this, you'd say, "Great job, everyone!"
- The "Software Health" Score: This is where the teams diverged wildly.
- The EvoSkill team (who didn't write code) had a toolbox that was just as messy as the team that did nothing.
- The One-Shot team (who wrote code without testing) actually made things worse. Their toolbox was full of broken, dangerous tools.
- The ARISE team (who wrote and tested code) had the healthiest toolbox. Even though they finished slightly fewer tasks, their tools were reusable, safe, and didn't break old features.
The Analogy:
Imagine three chefs making a stew.
- Chef A (No-Evolution) uses a pre-made mix. It tastes okay, but they can't fix it if they run out of salt.
- Chef B (EvoSkill) talks a lot about how to cook but never actually adds new ingredients.
- Chef C (ARISE) invents a new spice blend. It takes them a little longer to finish the stew, but the spice blend is high quality, reusable for next week, and doesn't ruin the soup.
The paper found that Chef C is the only one building a sustainable kitchen.
5. Key Takeaways for the Real World
- Task Success is a Lie: Just because an AI solves a problem doesn't mean it did a good job. It might have created a "technical debt" (a mess) that will cost us later.
- Testing is Non-Negotiable: The paper found that AI tools that are generated but not tested are actually worse than having no tools at all. They are liabilities.
- The "Judge" Matters: The AI needs a strict "judge" (a system that checks the code) to decide if a new tool is good enough to keep. If the judge is too easy, the AI just spits out garbage code.
- Cheaper Models Work: Surprisingly, a cheaper, faster AI model (Haiku) built a slightly better toolbox than a more expensive, smarter one (Sonnet) when using the right testing system.
The Bottom Line
We need to stop treating AI-generated code as a "black box" that just works. We need to treat it like software engineering. We need to check for bugs, redundancy, and safety. If we don't, we are building digital houses on sand, and they will eventually wash away.
EvolveTool-Bench is the new ruler we need to measure not just if the AI works, but how well it builds for the future.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.