Joint Optimization of Tool Creation and Use for Large Language Model Agents

Human progress has always depended on the tools we build. Rather than solving every new problem from scratch, we rely on instruments refined over generations, from the simple lever to the complex software that runs our modern world. Large language models, the advanced computer programs capable of understanding and generating human language, face a similar limitation. While they possess vast amounts of information stored in their internal memory, they struggle with tasks requiring exact calculation, access to up-to-date facts, or reliable logical reasoning. To overcome this, researchers have taught these models to use external tools, such as calculators or search engines, much like a human might reach for a calculator to solve a difficult math problem. However, current systems are limited because they can only use tools that humans have already written and provided. If a human has not built a specific tool for a new task, the model is stuck.

This limitation has sparked a shift toward a more dynamic approach: teaching models to create their own tools on demand. The idea is for a model to look at a problem, write a small piece of code to solve it, and then use that code to find the answer. Yet, existing methods for doing this have a fundamental flaw. They typically use a powerful, expensive model to write the tool and a weaker model to use it. Because the model that writes the tool never has to use it itself, it has no incentive to write tools that are actually clear, reliable, or easy to invoke. It is like a carpenter who builds a hammer but never has to swing it; they might make a handle that is too slippery or a head that is too heavy, simply because they do not feel the consequences of a bad design.

A team of researchers has introduced a new framework called SMITH to solve this disconnect. The core innovation is a training method that forces a single model to act as both the toolmaker and the tool user. In this system, the model is given a set of example problems and asked to write a tool, which consists of a piece of code and a clear description of how to use it. Immediately after, the same model must try to use that exact tool to solve a different, unseen problem. If the description is confusing or the code is broken, the model fails to solve the problem and receives a clear signal that it needs to improve. This creates a direct feedback loop where the model learns that a tool is only as good as its ability to be used by itself.

The researchers tested this approach using a model with four billion parameters, a size that is relatively small compared to the massive models often used in this field. They trained it on thirteen different types of reasoning tasks, ranging from arithmetic puzzles to logical games. The results were striking. The trained model achieved a high level of accuracy on problems it had never seen before, outperforming methods that rely on complex, human-designed instructions or even larger models that are simply prompted to write tools at the moment of use. In fact, the small model trained with this method performed better than an untrained model with thirty billion parameters when it came to creating tools for new tasks. This suggests that the ability to learn how to build and use tools is more important than simply having a larger amount of raw memory.

Perhaps the most surprising finding was how well these self-made tools transferred to other models. The tools written by the small, trained model were able to help a much smaller model with only 350 million parameters perform as well as a much larger, untrained model. Conversely, when the tools written by the small model were given to a very large model to use, the large model performed better than it did when it tried to write its own tools. This indicates that the tools created by the SMITH framework are genuinely high-quality and reusable, not just tricks that work for one specific model. The tools were so effective that they improved performance on entirely new types of problems, such as interpreting complex tables or answering questions about images, even though the model was never trained on data involving tables or images.

The study also revealed the importance of how the learning process is structured. The researchers found that simply asking a model to write code is not enough; the model must also be judged on whether the description of the tool matches the code and whether the tool actually works when invoked. By separating these different aspects of quality and rewarding the model for getting each one right, the system avoids common pitfalls where a model might write code that looks good but cannot be used, or write a description that is clear but does not match the code. This careful balancing act allowed the model to learn a robust skill: the ability to create a tool that is both correct and usable.

In the end, this work demonstrates that the path to more capable artificial intelligence may not lie in building ever-larger models, but in teaching models to be better at building and using their own capabilities. By closing the loop between creation and use, the researchers have shown that a model can learn to design tools that are reliable enough to be used by itself and by others. This approach offers a scalable way to expand the abilities of artificial intelligence, allowing it to adapt to new challenges by creating the specific instruments it needs, rather than waiting for humans to provide them. The findings suggest that the future of tool-augmented intelligence lies not in static libraries of pre-made tools, but in the dynamic, self-improving ability of models to craft their own solutions.