ProgRouter: Online Progress-Guided Orchestration for Multi-Agent LLM Workflows under Quality-Cost Tradeoffs

In the rapidly evolving world of artificial intelligence, a new generation of systems has emerged that can tackle complex problems by breaking them down into smaller steps. These systems, known as multi-agent workflows, operate like a team of specialists where different computer programs, each powered by a large language model, collaborate to solve a single task. One agent might plan the strategy, another might write code, and a third might check the results. While this approach allows machines to handle open-ended challenges that were previously impossible, it comes with a significant price tag. Every time the system asks a model to think or act, it consumes energy and money. Using the most powerful, capable models for every single step is often wasteful, while relying only on smaller, cheaper models can lead to errors that require expensive corrections later. The central challenge for researchers has been finding a way to dynamically decide which model to use at each specific moment, balancing the need for high-quality results against the limits of time and budget.

A team of researchers has developed a new system called PROGROUTER to solve this exact problem. Instead of making a single decision at the start of a task or blindly escalating to more powerful models when things go wrong, PROGROUTER acts as an intelligent conductor that listens to the workflow in real time. It constantly monitors how much progress the team has made, how difficult the remaining steps are, and how much of the budget has been spent. Based on this live assessment, it selects the most appropriate model for the next step, ensuring that powerful, expensive models are only called upon when they are truly needed to overcome a specific hurdle. This approach allows the system to maintain high performance while significantly reducing the total energy and cost required to complete a task.

The researchers tested this system on four distinct types of difficult tasks: writing computer code, solving mathematical problems, and answering complex questions that require searching through large amounts of information. They compared PROGROUTER against several existing methods, including strategies that always use the same model, strategies that start small and only switch to bigger models when stuck, and strategies that rely on past guesses about what works best. The results showed that fixed strategies, whether they used only small models or only large ones, consistently failed to meet the energy and cost limits set for the experiments. Small models often got stuck and required too many retries, while large models consumed too much energy with every single call. In contrast, PROGROUTER successfully navigated the middle ground. On the code-writing tests, it achieved a success rate of 93 percent while staying within a strict energy budget of 4,800 joules. On the math problems, it completed tasks in just 19 seconds using only 6,112 joules of energy, outperforming all other methods that met the budget constraints.

What makes PROGROUTER effective is its ability to understand the "state" of the work as it happens. The system uses a scoring mechanism that looks at the workflow from multiple angles. It checks if the overall plan is valid, how many sub-tasks have been completed, whether the work is moving forward or stalling, and the quality of the information generated so far. By combining these different views, the system can tell if a task is nearly finished or if it is facing a major obstacle. When the work is progressing smoothly, the system opts for smaller, faster, and cheaper models to keep costs down. However, if the system detects that the workflow is stuck or that a difficult step remains, it immediately switches to a more powerful model to push the task forward. This decision is made instantly and repeatedly throughout the entire process, ensuring that no energy is wasted on unnecessary heavy lifting, and no time is lost on steps that require more intelligence than the current model can provide.

The study also revealed that this adaptive approach works even in open-ended situations where there is no clear right or wrong answer, such as when a system must gather information from the internet to write a long, detailed report. In these scenarios, where progress is harder to measure, PROGROUTER still managed to produce the highest quality results among all tested methods. It achieved a precision score of 92.1 percent on these complex questions while staying within its energy limits. The researchers found that the system naturally learned to specialize, using small models for the majority of routine steps and reserving the largest, most capable models for the critical moments where their extra power would make a real difference. This dynamic behavior allowed the system to avoid the common pitfalls of other methods, such as getting stuck in loops with weak models or burning through a budget too quickly with powerful ones.

Ultimately, the work demonstrates that the future of efficient artificial intelligence lies not just in building better models, but in building better ways to use them. By treating the selection of a model as a continuous, data-driven decision rather than a one-time choice, PROGROUTER shows that it is possible to achieve high-quality results without the prohibitive costs that currently limit the scalability of these technologies. The system proves that with the right orchestration, a team of diverse models can work together more efficiently than any single model working alone, offering a path toward more sustainable and accessible AI systems that can handle complex, real-world challenges without exhausting their resources.