One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows

In the rapidly evolving world of artificial intelligence, a new generation of computer programs known as "agents" has emerged. Unlike earlier chatbots that simply answered questions or wrote text, these agents are designed to take action. They can browse the web, run software code, or call upon digital tools to perform tasks, much like a human employee would. For years, researchers have tested these agents by asking them to solve problems that have clear, easy-to-check answers, such as fixing a broken line of code or finding a specific fact on a website. Success in these areas was measured by whether the agent produced a correct final output or a valid command. However, the real world of business is rarely so simple. In a corporate office, work often involves a long chain of events where the computer system must be updated correctly, policies must be followed, and the state of a database must change in a specific way. If an agent makes a mistake in the middle of a process, or if it changes the wrong record, the consequences can be real and costly. The critical question for the future of this technology is not just whether an agent can find a way to succeed once, but whether it can do so reliably, every single time, without causing unintended damage.

A team of researchers from the University of Pittsburgh, Northwestern University, the University of California, Irvine, and Microsoft has built a new testing ground to answer this question. They created a digital environment called ThinkingBox, which acts as a safe, isolated sandbox where these AI agents can interact with simulated users and complex business tools. Inside this sandbox, the agents are given realistic tasks, such as processing a refund for an online order, changing a hotel reservation, updating an insurance claim, or handling an employee's request for IT support. The environment is designed to mimic the messy reality of office work, where information is often incomplete, rules are strict, and every action leaves a permanent mark on a digital record. The researchers did not just ask the agents to talk; they required them to actually change the state of the system. To ensure the tests were fair and repeatable, the sandbox resets itself completely after every attempt, ensuring that no two trials share the same data or hidden history.

The researchers then populated this environment with 507 distinct tasks across five different industries: retail, travel and hospitality, auto insurance, banking, and consulting. They invited a wide range of the most advanced artificial intelligence models available, including both proprietary systems from major tech companies and open-source models, to attempt these tasks. Each model was given twenty chances to solve each problem. The evaluation was rigorous. An agent did not pass simply because it gave a polite answer or made a tool call that looked correct on the surface. Instead, the system checked the final result against the required outcome. Did the refund actually appear in the database? Was the hotel room correctly marked as changed? Did the insurance claim get updated without accidentally altering the wrong customer's policy? The system also checked for "collateral effects," meaning it looked to see if the agent had accidentally changed something it was not supposed to touch, a common and dangerous error in real-world software.

The results revealed a significant gap between what the agents could do occasionally and what they could do dependably. The best-performing model managed to solve the task correctly on its first try about 65 percent of the time. While this might sound impressive, the picture changed when the researchers looked at reliability. When they checked how often a model could solve the task successfully in all twenty attempts, the number dropped dramatically to just 25 percent. This means that while a smart agent could often stumble upon a successful path if given enough tries, it could not consistently repeat that success without making errors. The researchers found that many failures were not due to the agents being unable to understand the request or failing to speak to the user. Instead, the most common problem was that the agents failed to recover when a tool gave an error message, or they performed the correct action on the wrong piece of data. For instance, an agent might successfully call a tool to update a record but apply the change to the wrong customer, or it might stop the process before completing a necessary step, leaving the system in a broken state.

The study also highlighted that different models struggled with different types of work. Some models were quite good at handling retail orders but performed poorly when dealing with complex insurance policies or banking regulations. This suggests that high performance in one area does not guarantee an agent will be reliable in another. The researchers observed that even when an agent produced a perfect-sounding final message to the user, the underlying database often remained unchanged or was corrupted. This finding challenges the current way many systems are evaluated, which often focuses on the quality of the conversation or the validity of the tool call rather than the actual outcome. The paper concludes that for artificial intelligence to be trusted with consequential business work, it must move beyond simply finding a successful path once and learn to execute that path with consistent, repeatable precision. The ThinkingBox sandbox and its benchmark are now available to the public, providing a new standard for measuring whether these digital workers are truly ready for the real world.