Federation Is Nearly Free, Reasoning Is Not: Tradeoffs for AI Co-Scientists in Protein Characterization Workflows

In the modern laboratory, a new kind of assistant is emerging, one that does not hold a pipette but instead holds a conversation. These are artificial intelligence systems designed to act as "co-scientists," autonomous agents that can read a biological sequence, decide which digital tools to use, and piece together an answer to a complex question. They operate by breaking a large goal into smaller steps, checking their work, and refining their path, much like a human researcher moving from a hypothesis to a conclusion. However, building these systems involves a difficult choice. One path relies on massive, flexible language models that can reason through uncertainty but are expensive, slow, and sometimes inconsistent. The other path uses classical, learned policies—systems trained through trial and error to follow a strict set of rules. These are cheap, fast, and perfectly consistent, but they lack the ability to explain their thinking or adapt to new situations. As scientists begin to deploy these tools across different institutions and computer networks, a critical question arises: does the way these agents are connected or the specific brain they use matter more than the strategy they follow?

A team of researchers at the Pacific Northwest National Laboratory set out to answer this by putting these different approaches to the test in a controlled environment. They focused on a routine but vital task in biology: characterizing proteins. A protein is a molecule that performs specific jobs within living cells, and its function is often determined by its sequence of building blocks. The researchers asked their AI agents to look at a protein sequence and predict its function by routing it through a series of digital tools, such as databases that compare sequences or software that predicts 3D structures. They tested the agents under two different network setups: a simple, single-server setup where all tools were nearby, and a more complex, federated setup where the tools were scattered across different servers, mimicking how real-world scientific labs might share data across borders.

The study compared two main types of agents. The first type used a powerful language model, essentially a sophisticated chatbot, to decide which tool to use next. These models were given either simple instructions or detailed, expert-level prompts to guide their reasoning. The second type used a classical reinforcement learning agent, a system trained through thousands of practice rounds to learn the most efficient path to a correct answer without any natural language reasoning. The researchers ran these experiments hundreds of times, measuring how often the agents got the right answer, how long it took, how much it cost, and whether the agents gave the same answer every time they were asked the same question.

The results revealed a clear hierarchy of importance. The most significant factor determining success was not the network setup or the specific instructions given, but the underlying intelligence of the model itself. When the agents used a top-tier language model, they achieved an accuracy rate of roughly 92 to 94 percent. When they used a smaller, less capable model, accuracy plummeted to between 40 and 50 percent. The way the tools were connected across the network had almost no effect on performance; whether the agents were working in a single room or across a distributed network, their success rates remained nearly identical. This suggests that for these types of tasks, the physical or digital distance between tools is not a major barrier to scientific discovery.

Perhaps the most striking finding concerned the trade-off between flexibility and reliability. The powerful language models could produce high-quality results, but they were expensive and inconsistent. Even the best language model gave a different answer for the same protein in about 2 to 3 percent of cases, and the cost to run these models was significant, ranging from roughly 63 cents to 93 cents per protein. In contrast, the classical learning agent, which had no ability to explain its reasoning or adapt its strategy, was perfect in every single trial. It achieved an accuracy of 88 percent, nearly matching the best language model, but it did so in about 15 seconds and at zero cost. It was also completely consistent, never changing its answer when asked the same question five times.

The researchers found that the choice of strategy depends entirely on the nature of the work. For routine tasks where the answer can be verified against known data, such as identifying a protein with a well-known function, the cheap, fast, and perfectly consistent classical agent is the superior choice. It delivers near-frontier accuracy without the cost or the risk of random errors. However, for open-ended scientific discovery where the answer is unknown and flexibility is required, the expensive language model remains necessary. The study suggests that the value of the detailed reasoning traces provided by language models scales with the novelty of the task; for simple recovery of known facts, the reasoning is less valuable than the certainty of a fixed policy. Ultimately, the work provides a practical guide for scientists building these systems: do not assume that a more complex, flexible brain is always better. For many scientific workflows, a simple, learned rule is not just sufficient, but the most efficient path forward.