Experimental reasoning in large language models

Science has always been a conversation between a question and the physical world. A researcher proposes an idea, designs a test to see if the world agrees, and then watches what happens. If the result is unexpected, the researcher changes the plan and tries again. This cycle of asking, acting, and adjusting is the engine of discovery. For decades, scientists have tried to build machines that can do this work for them. Early attempts focused on automation: robots that could follow a strict set of instructions, like a chef following a recipe to the letter. These machines are excellent at repetition, but they struggle when the kitchen catches fire or an ingredient is missing. They cannot think on their feet. The question now is whether the new generation of artificial intelligence, specifically large language models, can do more than just follow orders. Can these computer programs actually reason through a scientific experiment, deciding what to do next based on what they see, just as a human scientist would?

A team of researchers at Duke University set out to answer this by building a system they call PEARL. They connected a standard computer program, designed to understand and generate human language, to a simple laboratory robot and a webcam. The robot could move liquids between small containers, and the camera could watch the results. The researchers gave the system a goal, such as "create a rainbow in the tray of containers," but they did not provide a step-by-step guide on how to do it. They did not tell the system which colors to mix, in what order, or even which bottles contained which dyes. The system had to figure out the entire process on its own. It had to look at the available tools, decide what to mix, perform the action, watch the result, and then decide if it had succeeded or if it needed to try something different.

The system worked by dividing the thinking process into different roles that talked to each other. One part of the system planned the experiment, another predicted what would happen, and a third watched for mistakes. When the robot mixed two liquids and the color came out wrong, the system did not just keep going. It stopped, looked at the evidence, and realized its prediction was incorrect. It then revised its plan and tried a new mixture. In one test, the researchers asked the system to make purple using only red and green dyes. The system correctly realized that this was impossible and refused to mix them, asking for a better instruction instead. In another test, a human secretly rotated the tray of containers while the system was working. The system noticed that the colors were in the wrong places, updated its mental map of the tray, and adjusted its actions to mix the correct liquids in the new locations. It did this without being told that the tray had moved.

The researchers pushed this further by asking the system to perform a real biochemical test to measure how fast an enzyme works. This required precise measurements and careful timing. The system had to decide how much liquid to mix, when to take a picture, and how to analyze the changing colors over time. When the researchers introduced a problem, such as adding too much liquid to one container by mistake, the system spotted the error, ignored the bad data, and used the remaining good data to finish the calculation. When the initial mixture was too strong to measure accurately, the system realized it needed more information. Instead of guessing, it designed a new, smaller experiment just to dilute the mixture and see the true colors. It then used that new information to fix the main experiment. In every scenario, the system successfully translated a vague goal into a physical action, interpreted the outcome, and decided what to do next.

The study also tested how much help the system needed to succeed. The researchers tried running the same tasks with a simpler setup that had fewer internal roles and no shared memory of the experiment's history. This simpler version often failed to connect the dots between what it saw and what it should do next. It might spot a problem but forget to fix it, or get lost in the sequence of steps. However, when the system had a persistent record of the experiment and a few specialized roles to check each other's work, it succeeded almost every time. This suggests that the ability to reason through an experiment comes largely from the underlying intelligence of the language model itself, but that a little bit of structure helps it stay focused and reliable.

These findings suggest a shift in how we might think about laboratory automation. Instead of programming robots to follow every possible contingency in advance, we might soon be able to give them a goal and let them figure out the details. The system did not need a pre-written recipe for handling a rotated tray or a concentrated dye; it figured those situations out in the moment. The researchers emphasize that this is not yet a fully autonomous scientist that can invent new theories or ask deep questions about the universe. The experiments were limited to specific tasks with visual results. However, the work shows that general-purpose computer models can already act as a thinking layer between a scientific goal and the physical world. They can observe, interpret, and adapt, turning the act of experimentation into a part of the reasoning process itself. This moves the field from simple automation toward a future where machines can participate directly in the scientific process, handling the unexpected and learning from what they see.