VeriTrace: Human-Like Temporal Exploration Completes Agentic Action Space

Imagine you are trying to teach a robot to build a complex machine, like a clockwork toy, using only a written description. You give the robot the instructions, and it tries to build it. But sometimes, the toy doesn't work. In the world of computer science, this is called "hardware design," and the instructions are written in a special language called Verilog. For a long time, smart computer programs known as Large Language Models (LLMs) have been getting really good at reading these instructions and writing the code to build the machine. However, when the machine breaks, these programs often get stuck. They can see that the toy is broken, but they can't figure out why because they aren't allowed to look closely at the moving parts inside. They are like a mechanic who can hear an engine knocking but isn't allowed to open the hood or use a wrench to check specific gears. This paper, written by researchers at the University of Maryland, tackles this exact problem: how to give these AI "mechanics" the freedom to investigate the machine's inner workings just like a human expert would.

The researchers, Yu-Tung Liu and Cunxi Yu, noticed that while AI can write the initial code, it struggles to fix it when things go wrong. Previous attempts to help the AI debug involved giving it a "waveform"—a visual map of how the machine's signals change over time. But these old systems were like giving a detective a photo of a crime scene but forbidding them from choosing which clues to examine or which time of day to focus on. The AI was forced to look at a narrow, pre-selected view, which often missed the real cause of the error. The authors call this limitation an "incomplete action space." They argue that to truly fix the code, the AI needs to be able to choose which signals to inspect, when to look at them, and how long to keep investigating, mirroring the way a human engineer thinks.

To solve this, the team created a new system called VeriTrace. Think of VeriTrace as a team of specialized AI workers. One worker writes the code, another checks if it works, and a third, called the "Inspector," acts like a curious detective. Unlike previous systems, this Inspector has total freedom. It can say, "I think the problem is in this specific gear, but only during the second second of operation," and then zoom in to check exactly that. If the first guess is wrong, the Inspector doesn't give up; it forms a new theory, picks a different time window, and looks again. This process is called "Agentic Temporal Exploration." It allows the AI to build a hypothesis, test it against the evidence, and refine its understanding step-by-step, just like a human would.

The results of this new approach are quite impressive. When tested on a standard set of 156 coding challenges called VerilogEval-V2, VeriTrace achieved a perfect score of 100% Pass@1. This means that on the very first try of its final corrected code, it got every single problem right. This is the first time an open-source system has reached this level of perfection on this specific benchmark. Even when compared to other strong systems using the same underlying AI brain (Claude Sonnet 4.0), VeriTrace outperformed them by 5.1%, proving that giving the AI the freedom to explore and investigate is the key to closing the final gap in accuracy.

Interestingly, the researchers found that this freedom didn't just make the AI smarter; it also made it more efficient. By only asking for the specific pieces of information it needed at each step, rather than dumping the entire history of the machine's operation into its memory, VeriTrace reduced the amount of data it had to process by 18%. This suggests that letting the AI ask the right questions is not only better for finding the answer but also saves a lot of computational energy. The paper concludes that while the AI models themselves are powerful, the real breakthrough comes from how we let them use their tools. By giving them the agency to explore time and signals freely, we can turn them from rigid code generators into true problem-solvers.