FuzzingBrain-Bench V1: Evaluating Open-Ended Bug Discovery by LLMs
This paper introduces FuzzingBrain-Bench, a novel benchmark designed to evaluate the open-ended bug discovery capabilities of large language models by measuring their ability to generate inputs that trigger distinct crashes in sanitizer-instrumented harnesses across 77 challenges from 43 open-source projects.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the vast, invisible landscape of software that runs our modern world, hidden flaws are a constant reality. These flaws, known as vulnerabilities, are like tiny cracks in a dam; if left unpatched, they can allow attackers to breach systems, steal data, or shut down critical services. For decades, finding these cracks has been a manual, labor-intensive process where human experts meticulously inspect code, looking for patterns that suggest a weakness. However, as the volume of software grows, the number of reported flaws has reached record highs, overwhelming human capacity to keep up. This has led researchers to ask a new question: can artificial intelligence, specifically the large language models that can write and understand code, be taught to find these bugs faster and more effectively than before? The challenge is not just to spot a flaw on paper, but to prove it exists by creating a specific input that forces the software to fail, a process that requires deep understanding of how code behaves under stress.
A team of researchers at Texas A&M University has taken a significant step toward answering this question by creating a new testing ground called FuzzingBrain-Bench. Unlike previous tests that asked artificial intelligence models to simply identify a known weakness or reproduce a specific, pre-announced error, this new benchmark asks the models to act as independent explorers. The researchers provided the models with the source code of real, open-source software projects that were known to contain bugs, along with a specialized testing tool called a harness. This harness is a controlled environment designed to run the software and watch closely for any sign of failure. The models were not told where the bugs were or what they looked like. Instead, their task was to generate thousands of different inputs, feed them into the software, and see if they could trigger a crash. A crash, in this context, is a moment where the software stops working unexpectedly, often revealing a hidden vulnerability. The models were rewarded not for finding a specific, pre-chosen bug, but for discovering as many distinct types of failures as possible, regardless of whether those failures matched the original known issues.
The benchmark consisted of seventy-seven challenges drawn from forty-three different software projects, ranging from image processing libraries and video codecs to database tools and web servers. These projects were written in three major programming languages: C, C++, and Java. To ensure fairness and prevent the models from accessing external information, each challenge was packaged into a secure, isolated container. Inside this container, the model could only see the code and the testing tool; it had no access to the internet, no history of the bug reports, and no information about how the software was fixed. The model had to rely entirely on its own reasoning to craft inputs that would break the system. The researchers then measured success by counting how many unique ways the models could make the software crash. They used a system that grouped similar crashes together, ensuring that a model was credited for finding a new type of failure rather than just repeating the same mistake over and over.
When the researchers tested three different versions of an advanced artificial intelligence system, the results revealed both the potential and the current limits of this technology. The most capable version, known as Opus, managed to trigger a crash in sixty out of the seventy-seven challenges, successfully finding new ways to break the software in the majority of cases. A slightly less powerful version, Sonnet, succeeded in fifty challenges, while the fastest, most economical version, Haiku, found crashes in thirty-five. The researchers assigned a difficulty score to each challenge based on how hard it was for the models to find a bug. The most difficult challenges, where no model could find a crash at all, were those that required the most sophisticated reasoning. Even the best-performing model failed to find a single bug in thirteen of the challenges, suggesting that while these models are powerful tools, they are not yet perfect and still struggle with the most complex or obscure types of software defects.
The study also uncovered interesting differences in how the models approached the task. The most powerful model tended to stop its search earlier than the others, often finding a bug quickly and then moving on, while the other models tended to use their full time allowance, running more tests before giving up. This behavior meant that the most powerful model was sometimes less thorough on the hardest problems, even though it was better at solving the easier ones. The researchers noted that the cost of running these tests varied significantly; the most powerful model was more expensive to run, especially on difficult challenges where it spent more time and generated more data. However, the less expensive models were also less effective at finding the deepest bugs. The findings suggest that while artificial intelligence is becoming a viable partner in the hunt for software bugs, it is not yet a replacement for human expertise. The models are excellent at finding common or accessible flaws, but they still miss the most elusive ones, indicating that the future of software security will likely involve a collaboration between human experts and these increasingly capable digital assistants.
Ultimately, this work provides a clear, measurable way to judge how well artificial intelligence can find software bugs in the real world. By moving away from simple tests that ask models to match a known answer, the researchers created a more realistic simulation of how security experts actually work. The results show that current models can discover a wide variety of software failures, but they are not yet able to find every flaw. As the benchmark expands to include more challenges and more types of software, it will serve as a vital tool for tracking the progress of artificial intelligence in cybersecurity. The goal is not just to build models that can pass a test, but to develop systems that can reliably protect the digital infrastructure we all rely on, finding the cracks before they can be exploited by others.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.