ARQ: Agentic CodeQL Query Refinement for C/C++ Vulnerability Detection
ARQ is an agentic framework that automatically refines C/C++ CodeQL queries using execution-grounded evidence from synthesized programs and LLMs to significantly reduce false positives and negatives without requiring labeled datasets, achieving up to 119.8% more true positive detections and resolving long-standing issues in official repositories.
Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Software that runs our computers, phones, and cars is often built with languages like C and C++. These tools give programmers precise control over how a machine uses its memory, but that same power makes it easy to slip up. A small mistake in how a program handles memory can leave a door open for attackers to crash systems or steal data. To catch these errors before they cause harm, security experts use automated scanners that read code and look for dangerous patterns. These scanners rely on a set of rules, written by humans, that tell them exactly what a vulnerable piece of code looks like. If the rules are too strict, the scanner misses real dangers; if they are too loose, it screams about problems that aren't there, wasting time and trust.
For years, improving these rules has been a slow, manual job. Experts have to read code, guess where the scanner might fail, and rewrite the rules by hand. This process is difficult because the rules must be perfect, and the code they scan is vast and complex. A new approach called ARQ changes how this work is done. Instead of relying solely on human intuition, ARQ uses artificial intelligence to test and fix these rules automatically. It does not just guess; it runs the code it creates to see if the rules actually work. By building a loop where the system generates test programs, runs them, and then uses the results to improve the rules, ARQ finds and fixes errors that humans have missed for years.
The researchers behind ARQ started with a simple but powerful idea: a rule is only as good as its ability to distinguish between safe code and dangerous code. They gave an artificial intelligence a specific rule from a popular security scanner and asked it to create two very similar programs. One program was designed to be safe, and the other was designed to be dangerous. The AI then watched how the scanner reacted to both. If the scanner flagged the safe program as dangerous, the rule was too loose. If it missed the dangerous program, the rule was too strict. This mismatch between what the scanner said and what actually happened when the code ran became the evidence the AI needed to fix the rule.
To make sure the AI was not just guessing, the system used a special tool called a sanitizer. This tool acts like a high-precision sensor that watches a program while it runs. If the program tries to do something unsafe, like using memory that has already been freed, the sanitizer stops the program and reports the error immediately. This gave the researchers a ground truth. They could compare the scanner's verdict against the sanitizer's report. If the scanner said a program was safe, but the sanitizer found a crash, the researchers knew the scanner had made a mistake. The AI then used this confirmed mistake to rewrite the rule, making it smarter.
This process happened in a continuous loop. The AI would generate a new pair of test programs, run them, check the results, and refine the rule again. Crucially, every time the rule changed, the system re-tested it against all the previous examples to ensure the fix did not break anything that was already working. This prevented the AI from overcorrecting and creating new problems while solving old ones. The team tested this method on twelve different security rules used in real-world software. They ran these rules against two large collections of code designed to test security tools. The results showed that the AI-refined rules found significantly more real vulnerabilities than the original human-written rules. In some cases, the number of detected threats more than doubled, while the rate of false alarms stayed extremely low, remaining above ninety-eight percent accuracy.
The impact of this work went beyond test data. The researchers applied their refined rules to real, widely used software libraries that power much of the internet, such as tools for compressing data and displaying images. The improved rules uncovered two bugs that had been hiding in these libraries for years, waiting to be found. Furthermore, the system successfully fixed three specific issues that had been reported on a public software repository but had remained unresolved for as long as twenty-seven months. These issues had stumped human experts for over two years, yet the automated system resolved them by systematically testing and adjusting the detection rules.
This approach suggests a new way to maintain the safety of our digital infrastructure. Instead of waiting for humans to find a flaw and manually write a fix, systems can now continuously test their own rules against real execution. The method does not require massive databases of labeled errors or years of historical data to work. It simply needs the ability to run code and check the results. By grounding the artificial intelligence in the reality of how programs actually behave, rather than just how they look on paper, the researchers have created a tool that can keep security scanners sharp and effective as software evolves. The work demonstrates that when artificial intelligence is paired with concrete evidence from running code, it can solve complex engineering problems that have long resisted human effort.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.