Evaluating LLMs for Real-World Web Vulnerability Detection
This paper benchmarks six frontier and open-weight Large Language Models on their ability to detect real-world web vulnerabilities in WordPress plugins, revealing that while all models can identify valid issues, detection rates vary significantly by model and prompt design, with no model achieving consistent reporting or perfect accuracy across iterations.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you have a massive library of digital "instruction manuals" (code) for building websites. Some of these manuals have hidden traps—like a loose floorboard that leads to a basement full of thieves (hackers). These traps are called vulnerabilities.
For a long time, finding these traps required a team of expert security guards manually reading every single page of every manual. But recently, a new kind of "super-reader" has arrived: Large Language Models (LLMs). These are AI systems that can read and understand code almost like a human does.
This paper is essentially a report card testing how well six different "super-readers" can find these hidden traps in WordPress website plugins (the add-ons that give websites extra features).
Here is the breakdown of their experiment and findings, using simple analogies:
1. The Setup: The "Trap Hunt"
The researchers picked four popular WordPress plugins that were known to have specific, real-world traps (like SQL Injection, which is like a hacker tricking a database into spilling its secrets, or Cross-Site Scripting, which is like a hacker injecting a fake note into a public bulletin board).
They asked six different AI models to act as security auditors. They didn't just ask them once; they ran the test three times to see if the AI was consistent, like asking a student to take the same test three days in a row.
They also tried different ways of asking the question (called "prompts"):
- The "Lazy" Prompt: Just said, "Find the bugs."
- The "General" Prompt: Said, "You are a security expert. Find all the major types of bugs."
- The "Specific" Prompt: Said, "You are a security expert. Specifically look for this exact type of bug."
2. The Results: Who Passed the Test?
The results were a mix of impressive success and frustrating inconsistency.
- The Star Student: One model, Claude Opus 4.6, was the best at its job. It found about 63% of the known traps. It was the most reliable "detective."
- The Open-Source Surprise: A model called MiniMax M2.5, which can be run on your own computer (unlike the others which require paying a subscription), performed just as well as the top paid models, finding about 48% of the traps. This is like a home-cooked meal tasting just as good as a five-star restaurant dish.
- The Strugglers: The Qwen models (another open-source option) only found about 35% of the traps. They missed more than half the time.
The Big Takeaway: Even the best AI missed nearly 40% of the traps. No single AI found every trap in every plugin.
3. The "Magic Question" (Prompt Design)
The researchers discovered that how you ask the AI matters more than which AI you use.
- Specific is Better: If you told the AI, "Look specifically for SQL Injection," it found more bugs than if you said, "Look for anything wrong." It's like telling a metal detector user, "Look for gold coins," rather than "Look for anything metal."
- Complexity Doesn't Help: Giving the AI a long, complicated, step-by-step instruction manual didn't make it smarter. Simple, direct instructions worked just as well.
4. The "Coin Flip" Problem (Consistency)
This was the most surprising finding. The AI was unreliable.
If you asked the same AI the same question about the same code three times, it gave a different answer every time.
- Sometimes it would say, "I found a trap!"
- The next time, it would say, "Everything looks safe."
- The third time, it might say, "I found a trap, but it's a different one."
Only one model (Gemini) was consistent 95% of the time, but it was also the one that found the fewest traps. The best models were often inconsistent, acting like a coin flip. This means you can't trust a single scan; you have to run the scan multiple times to be sure.
5. The "Ghost Trap" (The One They All Missed)
There was one specific plugin with a tricky trap called "Stored Cross-Site Scripting." None of the six AIs found it, even after 90 attempts.
Why? The trap was hidden behind a very specific setting that wasn't turned on by default. The AIs assumed, "Since the setting isn't on, this code doesn't matter," and skipped over it. It was a case of the AI making a logical assumption that turned out to be wrong.
6. How Does This Compare to Old Tools?
The researchers also ran the code through a traditional security scanner (Semgrep), which is like using a metal detector that only beeps for specific shapes.
- The Result: The traditional scanner found zero of the known traps.
- The Lesson: The AIs were much better at finding these specific real-world traps than the old-school tools, but they still missed some.
The Final Verdict
The paper concludes that AI is a powerful assistant, but not a replacement for human security experts.
- It works: AI can find real, dangerous bugs in website code.
- It's not perfect: It misses about half the bugs and changes its mind frequently.
- How to use it: You get the best results by using a top-tier model (like Claude), giving it a specific question ("Find SQL bugs"), and running the test multiple times.
- The Warning: You cannot just trust the AI's report. Humans must still double-check the findings because the AI sometimes "hallucinates" (makes up bugs that don't exist) or misses complex traps.
In short: AI is a great new flashlight for finding holes in your website's armor, but you still need a human to hold the flashlight, check the shadows, and make sure the wall is actually secure.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.