Vibe Coding and Web Application Security: A Twin-Prompt Study
This preliminary study demonstrates that explicitly appending security requirements to natural-language prompts significantly reduces the number of confirmed vulnerabilities in AI-generated web applications, with the security-aware variants containing no critical or high-severity issues compared to their baseline counterparts.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the modern digital world, software is often built by asking a computer to write it for us. We type a description of what we want—a blog, a store, a task manager—and a powerful artificial intelligence, trained on millions of lines of code, generates the instructions needed to make it work. This practice, sometimes called "vibe coding," relies on the idea that the machine understands not just the function of the software, but also the hidden rules that keep it safe. However, there is a lingering doubt: does the machine naturally include the necessary locks and alarms, or does it simply build a working door without worrying about who might try to break in? Security in software is not just about making a program run; it is about ensuring that secrets stay secret, that users cannot pretend to be someone else, and that the system does not accidentally let strangers walk in. As these AI tools become more common, the question shifts from whether they can build code to whether they can build secure code without being explicitly told to do so.
A researcher at the University of Zagreb set out to answer this question by treating the AI like a student who might need a specific instruction to do the right thing. The study involved creating six different web applications, ranging from a simple blog to a shop with an administrative dashboard. For each application, the researcher asked the AI to build it twice. The first request was a standard description of what the software should do. The second request was identical in every way, except for one addition: a clear list of security rules, such as "do not store passwords in plain text" and "verify that every user is who they say they are." This setup allowed for a direct comparison between a version built with a general prompt and a version built with a specific security reminder, while keeping everything else—the AI model, the tools used, and the generation process—exactly the same.
To see what happened, the researcher did not just look at the code; they subjected all twelve applications to a rigorous series of tests. They used automated scanners that read the source code for dangerous patterns, tools that checked the software while it was running to see if it could be tricked, and a final round of manual testing where a human expert tried to break into the systems using creative methods that machines might miss. The results were striking. The versions built with the security reminder contained significantly fewer problems than the baseline versions. In fact, the secure versions had no critical or high-level security flaws, whereas the versions without the reminder suffered from serious issues, including vulnerabilities that could allow an attacker to take over a user's account or access private data. The reminder did not make the software perfect; it left behind some minor configuration issues that were common to both versions, but it successfully eliminated the most dangerous errors.
The study also revealed that finding these flaws is not a job for a single tool. The automated scanners that read the code found many problems that the scanners watching the running software missed, and vice versa. Most notably, the single most severe vulnerability found in the entire experiment—a flaw that allowed an attacker to forge a digital identity and take over an account—was not spotted by any of the automated tools. It was discovered only when a human manually tested the system. This suggests that while asking an AI to be secure helps, it does not replace the need for human oversight and multiple layers of checking. The researcher noted that because this was a small experiment with only one run per application, the results are a strong suggestion rather than a final proof, but the pattern was consistent across every single case. The work serves as a preliminary step, showing that a simple, explicit request for security can dramatically improve the safety of software generated by artificial intelligence, even if it cannot catch every single mistake on its own.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.