💬 NLP

Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks

This paper introduces the SUSEVIBES benchmark to demonstrate that current LLM-based "vibe coding" agents, while often functionally correct, generate highly insecure code in real-world software engineering tasks, with preliminary mitigation strategies proving ineffective.

Original authors: Songwen Zhao, Danqing Wang, Kexun Zhang, Jiaxuan Luo, Zhuo Li, Lei Li

Published 2026-02-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Songwen Zhao, Danqing Wang, Kexun Zhang, Jiaxuan Luo, Zhuo Li, Lei Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The "Vibe Coding" Safety Check: Why Your AI Intern Might Be a Security Nightmare

Imagine you hire a brilliant, super-fast new intern who can write code just by you chatting with them in natural language. You say, "Hey, build me a login system," and they instantly spit out a working program. This is "Vibe Coding." It's trendy, fast, and feels like magic.

But here's the scary question the paper asks: Is this magic safe?

The researchers built a giant stress test called SUSVIBES to find out. They treated these AI coding agents like interns and gave them 200 real-world tasks—like fixing a security hole in a popular website or adding a new feature to a massive software project.

Here is what they found, broken down with some simple analogies.

1. The "Perfectly Flawed" Intern

The most disturbing discovery is that the AI is great at doing the job, but terrible at doing it safely.

The Analogy: Imagine you ask an intern to build a bank vault door.
- Functionality: The door opens when you turn the handle. It locks when you push the button. It looks exactly like a vault. The AI gets an A+ for making the door work.
- Security: The door has a hidden keyhole on the outside that anyone can pick, or the hinges are made of paper. The AI gets an F for security.

The Stats:
When the best AI (Claude 4 Sonnet) was given a task, it successfully built the feature 61% of the time. But out of those successful builds, 89% were insecure.

Translation: If you let your AI intern build your software, there's a roughly 9 out of 10 chance they will leave the back door wide open for hackers, even if the front door works perfectly.

2. The "Kitchen" vs. The "Whole Restaurant"

Previous tests for AI coding were like asking the AI to bake a single cookie in a test kitchen. They only looked at one small file of code.

The Old Way: "Here is a recipe for a cookie. Make it." (Easy, safe, small scope).
The New Way (Vibe Coding): "Here is the entire restaurant, the supply chain, the kitchen, and the dining room. Go fix the plumbing and add a new menu item without breaking the fire alarm."

SUSVIBES is the "Whole Restaurant" test. It forces the AI to navigate complex, messy, real-world codebases. The AI gets lost in the weeds and misses the security traps that are hidden in the corners of the building.

3. The "Security Reminder" Didn't Work

The researchers tried to help the AI by giving it a gentle nudge, like a teacher saying, "Hey, remember to be careful with security!" They even tried giving the AI a cheat sheet of specific security rules.

The Result: It didn't help much. In fact, it made things worse.
The Analogy: It's like telling a nervous driver, "Drive safely, but also get to the destination fast." The driver gets so focused on "driving safely" (checking mirrors, slowing down) that they forget to actually drive the car, or they crash because they are overthinking.
The Outcome: When the AI tried too hard to be "secure," it stopped working correctly. When it focused on "working," it forgot to be "secure." It couldn't do both at the same time.

4. The "Blind Spots" of Different AIs

The paper tested different AI models (like Claude, Kimi, and Gemini). They found that different AIs are bad at different things.

The Analogy: Imagine three different security guards.
- Guard A is great at spotting people climbing the fence but misses people sneaking through the front door.
- Guard B is great at the front door but misses the back window.
- Guard C misses everything.
The Problem: There is no "Super Guard" yet. Even the smartest AI models have blind spots. If you rely on just one, you are guaranteed to have a vulnerability somewhere.

5. Real-World Consequences

The paper didn't just look at abstract code; they looked at real scenarios. Here are two examples of what the AI got wrong:

The "Time-Telling" Lock: In a password system, the AI wrote code that answered "No" instantly if the password was wrong, but took a tiny fraction of a second longer if the username didn't exist.
- Why this matters: A hacker can time how long the computer takes to answer. By measuring the delay, they can guess which usernames are real. It's like a lock that clicks louder when the key is wrong, letting a burglar know which door to pick.
The "Poisoned" Link: In a content management system, the AI let users post links without checking if they were dangerous.
- Why this matters: A hacker could post a link that looks normal but actually steals your login cookies when you click it. The AI built the feature perfectly, but forgot the "sanity check" that stops bad links.

The Bottom Line

"Vibe Coding" is currently a high-risk gamble.

The paper concludes that while AI is amazing at making software work, it is currently very bad at making software safe. If you are a company building something that handles money, health data, or private information, you cannot just let the AI "vibe" the code and hope for the best.

The takeaway: Treat AI-generated code like a car built by a robot that has never seen a traffic light. It might drive down the road, but you absolutely need a human mechanic to check the brakes before you let anyone ride in it.

1. Problem Statement

Vibe coding is an emerging programming paradigm where human engineers use natural language to instruct Large Language Model (LLM) agents to complete complex coding tasks with minimal supervision. While adoption is rising (with 75% of surveyed engineers using it), there is a critical lack of understanding regarding the security safety of the code generated by these agents.

Existing security benchmarks for AI-generated code suffer from three major limitations:

Scope: They focus on single files or functions, whereas vibe coding involves large, multi-file repositories.
Interaction: They test single-turn generation, whereas vibe coding agents operate in multi-turn loops with environment feedback.
Context: They rely solely on text inputs, ignoring the agent's ability to interact with execution environments and compilers.

The core question is: Are agent-generated solutions functionally correct but riddled with exploitable vulnerabilities when deployed in real-world, security-sensitive contexts?

2. Methodology: The SUSVIBES Benchmark

To address these gaps, the authors propose SUSVIBES (Security of Vibe Coding), a repository-level benchmark designed to evaluate the security risks of AI agents.

Benchmark Construction Pipeline

The authors developed a fully automatic pipeline to curate 200 realistic feature-request tasks from 108 real-world open-source Python projects (spanning 10 domains like Web, DevOps, and Identity). The pipeline involves:

Mining Vulnerability Fixes: Identifying historical commits ( $C_0$ ) that fixed known security vulnerabilities in open-source repositories.
Reverting and Masking: Reverting to the commit prior to the fix ( $C_{-1}$ ) and using an LLM agent to "mask out" the vulnerable feature implementation. This creates a codebase ( $C^M_{-1}$ ) where the feature is missing.
Task Generation: Synthesizing a natural language feature request (GitHub issue style) based on the masked code, ensuring no hints about the security fix are leaked.
Verification: Using a third agent to verify that the generated task description strictly requires the canonical secure implementation (found in $C_0$ ) and does not allow for "excessive" or insecure implementations.
Environment Setup: Automatically constructing Docker-based execution environments to run both functional and security unit tests.

Evaluation Metrics

The benchmark evaluates agents on two dimensions:

FUNCPASS: The solution passes all human-written functional unit tests.
SECPASS: The solution passes both functional tests and human-written security unit tests.
SECPASS $\perp$ FUNCPASS: A metric measuring security performance specifically on the subset of tasks where the agent was already functionally correct.

3. Key Contributions

SUSVIBES Benchmark: A repository-level benchmark with 200 tasks covering 77 distinct CWEs (Common Weakness Enumerations), requiring an average of 172 lines of code edits across multiple files.
Automatic Curation Pipeline: A scalable method to generate security-oriented coding tasks from real-world vulnerability fixes, complete with executable environments.
Comprehensive Evaluation: Experiments across 3 agent frameworks (SWE-Agent, OpenHands, Claude Code) and 3 frontier LLMs (Claude 4 Sonnet, Kimi K2, Gemini 2.5 Pro).
Mitigation Analysis: An investigation into whether prompt-based strategies (generic warnings, self-selection of CWEs, or oracle hints) can improve security.

4. Key Results

Performance Gap: Functionality vs. Security

The results reveal a stark disparity between functional correctness and security:

High Functionality, Low Security: Even the best-performing combination (SWE-Agent + Claude 4 Sonnet) achieved a 61.0% FUNCPASS rate but only a 10.5% SECPASS rate.
Vulnerability Rate: Among the solutions that were functionally correct, 82.8% contained security vulnerabilities.
General Trend: Across all 9 combinations, the average SECPASS was approximately 10%, indicating that if a user accepts a functionally correct agent solution, there is an ~80% chance it is insecure.

Model and Framework Sensitivity

Model Differences: Different LLMs exhibit different "blind spots." For example, Gemini 2.5 Pro was generally more secure than Claude 4 Sonnet on shared correctly solved tasks, but Claude 4 Sonnet was significantly more functionally capable.
Framework Differences: OpenHands demonstrated higher security than SWE-Agent when using the same LLM, suggesting agent scaffolding influences security outcomes.
CWE Specialization: Agents are cautious about different CWE types. There is low overlap (58%) in the specific CWEs that different models can successfully avoid, indicating no single model covers all security risks.

Failure of Preliminary Mitigations

The authors tested three prompting strategies to improve security:

Generic Security Guidance: Adding a standard reminder.
Self-Selection: Asking the agent to identify relevant CWEs before coding.
Oracle Hints: Explicitly telling the agent which CWE to avoid.

Findings:

These strategies failed to improve overall security.
More critically, they caused a significant drop in functional correctness (approx. 5–8% decrease).
Trade-off: Agents prioritized security checks so heavily that they began omitting necessary functional edge cases. The "Oracle" strategy improved security slightly but at the cost of breaking functionality.

Qualitative Case Studies

The paper analyzes specific vulnerabilities introduced by agents, including:

Timing Side-Channels: Agents implementing password verification that return immediately for invalid inputs, allowing attackers to enumerate valid usernames.
CRLF Injection: Agents failing to sanitize redirect URLs, allowing header injection attacks.
Stored XSS: Agents failing to validate URL schemes in rich text editors, allowing javascript: payloads.
Session Expiration: Agents failing to enforce max_age on session cookies, allowing replay attacks.

5. Significance and Conclusion

The paper concludes that current vibe coding practices are unsafe for security-sensitive applications. The gap between functional correctness and security is persistent and widening as models improve in reasoning but not necessarily in security alignment.

Critical Warning: Organizations deploying AI coding agents in domains like authentication or data handling must implement mandatory human security reviews; relying on "vibe coding" alone is dangerous.
Research Direction: Simple prompt engineering is insufficient. Future work must focus on security as a first-class objective in agent training (e.g., RL with security rewards) and developing agents that can balance functionality and security without sacrificing one for the other.
Community Impact: SUSVIBES provides a necessary platform for the community to benchmark and improve the security properties of software engineering agents.