🤖 AI

Secure Code Generation at Scale with Reflexion

This paper evaluates the effectiveness of reflexion prompting in enhancing the security of code generated by five instruction-tuned large language models, finding that while zero-shot baselines produce insecure code in 25-33% of cases, a three-round reflexion process significantly improves security metrics with the most substantial gains occurring in the first round.

Original authors: Arup Datta, Ahmed Aljohani, Hyunsook Do

Published 2026-06-08

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Arup Datta, Ahmed Aljohani, Hyunsook Do

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have hired a team of very talented, fast-talking junior programmers (the AI models) to write code for your software. You give them a simple instruction, like "Build a secure login page," and they type away furiously.

This paper asks a simple but critical question: Just because the code works, is it actually safe? And more importantly, if we tell them, "Hey, you made a mistake, try again," can they fix it?

Here is the breakdown of their findings using everyday analogies:

1. The "First Draft" Problem

The researchers found that when these AI models write code for the first time (called "zero-shot"), it's like a student taking a test without studying.

The Result: Roughly 25% to 33% of the code they write immediately has security holes. It's like handing in a house blueprint where the front door has no lock, or the windows are made of paper.
The Analogy: Think of it like a chef who is great at making a delicious cake but keeps forgetting to put a lid on the jar of poison they keep next to the sugar. The cake tastes fine (the code works), but it's dangerous to eat (the code is insecure).

2. The "Reflexion" Loop (The Second Chance)

The researchers didn't just let the chefs serve the cake. They introduced a process called Reflexion.

How it works: After the AI writes the code, a security scanner (a robot inspector) checks it and says, "You left the back door open," or "You used a weak lock." The AI then reads this feedback, thinks about it, and rewrites the code. They did this up to three times.
The Analogy: It's like a teacher grading a paper, circling the mistakes, and asking the student to rewrite it.
The Result: This "second chance" worked wonders. The security of the code jumped from about 71% safe to 79% safe.
The Catch: Most of the improvement happened on the first rewrite. By the second and third tries, the AI was mostly just polishing tiny details. It's like fixing the front door lock immediately, but then spending the next two hours trying to find a speck of dust on the doorknob.

3. The "Language" Difficulty

Not all programming languages are equally hard for the AI to handle securely.

The Easy Mode: Python was the safest. It's like asking the AI to build a treehouse; the AI knows the standard rules well.
The Hard Mode: C and C# were the most dangerous. These languages are like building a nuclear power plant; one tiny mistake in the wiring causes a meltdown. The AI struggled the most here.
The Middle Ground: Languages like Java, JavaScript, and C++ fell somewhere in between.

4. The "Tricky Bugs" vs. The "Obvious Bugs"

The researchers looked at what kind of mistakes the AI made.

The Easy Fixes: The AI was actually quite good at avoiding "templated" mistakes like leaving a password hard-coded in the text or leaving a backdoor for hackers to inject commands. It's like the AI knows, "Oh, I shouldn't write the password on the sticky note."
The Hard Fixes: The AI struggled the most with cryptography (encryption) and configuration errors. These are like trying to bake a cake with a specific, complex chemical reaction. If the temperature is off by one degree, the whole thing fails. The AI often got the math or the settings wrong.

5. The "Regression" Risk

One interesting finding was that sometimes, when the AI tried to fix a mistake, it accidentally broke something else that was working fine.

The Analogy: Imagine you are fixing a leaky faucet. You tighten the valve, but in doing so, you accidentally snap the pipe behind it.
The Finding: The AI did this occasionally, but not often enough to cancel out the benefits. The "Net Gain" (fixes minus new mistakes) was still positive.

The Bottom Line

The paper concludes that while AI is getting better at writing code, it still makes a lot of security mistakes on the first try. However, if you give it a chance to review its own work and fix it once or twice, you get significantly safer code.

Their advice: Don't expect perfection on the first try. Instead, build a system where the AI writes the code, gets a "security report card," and fixes the top issues immediately. Doing this one or two times captures almost all the benefits; doing it ten times is just a waste of time.

Technical Summary: Secure Code Generation at Scale with Reflexion

Problem Statement
While Large Language Models (LLMs) are widely adopted for drafting and refactoring code, the security of generated code remains a critical concern. Previous research has established that LLMs often produce code that is functionally correct but insecure, inheriting vulnerabilities from their training data. Existing security benchmarks have faced limitations, including narrow language coverage, reliance on small-scale datasets, and "cue contamination" where prompts or identifiers inadvertently leak vulnerability information, inflating insecurity metrics. Furthermore, most evaluations rely on single-turn (zero-shot) prompting, failing to assess whether iterative reflection and revision strategies can effectively improve security outcomes. There is a lack of large-scale, systematic evaluations that utilize high-coverage, validity-enhanced benchmarks to measure the efficacy of iterative self-correction in secure code generation.

Methodology
The authors conducted a large-scale evaluation using the Instruct Prime benchmark, a corrected version of CyberSecEval that removes compliance-required prompts and reduces cue contamination. The study involved the following components:

Models: Five instruction-tuned code LLMs were evaluated: Qwen2.5-Coder-32B, Codestral-22B, CodeLlama-34B, DeepSeek-Coder-33B, and OpenCoder-8B.
Prompting Strategies:
- Zero-Shot Baseline ( $t_0$ ): Models generated code based solely on the task description without security guidance.
- Reflexion Prompting ( $t_1, t_2, t_3$ ): A three-round iterative loop where the model receives its previous code, the results of a static security analysis, and specific hints regarding flagged Common Weakness Enumeration (CWE) categories. The model is instructed to revise the code to remove vulnerabilities while preserving functionality.
Security Assessment: The Insecure Code Detector (ICD), a static, CWE-aligned ruleset, was used to scan all generated code snippets.
Metrics: The study introduced three metrics to evaluate iterative performance:
- Repair: The percentage of previously insecure samples that became secure in the next round.
- Regression: The percentage of previously secure samples that became insecure in the next round.
- NetGain: The difference between Repair and Regression rates.
Post-Processing: To ensure fair evaluation, outputs were filtered to include only "Complete Code." Non-code or mixed outputs were marked as insecure to prevent models from bypassing security checks by generating text instead of code.

Key Contributions

Large-Scale Evaluation: The study provides a comprehensive assessment of secure code generation across five models, eight programming languages, and 50 CWE categories using the validated Instruct Prime benchmark.
Iterative Reflexion Analysis: It is the first systematic evaluation of self-reflection for secure code generation at scale, testing whether models can iteratively improve security through three rounds of revision.
Comprehensive Metrics: The paper moves beyond simple accuracy by reporting Repair, Regression, and NetGain metrics, offering a nuanced view of how iterative strategies improve security without introducing new vulnerabilities.
Replication Package: A full replication package including prompts, generation logs, and scoring artifacts is made available to enable auditing and reuse.

Results

Baseline Insecurity: At the zero-shot baseline ( $t_0$ $t_{0}$ ), insecurity is prevalent, with approximately 25–33% of generated programs being insecure.
- Language Variance: Python yielded the highest secure rates (78–88%), while C and C# showed the lowest (55–67%).
- CWE Variance: Templated vulnerabilities like XSS (CWE-79), code injection (CWE-94), and hard-coded secrets (CWE-798) were handled reliably (85–90% secure rates). Conversely, weak cryptography (CWE-338) and configuration-dependent bugs were the most difficult to avoid, with secure rates dropping to 10–35% for specific categories.
Impact of Reflexion: Applying the reflexion strategy improved the average secure-generation accuracy from 70.74% at $t_0$ to 79.43% at $t_3$ .
- Diminishing Returns: The largest gains occurred in the first round ( $t_1$ , +6.00 percentage points). Subsequent rounds ( $t_2, t_3$ ) yielded smaller improvements (+1.66 and +1.03 pp, respectively).
- Model Performance: QwenCoder and Codestral showed the most significant improvements (approx. +11 percentage points). OpenCoder showed more modest gains.
- Regressions: While repairs were common, later rounds occasionally introduced regressions (1–2%), where secure code was revised into insecure code.
Optimal Strategy: The data suggests that one to two rounds of reflexion capture the majority of security benefits, with additional iterations providing diminishing returns.

Significance and Claims
The paper claims that insecurity remains a widespread issue in LLM-generated code, with roughly one-quarter to one-third of initial outputs containing vulnerabilities. The study demonstrates that reflexion prompting is an effective, training-free mechanism to significantly reduce this insecurity rate. However, the authors emphasize that reflexion does not eliminate risk entirely; it primarily corrects the most direct and obvious errors in the first iteration.

The authors conclude that while iterative reflection improves security, the benefits are subject to diminishing returns and the risk of regression. They propose a practical policy: run one round of reflexion by default, continue only for cases that remain insecure, and stop once NetGain shows little improvement or when regressions approach the level of repairs. The study highlights that while LLMs are improving, specific vulnerability families (cryptographic weaknesses, data authenticity) and lower-level languages (C, C#) remain significant challenges that require further investment in secure defaults and high-quality training data.