🤖 AI

AVIATOR: Towards AI-Agentic Vulnerability Injection Workflow for High-Fidelity, Large-Scale Code Security Dataset

This paper introduces AVIATOR, the first AI-agentic framework that utilizes a coordinated workflow of specialized agents, tool-based analysis, and iterative self-correction to generate high-fidelity, large-scale vulnerability datasets, which significantly outperform existing injection techniques and substantially improve the performance of deep learning-based vulnerability detection models.

Original authors: Amine Lbath, Massih-Reza Amini, Aurelien Delaitre, Vadim Okun

Published 2026-03-19

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Amine Lbath, Massih-Reza Amini, Aurelien Delaitre, Vadim Okun

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a security guard (an AI) how to spot a pickpocket in a crowded city. To do this, you need to show the guard thousands of examples of pickpockets in action.

The problem? Real pickpockets are rare, and the ones we do know about are often messy, hard to label, or don't look like the "perfect" criminals we want to teach the guard. If you train the guard on bad examples, they might start arresting innocent people or missing the real thieves.

Enter AVIATOR.

Think of AVIATOR not as a single robot, but as a highly organized construction crew led by a master architect. Its job is to build "fake" pickpockets (software vulnerabilities) inside safe buildings (secure code) so the security guard can learn to spot them.

Here is how AVIATOR works, broken down into simple concepts:

1. The Old Way: The "Sticker" Approach

Previous methods of creating these fake vulnerabilities were like taking a safe building and just sticking a "Danger" sign on a random wall, or using a stamp to paste a pre-made "broken window" sticker onto a door.

The Problem: These stickers often don't fit. They look fake. Sometimes they break the building's structure. The security guard learns to spot the stickers, not the actual flaws.
The Result: The guard gets confused, misses real threats, or gets too many false alarms.

2. The AVIATOR Way: The "Master Architect" Crew

AVIATOR changes the game by using AI Agents. Instead of one robot trying to do everything at once, AVIATOR breaks the job down into a team of specialists, working like a human security expert would.

Here is the crew's workflow:

The Detective (Code Understanding): First, an agent reads the code like a detective reading a blueprint. It understands how the building works, where the doors are, and how people move through it.
The Librarian (Information Loader): This agent pulls up a file on exactly what kind of "flaw" needs to be created (e.g., "a loose floorboard" or "an unlocked back door").
The Reverse Engineer (Vulnerability Analyst): This agent looks at the safe code and asks, "What is preventing a break-in here?" It identifies the security guards (checks and balances) that are currently working.
The Saboteur (The Injector): This is the main worker. Using the info from the others, it carefully removes a specific guard or loosens a bolt. Crucially, it doesn't just smash the wall; it makes the break-in look like something a real human hacker would do. It uses a "cheat sheet" (Retrieval-Augmented Generation) of real-world break-ins to make sure the edit looks authentic.
The Quality Control Team (Validation): This is where AVIATOR shines. Before the "broken" building is sent to the security guard, a whole team checks it:
- The Diff Checker: "Did you actually change anything, or did you just move a chair?"
- The Static Analyzer: A robot tool that scans the code to ensure the building didn't collapse (syntax errors).
- The Fixer: If the Saboteur made a mistake, the Fixer tries to repair it or try again.

3. Why This Matters (The Results)

Because AVIATOR uses this "team of experts" approach instead of a single robot guessing:

It's More Realistic: The fake vulnerabilities it creates look and act exactly like real ones. It's like training the guard with actors who are actually pickpocketing, not just people holding signs.
It's Smarter: It can create a huge variety of flaws (133 different types!), whereas older methods could only make a few simple ones.
It's Cheaper and Faster: It costs 4.3 times less to run than the previous best method.
The Guard Gets Better: When the security guard (the AI vulnerability detector) was trained on AVIATOR's fake data, it got significantly better at spotting real threats. It found more bad guys (higher recall) without arresting innocent people (maintained precision).

The Bottom Line

Think of AVIATOR as a simulator for cyber-hacking. Instead of waiting for real hackers to break into systems to learn from them, AVIATOR builds a safe, controlled, and incredibly realistic training ground. It uses a team of AI specialists to craft perfect "what-if" scenarios, ensuring that the software defenders we train are ready for anything the real world throws at them.

In short: Old methods stamped fake flaws on code. AVIATOR builds realistic, complex, and diverse flaws so our AI defenders can learn the difference between a safe building and a broken one.

1. Problem Statement

The paper addresses the critical bottleneck in training and benchmarking Deep Learning-based Vulnerability Detection (DLVD) models: the lack of large-scale, high-fidelity, and accurately labeled datasets.

Limitations of Existing Data:
- Synthetic Datasets: Often lack structural complexity and realism, causing models to learn superficial cues.
- Real-World Datasets: Rely on heuristic labeling (e.g., from CVEs or fix commits), which introduces significant noise (label accuracy as low as 25% in some cases).
- Manual Annotation: High fidelity but not scalable.
Limitations of Existing Injection Techniques:
- Pattern-Mining Approaches (e.g., VGX, VulGen): Limited coverage (e.g., VGX supports only 23 CWE types), rely on rigid templates, and often perform single-statement edits that lack semantic realism.
- LLM-Based Approaches (e.g., VulScribeR): Optimize for downstream detection metrics rather than injection correctness, leading to lower injection fidelity (estimated ~80%) and potential label noise.

Goal: Create a framework that automatically injects realistic, diverse, and category-specific vulnerabilities into codebases with high fidelity, broad coverage, and low cost, to serve as a robust training signal for DLVD models.

2. Methodology: The AVIATOR Framework

AVIATOR (AI Agentic Vulnerability Injection And Transformation with Optimized Reasoning) is the first framework to treat vulnerability injection as a coordinated AI-agentic workflow rather than a monolithic generation task. It mimics the reasoning process of human cybersecurity experts.

A. Core Architecture

The framework decomposes the injection process into 13 specialized agents (9 AI agents, 4 tool agents) organized into two main modules:

Vulnerability Injection Module:
- Code Understanding Agent: Analyzes control/data flow to identify functional intent and potential sources/sinks.
- Information Loader: Retrieves detailed definitions of the target vulnerability (CWE).
- Vuln.-Specific Code Understanding: Identifies safeguards (e.g., sanitization checks) that prevent the vulnerability, effectively performing "reverse analysis."
- Vulnerability Injector (Core): Uses Retrieval-Augmented Generation (RAG) to fetch similar benign/vulnerable pairs and Low-Rank Adaptation (LoRA) fine-tuning to generate the injection.
- Polisher: Normalizes formatting and removes trivial edits (e.g., whitespace changes).
Validation Module:
- Difference Checkers: Detect trivial or missing edits.
- Static Analysis (Cppcheck): Runs lightweight static analysis to verify syntax and basic safety.
- Diff Analyzer & Verifier: LLMs that review the diff and static analysis reports to confirm the vulnerability is correctly instantiated.
- Fixer & Controlled Restart: If validation fails, the system attempts a constrained repair or restarts the injection process with a different trajectory (bounded by a max attempt limit to prevent infinite loops).

B. Learning Strategies

Supervised Fine-Tuning (SFT): The core injector agent is fine-tuned using LoRA on a dataset of aligned benign/vulnerable function pairs (from PrimeVul). The objective is next-token prediction to learn exact injection patterns.
Reinforcement Learning (GRPO): The paper also explored Group Relative Policy Optimization using CodeBLEU as a reward, but found SFT to be more effective and efficient.

3. Key Contributions

First AI-Agentic Workflow for Injection: Unlike previous monolithic LLM approaches, AVIATOR uses a multi-agent pipeline with self-correction loops, explicitly mirroring expert reasoning (understand $\to$ identify $\to$ modify $\to$ review).
High-Fidelity & Broad Coverage: The framework supports 133 CWE types (far exceeding VGX's 23), enabling the generation of diverse, context-aware vulnerabilities without relying on handcrafted patterns.
Efficient Scaling: Achieves high-quality injection with a syntax rejection rate of <2% and at 4.3x lower cost than the state-of-the-art LLM-based method (VulScribeR).
Superior Downstream Utility: Demonstrates that high-fidelity injection directly translates to better performance in downstream vulnerability detection models.

4. Experimental Results

The authors evaluated AVIATOR on three benchmarks: SARD100 (synthetic), FormAI (complex synthetic), and PrimeVul (real-world).

A. Injection Fidelity (RQ1)

Success Rates: AVIATOR achieved 95% on SARD100, 91% on FormAI, and 94% (verifiable weaknesses) on PrimeVul.
Comparison: Outperformed VGX (90% but limited to 23 CWEs) and VulScribeR (82% estimated). AVIATOR's average success rate across benchmarks was 93%.

B. Impact of Fine-Tuning (RQ2)

SFT vs. GRPO: Supervised Fine-Tuning (SFT) significantly improved performance (e.g., +6% on FormAI) and reduced variance compared to the base model. GRPO failed to outperform the base model despite higher computational costs.
Generalization: Fine-tuning was done on PrimeVul but yielded gains on SARD100 and FormAI, proving strong generalization.

C. Agentic Workflow Contribution (RQ3)

Decomposition is Key: Moving from a monolithic 1-agent setup (31% success) to the full 13-agent workflow (91% success) demonstrated that task decomposition and iterative validation are the primary drivers of success.
Model Specialization: Specialized coding models (Qwen2.5-Coder-32B) significantly outperformed general-purpose models (Llama-4-Maverick).

D. Downstream DLVD Performance (RQ5)

F1 Score Gains: When used to augment training data for DLVD models (Devign, Reveal, LineVul, CodeT5), AVIATOR improved the average F1 score by:
- +21.8% over no augmentation.
- +24.6% over VGX.
- +2.9% over the prior SOTA VulScribeR.
Precision-Recall Balance: Unlike VGX (high recall, low precision) or VulScribeR, AVIATOR improved Recall (+7.4%) without sacrificing Precision, indicating cleaner, more learnable training signals.
Distributional Fidelity: AVIATOR induced the lowest Kullback-Leibler (KL) divergence from the original dataset distribution, meaning it enriched the data without skewing it toward narrow templates.

5. Significance and Conclusion

AVIATOR represents a paradigm shift in software security data generation. By moving from pattern-based or single-prompt LLM generation to a structured, self-correcting agentic workflow, it solves the trade-off between scale and fidelity.

Practical Impact: It enables the construction of large-scale, reliable benchmark datasets essential for training next-generation AI security tools.
Cost Efficiency: The ability to generate high-quality data at a fraction of the cost of proprietary LLM pipelines makes large-scale security research more accessible.
Future Work: The authors plan to extend the framework from function-level edits to multi-function and repository-level injection to handle broader project contexts.

In summary, AVIATOR proves that AI-agentic workflows can effectively replicate human expert reasoning to generate high-quality security data, significantly boosting the performance of automated vulnerability detection systems.