💻 computer science

Does Teaming-Up LLMs Improve Secure Code Generation? A Comprehensive Evaluation with Multi-LLMSecCodeEval

This paper introduces the MULTI-LLMSECCODEEVAL framework to demonstrate that carefully orchestrated multi-LLM ensembles integrated with static analysis and structured collaboration significantly outperform single large language models in generating secure code, proving that system design is more critical than model scale for security.

Original authors: Bushra Sabir, Shigang Liu, Seung Ick Jang, Sharif Abuadbba, Yansong Gao, Kristen Moore, SangCheol Kim, Hyoungshick Kim, Surya Nepal

Published 2026-03-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Bushra Sabir, Shigang Liu, Seung Ick Jang, Sharif Abuadbba, Yansong Gao, Kristen Moore, SangCheol Kim, Hyoungshick Kim, Surya Nepal

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are building a house. You want the doors to lock automatically, the windows to be shatter-proof, and the electrical wiring to be safe. In the world of software, this is called secure code generation.

For a while, we've been using "AI Architects" (Large Language Models or LLMs) to draw up these blueprints and build the walls. These AI models are incredibly smart; they can write code faster than any human. But here's the problem: They are also incredibly careless.

Just like a brilliant architect who forgets to install a smoke detector or leaves a backdoor unlocked, these AIs often write code that looks perfect but has hidden security holes (vulnerabilities) that hackers can exploit.

This paper asks a simple question: If we get a team of AI architects to work together, will they build a safer house than a single AI working alone?

The Experiment: The "Multi-LLM" Construction Crew

The researchers set up a massive construction site called MULTI-LLMSECCODEEVAL. Instead of hiring just one AI, they tried different team configurations to see which one built the safest code.

Here are the four main teams they tested, explained with analogies:

1. The Lone Wolf (Single LLM)

The Analogy: One genius architect working alone in a room.
The Result: Fast, but prone to mistakes. Sometimes they forget to lock the door; other times they use the wrong type of wood. The study found that even the "smartest" single AI left dangerous holes in the code about 30% to 60% of the time.

2. The Committee of Peers (Ensemble)

The Analogy: Hiring five different architects, asking them all to draw the same blueprint, and then having a Safety Inspector (a static analysis tool like CodeQL) check every single drawing.
The Result: Much better! By having multiple AI models generate ideas and then using a strict, rule-based inspector to reject anything unsafe, the team reduced errors significantly.
The Magic: It's like having a committee vote on a plan, but the final decision is made by a robot that knows the building code by heart. This approach improved security by up to 47% compared to the lone wolf.

3. The Debate Club (Collaborative)

The Analogy: A group of architects sitting around a table arguing back and forth. "I think this door is safe." "No, I think that window is weak." They try to convince each other to fix the design without a strict inspector.
The Result: This was okay, but not great. The AIs sometimes convinced each other that a bad idea was actually good. Without a strict rule-checker, they just argued in circles. This only gave a small improvement (about 9–22%).

4. The Super-Team (Hybrid Pipeline)

The Analogy: This is the ultimate dream team.
1. Generation: Five architects draw blueprints.
2. Inspection: The Safety Inspector (CodeQL) checks them and rejects the unsafe ones.
3. Debate: The remaining safe-ish blueprints are sent to a "Debate Club" of AIs to argue over the best way to fix the remaining tiny flaws.
4. Final Polish: The best version is chosen and double-checked.
The Result: The Winner. This team built the safest code, achieving nearly 99% security on some tests.

Key Lessons from the Construction Site

The paper discovered some surprising truths that go against what we usually hear about AI:

Bigger isn't always better: You might think hiring the most expensive, massive AI (like a "Giant Brain") would be best. But the study found that a team of smaller, specialized AIs working together often did a better job than one giant AI working alone. It's like a team of specialized electricians, plumbers, and carpenters beating a single "super-general contractor."
The Inspector is crucial: The biggest jump in safety didn't come from the AIs talking to each other; it came from using a strict, rule-based tool (Static Analysis) to check their work. AIs are great at creativity, but they are terrible at following strict safety rules. You need a robot inspector to catch the mistakes.
Complexity is the enemy: The AIs were great at building simple sheds (easy tasks). But when asked to build a skyscraper with complex wiring (hard tasks), they started making mistakes again. The more complex the job, the more you need a team and a strict inspector.

The Bottom Line

The paper concludes that security doesn't come from making the AI smarter; it comes from how we organize the team.

If you want safe software, don't just ask one AI to "write secure code." Instead:

Ask multiple AIs to try.
Run their work through a strict, automated safety checker.
Have the AIs collaborate to fix the remaining issues.

By orchestrating a team of AIs with a strict inspector, we can turn the "careless genius" into a "reliable security expert."

1. Problem Statement

Large Language Models (LLMs) are increasingly used for automated code generation, vulnerability detection, and patching. However, individual LLMs exhibit significant limitations in security-sensitive tasks:

Unreliability: Even state-of-the-art models (e.g., GPT-4, CodeLlama) frequently generate code with vulnerabilities (e.g., path injection, resource leaks) or fail to detect subtle semantic defects.
Inconsistency: Minor prompt variations or variable renaming can lead to incorrect vulnerability assessments.
Lack of End-to-End Coordination: Prior research often addresses isolated stages (generation or detection or patching) using single models. There is a gap in systematically evaluating how combining multiple LLMs with static analysis tools across the full vulnerability management lifecycle (Generation $\to$ Detection $\to$ Patching) impacts security.

2. Methodology: Multi-LLMSecCodeEval Framework

The authors introduce MULTI-LLMSECCODEEVAL, a unified, automated framework designed to evaluate and enhance secure code generation through multi-model orchestration.

A. Framework Architecture

The framework structures the vulnerability management lifecycle into three automated stages:

Code Generation: LLMs generate code from natural language (NL) prompts.
Vulnerability Detection: Code is analyzed using Static Analysis Tools (SAT) like CodeQL and/or LLM reasoning to identify flaws.
Automated Patching: Detected vulnerabilities are fixed by LLMs, followed by re-evaluation.

B. Experimental Setup

Datasets:
- SecLLMEval: 112 challenging NL prompts mapped to MITRE Top 25 CWEs (38 C, 74 Python).
- SecLLMHolmes: 106 real-world vulnerable code snippets converted to NL prompts (69 C, 37 Python).
Models: A diverse set of 4 LLMs was used: CodeLlama-7B, Mistral-7B, Meta-LLaMA 3, and GPT-3.5/GPT-4o.
Pipeline Designs: Ten distinct pipelines were implemented to test different coordination strategies:
- Single-Model (P1): Baseline generation/detection/patching by one LLM.
- Ensemble (P2–P5): Parallel generation with majority voting, CodeQL filtering, and ensemble-based patching.
- Collaborative (P6–P9): Purely LLM-based strategies including cross-model verification, chain-of-debate, and consensus without SAT.
- Hybrid (P10): Combines ensemble generation, SAT filtering, cross-LLM verification, and sequential patching.

C. Evaluation Metrics

Secure Code Rate: Fraction of outputs deemed secure by the evaluator.
Detection Metrics: Precision, Recall, and F1-Score for vulnerability identification.
Cost & Runtime: Analysis of API costs and execution time (dominated by CodeQL).

3. Key Contributions

Unified Framework: Introduction of MULTI-LLMSECCODEEVAL, the first framework to systematically evaluate multi-LLM protocols across the entire vulnerability lifecycle (Gen-Detect-Patch) with static analysis integration.
Comprehensive Benchmarking: Evaluation of four SOTA LLMs across two datasets, multiple languages, and CWE categories, demonstrating that no single model provides consistent security guarantees.
Hybrid Pipeline Superiority: Proposal and validation of a hybrid pipeline (P10) that integrates ensemble generation, static analysis, and cross-LLM verification, achieving the highest security rates.
Ablation Insights: Discovery that model scale does not guarantee security; smaller, structured multi-model ensembles often outperform large monolithic models.
Open Science: Public release of the framework, datasets, and evaluation artifacts (restricted to verified researchers to prevent misuse).

4. Key Results

A. Performance Gains

Ensemble + Static Analysis: Pipelines combining multiple LLMs with CodeQL (P2–P5) improved secure code generation by up to 47.3% on SecLLMEval and 19.3% on SecLLMHolmes compared to single-LLM baselines.
Pure LLM Collaboration: Collaborative strategies without static analysis (P6–P9) yielded modest gains (8.9–22.3%) and were significantly inferior to SAT-augmented approaches.
Hybrid Pipeline (P10): The hybrid approach achieved the best performance:
- 97.32% secure rate on SecLLMEval.
- 99.06% secure rate on SecLLMHolmes.
- Outperformed the best ensemble baseline by 1.78–4.72% and collaborative baselines by 19.81–26.78%.

B. Critical Findings

Scale vs. Structure: Smaller, specialized models (e.g., CodeLlama-7B, Mistral-7B) orchestrated in a structured ensemble often outperformed larger general-purpose models (e.g., GPT-4o, Meta-LLaMA) in complex tasks.
Role of Static Analysis: Static analysis (CodeQL) is the primary driver of security gains. LLM-only collaboration struggles with subtle or context-dependent CWEs that require rule-based verification.
Prompt Sensitivity: Vulnerability detection is highly sensitive to prompt diversity (performance drops to near-zero with variations), whereas code generation and patching are more robust to phrasing changes.
Task Complexity: Security performance degrades significantly as task complexity increases. Medium and hard tasks expose systematic blind spots (e.g., CWE-022, CWE-215) that single models cannot resolve.
Patch vs. Detect: LLMs are significantly better at patching known vulnerabilities (~70–80% success) than at detecting or classifying them correctly.

5. Significance and Implications

Paradigm Shift: The paper challenges the notion that "bigger is better" for secure code generation. Instead, it argues that carefully orchestrated multi-model system design (combining ensembling, static analysis, and verification) is the key to reliability.
Practical Deployment: The hybrid pipeline (P10) offers a practical blueprint for secure AI-assisted development, reducing unresolved vulnerabilities by over 90% compared to naive baselines.
Cost-Efficiency: While static analysis adds runtime overhead, the framework demonstrates that using smaller, open-source models in an ensemble is more cost-effective and secure than relying solely on expensive, large proprietary models.
Future Directions: The authors highlight the need for Retrieval-Augmented Generation (RAG) to address persistent blind spots (e.g., integer overflows) and the importance of integrating these pipelines into CI/CD workflows with human audit trails.

In conclusion, MULTI-LLMSECCODEEVAL demonstrates that secure code does not emerge from scale alone but arises from the deterministic orchestration of multiple models, static analysis tools, and collaborative verification strategies.