💻 computer science

Stochastic Code, Deterministic Defense: Assessing the Security Blind Spots in GenAI-Driven CI/CD Pipelines

This study empirically demonstrates that traditional deterministic security tools fail to detect context-dependent, syntactically valid vulnerabilities introduced by Generative AI in CI/CD pipelines, highlighting an urgent need for probabilistic, intent-aware verification frameworks.

Original authors: Mohammed BEDJAOUI, Sidi Mohammed BENSLIMANE, Mohammed Yassine KAZI TANI

Published 2026-08-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mohammed BEDJAOUI, Sidi Mohammed BENSLIMANE, Mohammed Yassine KAZI TANI

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where software isn't just written by humans typing on keyboards, but is instead "dreamed up" by a super-smart robot that guesses the next word, the next line, and the next function based on what it has seen before. This is the realm of Generative AI (or GenAI), a technology that is rapidly changing how we build digital tools. In the modern tech world, these tools are often assembled in a high-speed factory called a CI/CD pipeline. Think of this pipeline as a conveyor belt where code is written, tested, packaged, and shipped out to the internet in seconds.

For years, the security guards watching this conveyor belt have been deterministic scanners. These are like strict bouncers with a clipboard of "banned moves." If they see a specific pattern they know is dangerous—like a known lockpick or a forbidden spell—they stop the code. They are great at catching the obvious bad guys. But here's the twist: GenAI doesn't just copy-paste old bad moves; it creates new variations every single time, like a jazz musician improvising a melody. The big question for scientists and engineers is: Can our old, strict bouncers catch a jazz musician who is playing a song that sounds perfect but is secretly dangerous? This paper dives into that exact mystery, asking if our current security tools are blind to the unique, unpredictable mistakes that AI makes.

The Story of the "Silent Decay"

In this study, researchers Mohammed Bedjaoui, Sidi Mohammed Benslimane, and Mohammed Yassine Kazi Tani set up a digital experiment to see what happens when you let a local AI robot write code for a real-world application and then try to check it with standard security tools. They didn't just guess; they ran a controlled simulation 50 times, acting like a mad scientist in a lab, to see if the AI could sneak a "bad idea" past the security guards.

The Setup: A Robot Writer and a Strict Guard
The team built a miniature version of a modern software factory. They used a local AI model (a "Code Llama" robot) to write two types of code:

The Logic Flaw (V1): The AI was asked to write a security check for a website. It wrote code that looked perfect and followed all the grammar rules, but it forgot a crucial step: it didn't actually check if the user was who they claimed to be. It was like a bouncer at a club who checks your ID card but forgets to check if the photo matches your face.
The Infrastructure Drift (V3): The AI was asked to write instructions for a server (a Kubernetes manifest). It wrote instructions that looked syntactically correct but set up a "health check" that was too simple. It was like a doctor who checks if a patient is breathing but ignores whether their heart is beating. The server would stay "alive" even if the software inside it was completely broken.

The Test: Can the Old Tools See the New Mistakes?
Once the AI wrote the code, it was sent through a standard security pipeline using three famous "bouncers": Semgrep, Trivy, and Kube-linter. These tools are the industry standard, designed to catch known errors by looking for specific patterns.

The results were startling. In the 50 runs of the experiment:

The AI successfully generated code that passed the build process 46 times for the Logic Flaw scenario and 48 times for the Infrastructure Drift scenario. The code was "valid" enough to be shipped.
When the security tools scanned this code, they failed to detect the vulnerability 100% of the time (or 96% in one specific count, with only 2 false alarms out of 48).
The statistical analysis showed that this wasn't just bad luck; the probability of these tools missing the errors by chance was less than 0.001 (p < .001).

The Big Reveal: The "Blind Spot"
The paper concludes that there is a massive "blind spot" in our current security systems. The AI didn't write "evil" code; it wrote code that focused on functionality rather than security. Because the code followed all the grammar rules, the deterministic scanners (the bouncers with the clipboards) said, "All clear!" and let it through.

The researchers call this "Silent Decay." It's a scenario where the software factory keeps running smoothly, the lights stay green, and the security alarms never ring, but the final product is fundamentally broken and insecure. The study explicitly rules out the idea that these tools are failing because the code was too messy or "hallucinated" nonsense; the code was clean, valid, and just missed the point of security.

What This Means
The authors argue that we cannot rely on the old way of checking code anymore. If the code is generated by a probabilistic AI (one that guesses and varies its output), a deterministic scanner (one that looks for fixed patterns) will always miss the subtle, context-dependent mistakes. The paper suggests that we need a new kind of defense—one that understands the intent of the code, not just its shape. They propose using other AI agents to audit the code, or using "neuro-symbolic" methods that combine human-like understanding with mathematical logic.

In short, the study proves that in the age of AI, a "green light" from a security scanner doesn't mean the code is safe. It just means the code looks good on paper. The real danger is hiding in the gaps between the lines, where the AI's creativity outpaces our ability to check it.

Technical Summary: Stochastic Code, Deterministic Defense

Problem Statement
The integration of Generative AI (GenAI) into Continuous Integration/Continuous Deployment (CI/CD) pipelines introduces a fundamental misalignment between the probabilistic nature of AI-generated artifacts and the deterministic, signature-based logic of traditional security tools. While standard DevSecOps practices rely on Static Application Security Testing (SAST) and Infrastructure-as-Code (IaC) scanners to detect known vulnerabilities (e.g., SQL injection, CVEs), these tools struggle with "stochastic vulnerabilities." These are context-dependent flaws where code is syntactically correct but semantically insecure. The paper identifies a critical "blind spot": standard tools often fail to detect insecure logic or configuration drifts introduced by Large Language Models (LLMs) because the vulnerabilities manifest as the absence of security controls rather than the presence of malicious patterns, leading to "silent decay" in automated software supply chains.

Methodology
The study employs a controlled, empirical experimental design to quantify the detection failure rate of industry-standard tools against stochastic vulnerabilities.

Experimental Environment: A local, closed-loop DevSecOps pipeline was constructed using a GitLab Runner, Minikube (Kubernetes), and an uncensored local LLM (codellama:7b-instruct via Ollama) to simulate an unrestricted code contributor. The target application was aureuserp, an open-source Laravel-based ERP system.
Injection Scenarios: The experiment focused on two specific "blind spot" scenarios, excluding "False Positives" (which typically break the build) to isolate "False Negatives" (silent failures):
1. Logic Flaws (V1): The LLM generated PHP middleware that checked for a specific header but omitted critical session validation (Broken Access Control).
2. Infrastructure Configuration Drift (V3): The LLM generated Kubernetes manifests with a liveness probe pointing to a generic root path (/), which returns a 200 OK even if the application logic is broken, creating a "Zombie Container."
Protocol: The experiment ran $N=50$ independent iterations. Valid runs (where the AI generated syntactically correct code) were subjected to a security scan using Semgrep (SAST), Trivy (IaC/Container), and Kube-linter.
Statistical Analysis: The study utilized a One-Sample Binomial Test to compare the observed detection rates against a theoretical benchmark of 0.95 (95% detection rate), rejecting the Null Hypothesis ( $H_0$ ) if the tools failed to meet this threshold.

Key Results
The empirical analysis revealed a systemic inability of deterministic tools to detect AI-induced vulnerabilities:

Logic Flaws (V1): Semgrep failed to detect the broken access control vulnerability in 100% of the 46 valid runs (Observed Proportion = 0.00). The statistical significance was $p < 0.001$ , leading to the rejection of the hypothesis that standard SAST tools are effective against these logic flaws.
Infrastructure Drift (V3): Trivy and Kube-linter failed to detect the insecure liveness probe configuration in 100% of the 48 valid runs (Observed Proportion = 0.00). The tools validated the syntax of the YAML but missed the functional intent, confirming that syntactic compliance checks are insufficient for cloud-native security in this context.
Validity: The high rate of valid artifact generation (>90%) confirmed that local LLMs can produce code that successfully passes compilation and security gates while remaining fundamentally insecure.

Significance and Claims
The paper claims that current deterministic defenses are insufficient for GenAI-driven pipelines. The core contribution is the empirical demonstration that the "if the scanner passes, the code is safe" assumption is no longer valid in the era of probabilistic code generation.

The "Blind Spot": The study highlights that SAST and IaC tools are designed to flag "dirty patterns" (presence of bad code) but struggle to verify "negative constraints" (absence of necessary checks) or understand functional intent.
Silent Decay: The research underscores the danger of "silent decay," where pipelines remain "green" and builds succeed, yet the deployed artifacts contain critical security gaps that bypass all automated validation.
Paradigm Shift: The authors argue that securing the next generation of AI-augmented supply chains requires a shift from static pattern matching to "intent-aware verification" and probabilistic security frameworks. They suggest that future defenses may need to be probabilistic themselves (e.g., using LLMs to audit LLMs) or rely on runtime verification (e.g., eBPF) to catch logic flaws that static analysis misses.

Limitations and Scope
The authors modestly acknowledge that the study utilized a quantized local model (7B parameters) rather than commercial, RLHF-filtered models (e.g., GPT-4), which might exhibit fewer insecure outputs. However, they argue this is relevant for organizations deploying local LLMs for data privacy. Additionally, the study used default tool configurations; while custom rules could theoretically catch these specific flaws, the authors note that relying on custom rules undermines the goal of automated, general-purpose security. The study focuses specifically on the "Blind Spot" of detection failure, not on the generation of overtly malicious payloads.

The Story of the "Silent Decay"

Technical Summary: Stochastic Code, Deterministic Defense

More like this