💻 computer science

The Impact of AI-Assisted Development on Software Security: A Study of Gemini and Developer Experience

This study of 159 developers reveals that while Google's AI tool Gemini does not significantly alter code security outcomes regardless of its version (free or paid), a developer's own programming experience remains the critical factor in producing secure software and cannot be fully replaced by AI assistance.

Original authors: Nadine Jost, Benjamin Berens, Manuel Karl, Stefan Albert Horstmann, Martin Johns, Alena Naiakshina

Published 2026-03-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nadine Jost, Benjamin Berens, Manuel Karl, Stefan Albert Horstmann, Martin Johns, Alena Naiakshina

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are building a house. You have a team of carpenters, but there's a massive shortage of skilled workers. To keep up with demand, you decide to hire a super-fast, super-smart robot assistant (let's call him "Gemini") who can draft blueprints and cut wood in seconds.

The big question for the construction industry is: Can this robot replace the need for experienced human carpenters, especially when it comes to making the house secure against burglars?

This paper is a scientific experiment that tried to answer that question. Here is the breakdown of what they did and what they found, using simple analogies.

The Experiment: The "Linktree" Challenge

The researchers gathered 159 freelance developers (think of them as professional carpenters) and gave them a specific task: Build a simple website where users can sign up, log in, and manage a list of links (like a digital business card).

They split the carpenters into three groups:

The Solo Group: No robot help. Just them and their own skills.
The Free Robot Group: They could use the free version of Google's AI, Gemini.
The Premium Robot Group: They could use the paid, "Advanced" version of Gemini.

They then inspected the finished websites for five common "security holes" (like unlocked doors, broken locks, or weak foundations):

XSS: Letting bad guys inject their own code into your site.
CSRF: Tricking a user into doing something they didn't mean to do.
Bad Input Validation: Accepting weird data that breaks the system.
SQL Injection: Sneaking database commands through the front door.
Password Storage: Storing passwords in plain text (like writing your PIN on a sticky note).

The Big Findings

1. Experience is King (The Veteran vs. The Rookie)

The Result: The most experienced developers built the safest houses, regardless of whether they used a robot or not.
The Analogy: Imagine a master carpenter who has built 1,000 houses. Even if they use a basic hammer, they know exactly where to put the nails so the roof doesn't leak. A rookie, even with a high-tech laser cutter, might still build a wobbly wall because they don't understand why the wall needs to be strong.
Takeaway: AI is a great tool, but it cannot replace the deep, gut-level intuition that comes from years of experience. If you are a junior developer, the AI can help you learn, but it can't do the critical thinking for you.

2. The Robot Didn't Make the House Safer (The "Magic Wand" Myth)

The Result: Surprisingly, the groups using the AI (both free and paid) did not build significantly safer websites than the group working alone.
The Analogy: You might think giving a carpenter a "magic wand" that auto-fixes errors would make the house bulletproof. But in this study, the magic wand just handed them a blueprint that had some hidden traps. The developers had to catch those traps themselves.
Takeaway: AI generates code quickly, but it often generates code with security holes. It didn't magically fix the security problems; it just moved the work around.

3. Free vs. Paid: Is the "Premium" Version Worth It?

The Result: There was no difference in security between the free version of Gemini and the paid "Advanced" version.
The Analogy: It's like buying a "Deluxe" box of crayons versus a "Basic" box. You might expect the Deluxe box to have colors that are safer or brighter, but when it came to drawing a secure house, both boxes produced the exact same shaky lines.
Takeaway: Companies might not need to spend extra money on the premium AI subscriptions if their main goal is just to make software more secure. The free version was just as good (or just as bad) as the paid one.

4. The "Trust" Trap

The Result: Developers generally trusted the AI, but the ones with security experience were more skeptical.
The Analogy:

The Novice: "The robot said the door is locked, so I believe it!" (They might miss a hidden backdoor).
The Veteran: "The robot said the door is locked, but I'm going to kick it to make sure." (They verify the AI's work).
Takeaway: The more experienced you are, the less you blindly trust the AI. You use it as a starting point, then you double-check its work.

Why Did Some People Do Better with AI?

Interestingly, developers who had no prior security experience actually did slightly better when they used the AI than when they worked alone.
The Analogy: A rookie carpenter who has never seen a lock might forget to install one entirely. But if a robot says, "Hey, you need a lock here," the rookie follows the instruction and ends up with a locked door. The AI acted as a safety net for the beginners, reminding them of things they forgot.

The Final Verdict: What Should We Do?

The paper concludes with a warning for the software industry:

Don't fire your senior developers and replace them with AI.

If companies think AI can do the job of a skilled security expert, they are making a dangerous mistake.

AI is a Co-Pilot, not the Captain: It's great for speeding up routine tasks and helping beginners learn.
Human Expertise is Non-Negotiable: You still need experienced humans to look at the AI's work, spot the hidden traps, and ensure the "house" is actually secure.
Don't Pay for "Better" Security: If you are just looking for better security, the free version of AI is likely just as effective as the expensive one.

In short: AI is a powerful assistant, but it's not a substitute for a skilled craftsman. If you want a secure house, you still need a human who knows how to build it.

1. Problem Statement

The software industry faces a critical shortage of skilled developers, particularly in security-critical domains. To bridge this gap, organizations are increasingly adopting AI-powered development tools (Large Language Models or LLMs) to automate tasks and reduce reliance on human expertise. However, there is a lack of empirical evidence regarding:

How AI tools affect the security of the resulting software.
Whether the type of AI tool (free vs. paid versions) impacts security outcomes.
How developer experience (general programming vs. security-specific) interacts with AI assistance.
Whether AI can substitute for human expertise in producing secure code.

Specifically, while tools like GitHub Copilot and ChatGPT have been studied, Google's Gemini (both free and paid "Advanced" versions) remains underexplored in the context of software security.

2. Methodology

The authors conducted a quantitative, remote field study involving 159 software developers recruited via Upwork.

Experimental Design: Participants were randomly assigned to one of three groups:
1. No-AI (Control): Instructed not to use any AI tools.
2. Free-AI: Instructed to use the free version of Gemini.
3. Paid-AI: Provided with credentials for a Google One subscription including access to Gemini Advanced (paid version).
Task: Participants were tasked with building a web application (similar to "Linktree") using Python and Flask. The task required implementing:
1. User registration.
2. User login.
3. Adding websites to a user profile.
4. Deleting websites.
- Constraint: The task was explicitly designed to be susceptible to common web vulnerabilities.
Security Evaluation: Submissions were evaluated against five specific vulnerability categories derived from the OWASP Top Ten and CWE Top 25:
1. Cross-Site Scripting (XSS)
2. Cross-Site Request Forgery (CSRF)
3. Improper Input Validation
4. SQL Injection
5. Cryptographic Failures (specifically insecure password storage).
- Scoring: A binary security score (0–5) was assigned based on the presence/absence of these vulnerabilities.
Data Analysis:
- Quantitative: Logistic regression (Binomial GLM) was used to model the security score, controlling for programming experience, security experience, self-efficacy, and workload.
- Qualitative: Thematic analysis of open-ended survey responses regarding trust, challenges, and AI usage patterns.
- Tools: Code was manually reviewed by two researchers and cross-verified using static analysis tools (Bandit and SonarQube).

3. Key Contributions

First Study on Gemini: This is one of the first empirical studies specifically analyzing the security impact of Google's Gemini (both free and paid tiers) on software development.
Free vs. Paid Comparison: It provides rare empirical data comparing the security outcomes of free versus paid LLM versions, challenging the assumption that paid versions inherently produce more secure code.
Experience Interaction: It investigates the complex interaction between developer expertise (general vs. security-specific) and AI assistance, revealing counter-intuitive findings regarding security experience.
Comprehensive Vulnerability Analysis: The study moves beyond general code quality to specifically measure the mitigation of five distinct, high-impact web vulnerabilities.

4. Key Results

A. Impact of Developer Experience (RQ1 & RQ2)

Programming Experience: There is a significant positive correlation between general programming experience and code security. Developers with more years of experience produced significantly more secure code.
- Statistical Finding: A one-standard-deviation increase in programming experience increased the odds of a higher security score by a factor of 1.43 ( $p=0.002$ ).
Security Experience: Surprisingly, prior security experience did not significantly improve security scores. In fact, developers with self-reported security experience tended to produce slightly less secure code than those without, though this was not statistically significant ( $p=0.058$ $p = 0.058$ ).
- Qualitative Insight: Experienced developers may suffer from "security overconfidence," leading them to overlook vulnerabilities, whereas less experienced developers relied more heavily on AI suggestions or external resources.

B. Impact of AI Tool Type (RQ3)

No Significant Difference: There was no statistically significant difference in security scores between the No-AI, Free-AI, and Paid-AI groups.
Paid vs. Free: The paid version (Gemini Advanced) did not produce significantly more secure code than the free version.
Specific Vulnerability Improvements:
- Password Storage: AI assistance reduced insecure password storage by 21.7%.
- SQL Injection: AI assistance reduced SQL injection vulnerabilities by 10.5%.
- XSS, CSRF, Input Validation: AI had minimal to no impact on these vulnerabilities (improvements of <4% or negligible).
Workload: The Paid-AI group reported significantly lower workload (NASA-TLX scores) compared to the other groups, suggesting the paid version may be more efficient, even if not more secure.

C. User Trust (RQ4)

Trust Levels: There was no significant difference in trust levels between users of the free and paid versions of Gemini.
Trust Drivers: Developers trusted AI suggestions primarily when they could verify them against their own knowledge or external documentation (e.g., Stack Overflow, official docs).
Mistrust Drivers: Mistrust arose from hallucinations, outdated security libraries, incorrect code, and the AI ignoring context.
Preference: A majority of the control group (No-AI) expressed they would have preferred to use AI assistance.

5. Significance and Implications

AI is a Supplement, Not a Substitute: The study concludes that AI tools cannot fully substitute for programming experience. While AI can assist novices, experienced developers remain essential for producing secure code. Replacing junior developers with AI could degrade the long-term security posture of the industry by preventing the accumulation of foundational skills.
Paid Versions Do Not Guarantee Security: Organizations should not assume that investing in paid AI tiers (e.g., Gemini Advanced) will result in significantly more secure software compared to free versions.
Security Process Remains Critical: Since AI does not eliminate vulnerabilities (and sometimes introduces new ones), traditional security processes (code reviews, static analysis, penetration testing) remain mandatory regardless of AI usage.
Developer Behavior: Developers with security experience may need to be more vigilant against overconfidence. Conversely, developers without security experience should use AI as a "second pair of eyes" but must validate all suggestions, particularly regarding authentication and input validation.
AI Model Training: The findings suggest that AI training data needs improvement, specifically regarding the inclusion of the latest security libraries and the reduction of outdated or insecure code patterns (e.g., plain text password storage).

Conclusion

The paper demonstrates that while AI tools like Gemini can improve developer productivity and assist with specific tasks like password hashing, they do not significantly enhance the overall security of software compared to human-only development. The most critical factor for secure software remains human programming experience. Organizations should view AI as a productivity booster rather than a security solution.