🤖 AI

Understanding the Impact of AI Code Assistants on Security API Usage: An Empirical Study

This empirical study of 44 professional developers reveals that while GitHub Copilot improves functional correctness and marginally reduces some insecure patterns, it fails to significantly enhance secure security API usage, often leaving developers unaware of remaining vulnerabilities.

Original authors: Zahra Mousavi, Chadni Islam, M. Ali Babar, Alsharif Abuadbba, Kristen Moore

Published 2026-07-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zahra Mousavi, Chadni Islam, M. Ali Babar, Alsharif Abuadbba, Kristen Moore

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you're building a super-secret clubhouse, and you need to install high-tech locks, alarms, and secret tunnels to keep intruders out. These locks are like Security APIs—special tools developers use to protect software. But here's the twist: these locks are tricky. If you install them wrong, the "secret" tunnel might just be a wide-open door for hackers.

Now, imagine you have a super-smart robot assistant named Copilot that can write code for you. You might think, "Great! The robot knows all the rules, so my clubhouse will be Fort Knox!" But a new study by Zahra Mousavi and her team at universities and research labs in Australia decided to put this idea to the test. They didn't just ask the robot to build a lock; they watched 44 real professional developers trying to build these locks, both with the robot's help and without it.

Here is what they found, served up with a side of reality.

The Robot is a Great Helper, But a Bad Security Guard

First, the good news: Copilot is a fantastic speedster. When the developers had to build a complex lock (specifically, a Google OAuth system for accessing user data), the robot helped them finish the job much faster.

Without the robot, only 45% of the developers (10 out of 22) actually got the lock working at all.
With the robot, 91% of the developers (20 out of 22) got it working.
The robot also cut the time it took to finish the complex task by 63%, dropping the average time from 62 minutes down to 23 minutes.

So, if your goal is just to get the door built and swinging, the robot is a huge win.

But here is the catch: Just because the door is built doesn't mean it's locked. In fact, none of the developers, whether they used the robot or not, managed to build a fully secure lock for the tasks they were given. Every single attempt on the evaluated tasks had at least one major flaw.

The "Trust Me" Trap

The study looked at two specific types of locks: JSSE (for secure internet connections) and Google OAuth (for logging into apps). They found that the robot didn't fix the security problems; in many cases, it just helped build the broken locks faster.

The "Trust All" Mistake: Imagine a guard who lets anyone in because they look nice. In the study, some developers (and the robot) set up the system to "trust all certificates." This is like telling your guard, "If anyone shows a badge, let them in, even if it's a fake one." This is a huge security hole.
The "Secret Code" Mistake: Some developers wrote their secret passwords directly into the code, like writing your house key under the doormat. The robot sometimes suggested this too!
The "No Check" Mistake: When connecting to a server, you need to check if you're actually talking to the right person. Many implementations skipped this step, leaving the door open for a "Man-in-the-Middle" attacker (someone pretending to be your friend to steal your stuff).

The researchers found that while the robot helped fix some small mistakes, it didn't stop the big security failures. In fact, 100% of the evaluated tasks resulted in code with security issues. The robot didn't make the code more secure than the humans working alone; it just made the insecure code appear faster.

The Human Blind Spot

Here is the most surprising part of the story. The researchers asked the developers, "Do you think your code is secure?"

Even though none of the code was actually secure, most developers said, "Yep, I think I solved this securely!"
Only a tiny handful of people realized they had made a mistake.
When talking to the robot, almost no one asked, "Hey, is this safe?" They just asked, "Make it work."

It's like a group of people building a bridge and asking a robot for help. The robot helps them lay the bricks quickly, but nobody checks if the bridge can actually hold weight. When asked, "Do you think this bridge will hold?" most people say, "Sure!" even though it's about to collapse.

Can We Teach the Robot Better?

The researchers tried one more thing. They asked the developers to specifically tell the robot, "Hey, check this for security!" or "You are a security expert, fix this!"

When they did this, the robot did fix some of the mistakes in the simpler lock (JSSE).
However, for the more complex lock (OAuth), the robot still missed many critical security holes, even when told to be careful.

The Bottom Line

This study suggests that while AI assistants like Copilot are amazing at helping us write code quickly and correctly, they are not a magic shield against hackers.

The Robot: Great at speed and function, but not a security expert.
The Humans: Often too trusting of the robot and not checking the security themselves.
The Result: We are building secure-looking systems that are actually full of holes.

The researchers conclude that we can't just rely on the robot to keep us safe. We need to be the ones asking the hard security questions, checking the work, and making sure the "locks" we build are actually locked. The robot is a powerful tool, but it's not a replacement for a security-conscious mind.

Technical Summary: Understanding the Impact of AI Code Assistants on Security API Usage

Problem Statement

Security Application Programming Interfaces (APIs), such as SSL/TLS and OAuth, are fundamental to modern software security but are frequently misused due to their complexity, insufficient documentation, and a lack of developer training. Such misuse leads to severe vulnerabilities, including Man-in-the-Middle (MitM) attacks and data breaches. While Large Language Model (LLM)-based AI code assistants like GitHub Copilot are increasingly adopted to accelerate development, their impact on the secure usage of these critical APIs remains under-explored.

Existing research has largely focused on model outputs generated from predefined prompts without developer intervention, or on general vulnerability patterns rather than specific API misuse. There is a critical gap in understanding how professional developers interact with AI assistants when implementing security APIs, how these tools influence the final security of the code, and the extent to which developers recognize and mitigate security issues in AI-assisted workflows.

Methodology

The authors conducted a within-subject empirical study involving 44 professional Java developers. The study design compared two conditions for each participant:

Experimental Condition: Completing a task with GitHub Copilot assistance.
Control Condition: Completing a different task without any AI assistance.

Tasks:
Participants were assigned two programming tasks involving distinct security APIs:

JSSE Task: Implementing a createSSLSocket method to establish secure SSL/TLS connections.
Google OAuth Task: Implementing an authorize method for a desktop application to access user Gmail data via OAuth.

Procedure:

Participants worked in a controlled AWS virtual environment with VS Code.
The study utilized counterbalancing to mitigate learning and fatigue effects, splitting participants into four groups based on task order and Copilot availability.
Sessions were screen-recorded, and participants completed post-task surveys regarding their security awareness and self-assessment of code security.
Evaluation: Code was analyzed for functional correctness and security. Security analysis involved a manual review by two security experts using a taxonomy of security API misuses (Mousavi et al.). A logistic regression model was used to analyze the impact of Copilot and participant background on outcomes. Additionally, a post-study analysis involved re-prompting Copilot with security-focused instructions to test its ability to identify and fix misuses.

Key Contributions

This study presents the first developer-centered empirical investigation into the use of AI code assistants for security API tasks in realistic settings. The primary contributions are:

Empirical Evidence on Copilot's Impact: A comparative analysis of Copilot-assisted versus unaided development regarding functional correctness and secure API usage.
Identification of Persistent Misuses: A detailed categorization of security API misuses that persist, are introduced, or are partially mitigated when using Copilot.
Analysis of Developer Awareness: An investigation into how developers engage with Copilot regarding security, including their prompt strategies and their ability to recognize insecure implementations.
Prompt Engineering Insights: An evaluation of whether targeted security-focused prompts can effectively guide Copilot to generate more secure code.

Key Results

Functional Correctness (RQ1)

Copilot significantly improved functional correctness, particularly for the more complex OAuth task.

JSSE Task: All participants produced functional code regardless of Copilot use, though Copilot users completed the task faster (14 mins vs. 26 mins).
OAuth Task: Success rates increased from 45% (unaided) to 91% (Copilot-assisted), with completion times decreasing by 63%.
Statistical Significance: Logistic regression confirmed a statistically significant positive effect of Copilot on functional outcomes ( $p = 0.007$ ).

Security Outcomes (RQ2)

Despite improvements in functionality, no participant produced a fully secure implementation in either condition.

JSSE Misuses: Three types were identified: Insecure SSL/TLS standards (M1), trusting all certificates (M2), and missing hostname verification (M3). Copilot showed a slight, non-significant reduction in M1 and M2 but did not prevent M3.
OAuth Misuses: Six critical misuses were found, including local storage of secrets (M4), local storage of tokens (M5), lack of state parameters (M6), lack of SP authentication (M7), lack of PKCE (M8), and inadequate transmission protection (M9).
Overall Impact: Copilot did not significantly improve secure API usage. The rate of misuse was high across both conditions, with 100% of implementations containing at least one security flaw.

Security Awareness (RQ3)

Developers demonstrated limited security awareness when engaging with Copilot.

Prompting: Only 7 out of 44 participants explicitly mentioned security in their prompts. In most cases, security language was verbatim from the task description rather than a genuine concern. Only two participants demonstrated deliberate security consideration in their prompts.
Self-Assessment: There was a significant mismatch between perceived and actual security. Most participants believed their solutions were secure despite the presence of critical vulnerabilities.
Post-Study Prompting: When explicitly prompted to "review the code and improve it based on security best practices," Copilot successfully corrected some JSSE misuses (e.g., M1 and M2) and offered recommendations for OAuth token storage. However, it failed to address other critical OAuth issues like the lack of state parameters or PKCE, indicating limitations in its security reasoning for complex flows.

Significance and Claims

The paper claims that the security challenges in AI-assisted development are not solely a limitation of the AI model's output but also a developer-awareness problem. Even when informed that security would be evaluated, developers rarely engaged with the tool from a security perspective and often failed to recognize insecure code.

The study concludes that while AI assistants like Copilot enhance productivity and functional correctness, they do not currently enable developers to produce fully secure implementations of security APIs. The authors emphasize that:

Blind reliance on AI-generated code introduces serious vulnerabilities.
Security-oriented prompting can yield improvements but is not a panacea for all misuse types.
Effective use of AI in secure development requires developers to maintain a strong foundation in security principles and critically assess AI-generated code.

The authors position this work as a foundational step toward understanding the intersection of AI and software security, calling for future research into security-aware tooling, improved API usability, and better alignment of LLMs with security best practices.