Can AI Write Compliant Code, and to What Extent? Evaluating SOC 2 Compliance of Claude Fable 5, Claude Opus 4.8, and Claude Opus 5 Across Four Use Cases
This study evaluates the SOC 2 compliance of three frontier AI models across four use cases, finding that while unprompted code generation often contains critical vulnerabilities and misses key security controls, adding a single sentence referencing SOC 2 standards significantly improves conformance to 86–100% and eliminates insecure constructions, though it fails to address controls outside the model's immediate task conception and highlights the unreliability of automated pattern-matching scoring in favor of semantic verification.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
The Invisible Safety Net
Imagine you are hiring a super-smart, tireless apprentice to build you a house. This apprentice, an Artificial Intelligence, can draft blueprints, mix concrete, and install windows faster than any human team. But here's the catch: the apprentice has never been told about building codes. It knows how to make a house that looks good and stands up, but it might forget to install the smoke detectors, lock the back door, or reinforce the foundation against earthquakes. In the world of software, this is exactly what happens when teams ask AI to write code. The code works, but it might be missing the invisible safety rules required to keep data safe from hackers.
This paper dives into a specific corner of computer science called compliance, which is just a fancy word for "following the rulebook." Specifically, the researchers looked at SOC 2, a set of rules companies use to prove they are protecting customer data. They wanted to know two things: If you ask an AI to write code without mentioning safety, will it accidentally follow the rules anyway? And if you just add one tiny sentence saying, "Hey, this needs to follow SOC 2 rules," does that fix everything? It's a bit like asking: "If I tell a chef to make a sandwich, will they remember to wash their hands? And if I whisper 'health code,' will they suddenly remember to wear gloves?"
The Great AI Code Test
To find the answers, the researchers at Vanta set up a massive, controlled experiment. They treated three different versions of a powerful AI (named Claude Fable 5, Claude Opus 4.8, and Claude Opus 5) like contestants in a cooking competition. They gave each AI four different "recipes" to code: a tool to manage cloud storage, a login system, a database setup, and a file-upload handler.
They ran the test twice for every recipe. In the first round, they gave the AI a neutral prompt, like "Write a tool to upload files." In the second round, they gave the exact same prompt but added one single sentence: "This data is sensitive and must comply with SOC 2 standards."
The researchers then acted as strict inspectors. They didn't just run the code to see if it worked; they checked every single line against a checklist of safety rules. They looked for things like encryption (scrambling data so thieves can't read it), access controls (making sure only the right people can see files), and logging (keeping a diary of who did what).
What They Found: The "Magic Sentence" and the Blind Spots
The results were a mix of surprising good news and some very important warnings.
1. The "Magic Sentence" Works Wonders (But Not Everything)
When the AI was asked to write code without any safety instructions, it was hit-or-miss. Depending on the task, the code followed the rules between 47% and 88% of the time. It was good at the basics, like password protection, but terrible at the boring, invisible stuff like locking down cloud storage buckets.
However, the moment the researchers added that one single sentence about SOC 2, the AI's performance skyrocketed. Every single test jumped to between 86% and 100% compliance. That one sentence was worth 23 to 50 percentage points of improvement. It was as if the AI suddenly woke up and remembered, "Oh right, I need to wear a helmet!" Crucially, this sentence also removed every insecure construction the researchers found, such as open debuggers or unauthenticated downloads.
2. The "Idiom" Problem
The study found that the AI follows rules that are part of the "normal way" of writing code. For example, it almost always remembered to hash passwords (scrambling them so they can't be stolen) because that's just how you write a login system. But it forgot things that aren't part of the "standard recipe," like adding extra locks to a cloud storage bucket or setting up a multi-factor authentication (MFA) hook.
Even with the "magic sentence," a stable set of controls remained missing because they sit outside the model's default conception of the task. Specific features like MFA hooks, secure cookie flags, and account lifecycle management were not automatically added just by naming the standard; they had to be named individually in the prompt. The magic sentence fixed the big picture and the hidden dangers, but it couldn't fill in every tiny, specific detail unless you asked for them by name.
3. The "Invisible" Dangers
Here is where it gets scary. In three out of sixteen "neutral" tests (where no safety was mentioned), the AI wrote code that was actually dangerous, even though it looked fine.
- One AI left a "debugger" open, which is like leaving a backdoor in a house that lets anyone walk in and take control of the computer.
- Two others created download links that didn't check who was clicking them, allowing anyone to download other people's private files.
- Another AI created a database that would accidentally delete all its data if you tried to shut it down in a non-production environment.
The researchers' first automated scanner missed all of these dangerous mistakes. It was only when a human looked closely that they realized the code was broken. This proves that standard computer scanners aren't smart enough to catch these "invisible" errors in AI code.
4. The Model Doesn't Matter Much
The researchers expected that the newest, smartest AI (Opus 5) would be much better than the older ones. But the difference was tiny. The "magic sentence" (the prompt) made a much bigger difference than changing the AI model itself. If you are trying to get compliant code, picking a newer AI model won't help you as much as simply telling the AI to follow the rules.
The Bottom Line
This paper teaches us that AI is a fantastic builder, but it's not a safety inspector. If you ask it to build something without mentioning safety, it will build a house that stands up but might lack smoke detectors or locks. If you add a single sentence about following the rules, it will suddenly install most of the safety gear and remove the dangerous backdoors.
However, there are still gaps. The AI won't automatically install specific, complex safety features (like MFA or specific cookie settings) unless you ask for them by name, even if you mention the rules. And, perhaps most importantly, you can't just trust a computer program to check the AI's work; you need a human to look closely, because the AI can hide dangerous mistakes in clever ways that simple scanners miss.
The lesson for anyone using AI to write code is simple: Don't assume the AI knows the rules. You have to tell it the rules, and then you have to double-check its work, because a house that looks perfect might still have a hidden trapdoor.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.