💬 NLP

DevIntent: How Much Does LLM-Generated Code Violate Developer Intent?

This paper introduces the Intent Violation Rate (IVR) metric and a corresponding benchmark to reveal that while LLM-generated code frequently passes standard visible tests, it systematically violates developers' implicit intentions in over half of the cases, suggesting that current pass rates significantly overestimate the alignment between generated code and true developer intent.

Original authors: Susana Haing, Natan Vidra, Spurthi Setty

Published 2026-08-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Susana Haing, Natan Vidra, Spurthi Setty

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a super-talented, lightning-fast robot chef to cook dinner for your family. You tell it, "Make me a sandwich." The robot whirs to life, grabs the bread, slices the turkey, and plates a beautiful, perfect-looking sandwich. You take a bite, and it tastes exactly like a turkey sandwich. By every standard test you can think of—does it have bread? Yes. Does it have turkey? Yes. Is it edible? Yes. The robot has passed the test with flying colors.

But here's the twist: your family is vegetarian, and you forgot to mention that. The robot, being a literal-minded machine, didn't ask for clarification; it just assumed "sandwich" meant "sandwich with meat." It followed your instructions perfectly, but it completely missed your intent. In the world of computer science, specifically in the field of Artificial Intelligence (AI), this is a growing headache. We have powerful AI models that can write computer code, but they often act like that literal-minded chef. They can write code that passes all the official "tests" a programmer writes, yet the code does something slightly different from what the programmer actually wanted. This paper dives into that gap between "what the code does" and "what the human meant."

The researchers behind this study, working with models like Claude Sonnet 4.6 and OpenAI's GPT-4.1, decided to stop just checking if the code "works" and start checking if it "gets it." They created a new way to measure this, which they call the Intent Violation Rate (IVR). Think of IVR as a "mind-reading test" for AI. Instead of just asking, "Did the code run without crashing?" they ask, "Did the code do exactly what the human was thinking, even the parts the human forgot to say out loud?"

To test this, the team built a special playground of 49 tricky coding puzzles. They took clear, detailed instructions (the "Gold Prompt") and stripped away the important, unspoken details to create a vague, ambiguous version (the "Ambiguous Prompt"). They then asked the AI to solve the puzzle using only the vague version. The AI was given a basic checklist (the "Stated Test") to prove it solved the main problem, but the researchers also hid a secret checklist (the "Hidden Constraints") that captured the details the AI was supposed to guess. For example, if the vague prompt was "Sort these numbers," the AI might just list them. But the hidden constraint might be "Sort them from smallest to largest." If the AI listed them in random order, it would pass the basic test but fail the hidden one, violating the developer's intent.

The results were a bit of a shock. When the researchers looked at how often the AI passed the basic, visible tests, the numbers looked amazing: Claude Sonnet 4.6 passed 94.3% of the time, and GPT-4.1 passed 92.7% of the time. If you only looked at those numbers, you'd think the AI was a coding genius. But when they checked the secret "Intent Violation Rate," the picture changed dramatically.

Even though the code passed the visible tests, it violated the developer's hidden intent in 54.5% of the cases for Claude and 63.5% for GPT-4.1. That means in more than half of the problems, the AI wrote code that technically worked but was fundamentally wrong about what the human wanted. The researchers found that this wasn't just random noise or a few bad guesses. The AI's behavior was surprisingly consistent and "bimodal," meaning it tended to either get the hidden intent perfectly right or completely miss it, with very few "almost right" answers in between.

The study suggests that simply counting how many tests a piece of code passes is not enough to know if it's good. A high pass rate might give developers a false sense of security, making them think the code is ready to deploy when it's actually missing crucial, unspoken requirements. The authors note that while their findings are based on a specific set of 49 problems and two specific AI models, the pattern they found is systematic. They argue that we need new ways to measure code quality that look beyond the surface-level "pass/fail" results to ensure the AI is truly understanding the human's vision, not just gaming the test.

Technical Summary: DevIntent - How Much Does LLM-Generated Code Violate Developer Intent?

Problem Statement

Current benchmarks for Large Language Models (LLMs) in code generation primarily rely on test pass rates (e.g., Pass@1) to measure functional correctness. However, this metric is insufficient because it fails to detect intent violations. An LLM may generate code that satisfies all explicitly stated test cases while simultaneously violating implicit constraints or unstated developer intentions. Prior work has documented cases where models "game" specifications or overfit to assert statements, but this paper focuses on a distinct failure mode: outputs that legitimately pass valid stated tests yet fail to capture the developer's actual, unstated intent.

Methodology

1. Metric Definition: Intent Violation Rate (IVR)

The authors introduce the Intent Violation Rate (IVR) to quantify the gap between stated test success and developer intent.
$IVR = \frac{n_{violating}}{n_{stated\_passed}}$
Where:

$n_{stated\_passed}$ is the number of generated solutions passing all visible, stated tests.
$n_{violating}$ is the subset of those solutions that fail at least one hidden constraint test.

IVR is calculated only over solutions that pass the initial public tests; solutions failing the stated tests are excluded. The reported IVR is the mean of per-problem IVRs.

2. Benchmark Construction

The study utilizes a pilot benchmark of 49 algorithm specification problems derived from HumanEval+.

Prompt Engineering: For each problem, a "gold prompt" (containing full specifications) was manually stripped of specific constraints to create an ambiguous prompt. The LLM receives only this ambiguous prompt and the stated test ( $C1$ ).
Hidden Constraints: The constraints removed from the gold prompt were encoded as hidden constraint tests ( $C2–C4$ ). These tests capture the unstated intent (e.g., sorting order, specific data types) that the LLM is not explicitly told to satisfy.
Validation: A leakage audit ensured that stated tests ( $C1$ ) could be satisfied without knowledge of hidden constraints. Canonical solutions were run against the full suite to confirm that hidden constraints were satisfiable by correct code (yielding an IVR of 0 for ground truth).

3. Evaluation Pipeline

The evaluation involved two models: Claude Sonnet 4.6 and OpenAI GPT 4.1.

Generation: 5 solutions were generated per problem at a temperature of 1.0 with a 1024-token limit.
Execution: Solutions were run against stated tests ( $C1$ ) and hidden tests ( $C2–C4$ ) with a 5-second timeout per test.
Analysis: Per-problem IVR was calculated, and the overall IVR was determined as the mean across problems with at least one stated-test-passing solution. Confidence intervals were estimated via bootstrap resampling.

Key Results

High Stated Pass Rates vs. High Intent Violation

Despite high functional correctness on stated tests, both models exhibited significant intent violations:

Claude Sonnet 4.6: Passed stated tests 94.3% of the time but violated intent in 54.5% of qualifying problems (95% CI: 40.4–68.1%).
OpenAI GPT 4.1: Passed stated tests 92.7% of the time but violated intent in 63.5% of qualifying problems (95% CI: 50.0–76.5%).

Systematic, Bimodal Distribution

The distribution of per-problem IVR was found to be bimodal, clustering near 0 (no violations) or 1 (all solutions violate intent).

95.7% of qualifying problems for Sonnet 4.6 and 91.3% for GPT 4.1 fell into these extreme categories.
This indicates that intent violations are systematic and near-deterministic rather than random sampling noise. Models either fully capture the hidden constraints or fail to capture them entirely.

Constraint Hierarchy

Failure rates on hidden constraints declined monotonically from $C2$ to $C4$ . For both models, the first stripped constraint ( $C2$ ) had the highest failure rate (approx. 43–44%), suggesting that the first removed constraint is often the most central to the task's intent.

Significance and Claims

The paper claims that standard pass rates significantly overstate how well LLM-generated code reflects developer intent.

Measurement Gap: Functional correctness metrics are insufficient for deployment because they cannot detect scenarios where code satisfies the test suite but violates implicit assumptions.
Systematic Failure: The bimodal nature of the results suggests that current models do not "guess" intent randomly; they either infer it correctly or ignore it completely, leading to a high risk of silent failures in ambiguous prompts.
Diagnostic Tool: IVR serves as a relative diagnostic to measure the gap between stated specifications and actual intent, highlighting the need for evaluation methods that go beyond simple test pass rates.

Limitations

The authors acknowledge several constraints:

Scope: The benchmark is limited to 49 Python algorithmic problems; results may differ for other languages or task types (e.g., data transformation).
Model Specificity: Evaluation was restricted to Sonnet 4.6 and GPT 4.1 at a single temperature.
Statistical Power: The small sample size ( $N=49$ ) and overlapping confidence intervals mean the difference between the two models' violation rates is not statistically distinguishable.
Subjectivity: The definition of "intent" relies on a single annotator's judgment of the gold prompt, making the metric normative rather than an absolute ground truth.
Data Contamination: As the benchmark is derived from the public HumanEval+ dataset, there is a risk that models encountered the original problems during training, though the authors note the ambiguous prompts were manually constructed to mitigate this.