🤖 AI

Are LLMs Reliable Code Reviewers? Systematic Overcorrection in Requirement Conformance Judgement

This paper reveals that large language models systematically overcorrect by misclassifying correct code as non-compliant with natural language requirements, a flaw exacerbated by detailed prompting, and proposes a fix-guided verification filter to mitigate these reliability issues in automated code review.

Original authors: Haolin Jin, Huaming Chen

Published 2026-03-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Haolin Jin, Huaming Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have hired a very smart, well-read, but slightly paranoid AI Assistant to help you review your code. You give it a recipe (the requirements) and a dish (the code), and ask, "Does this dish match the recipe?"

You expect the AI to be a fair judge. But this paper reveals a shocking truth: The AI is actually a "False Alarm Machine." It is so eager to find mistakes that it frequently rejects perfectly good dishes, claiming they are burnt or undercooked when they are actually delicious.

Here is the breakdown of the paper using simple analogies:

1. The Problem: The "Paranoid Chef"

The researchers tested several top-tier AI models (like GPT-4o, Claude, and others) on a massive set of coding tasks. They gave the AI two types of code:

The Perfect Dish: Code that works exactly as the recipe says.
The Burnt Dish: Code that has obvious bugs.

The Result: The AI was surprisingly bad at distinguishing the two.

The Big Mistake: The AI frequently said "NO" to the Perfect Dish. It rejected correct code, inventing reasons why it didn't work. This is called Over-Correction.
The Surprise: The researchers thought that if they asked the AI to "explain its reasoning" or "suggest a fix," it would get smarter. Instead, it got worse. The more detailed the instructions, the more paranoid the AI became. It started hallucinating problems that didn't exist, like claiming a cake was too sweet because the recipe didn't explicitly say "add sugar."

2. Why Does This Happen? (The "Imaginary Rules")

The paper found that when the AI rejects a correct piece of code, it usually makes up one of four types of fake problems:

The "Ghost Constraint": The AI invents a rule that wasn't in the recipe (e.g., "The recipe didn't say to use a metal spoon, so this wooden spoon is wrong").
The "Edge Case" Obsession: The AI worries about scenarios that will never happen (e.g., "What if the user types a negative number? The recipe didn't say to handle that!").
The "Logic" Confusion: The AI claims the math is wrong without actually doing the math.
The "Boundary" Blunder: It gets confused about simple things like whether a number should be "less than" or "less than or equal to."

The Analogy: Imagine a security guard at a club who is so strict that he denies entry to people with valid tickets because they are wearing the "wrong shade of blue" or because they didn't bring a specific type of ID that the bouncer imagined was required.

3. The "Explanation Trap"

Common sense suggests that if you ask someone to explain why they made a decision, they will think harder and be more accurate.

The Reality: For these AI models, asking for an explanation is like asking a nervous student to justify their answer on a test. They start over-thinking, inventing complex reasons to support their initial (wrong) gut feeling.
The Contradiction: Sometimes, the AI says "NO" (reject the code), but its written explanation actually says, "This code looks fine, but I'm rejecting it anyway." The AI is lying to itself to sound authoritative.

4. The Solution: The "Taste Test" Filter

Since the AI is bad at talking about the code, the researchers proposed a solution that stops the AI from just guessing and forces it to prove it.

They created a "Fix-Guided Verification Filter." Here is how it works:

The AI Rejects: The AI says, "This code is bad," and suggests a "fix" (a patch).
The Reality Check: Instead of trusting the AI's words, the system takes the Original Code and the AI's Fix and runs them through a real computer test (like a taste test).
The Verdict:
- If the Original Code passes the test, but the AI's Fix breaks it (or does the exact same thing), the system knows the AI was wrong. It ignores the AI and says, "Actually, the code is fine."
- If the Original Code fails and the Fix passes, then the AI was right.

The Analogy: Imagine a food critic who says, "This soup is too salty." Instead of taking his word for it, you hand him a spoon. If he can't prove it's salty by tasting it, or if his "fix" (adding water) ruins the soup, you ignore his review. You let the actual taste (the test results) be the final judge.

5. The Takeaway

Don't trust the AI's opinion blindly: Even the smartest AI models are prone to "paranoia" when reviewing code without running tests.
More words $\neq$ Better answers: Asking for long explanations often makes the AI more confident in its mistakes.
Run the code: The only way to be sure is to actually run the code (or the AI's proposed fix) to see if it works.

In short: The AI is a great "idea generator," but a terrible "final judge." To use it safely, we need to treat its suggestions as hypotheses that must be proven by running the code, not just by reading its confident explanations.

1. Problem Statement

The paper addresses a critical gap in the deployment of Large Language Models (LLMs) for automated software engineering: the reliability of LLMs in judging code conformance to natural language requirements without executing test cases.

While LLMs are increasingly used as "virtual reviewers" to verify if code meets specifications, it is unclear if they can accurately distinguish between correct and buggy implementations based solely on text. The authors hypothesize that LLMs suffer from systematic overcorrection bias, where they frequently misclassify correct code as defective (False Negatives), especially when prompted to provide detailed explanations or propose fixes. This phenomenon undermines the trustworthiness of LLM-based review pipelines, potentially causing unnecessary rework or blocking valid code.

2. Methodology

Datasets and Construction

The authors constructed a unified benchmark suite by pairing correct (canonical) and buggy implementations across three widely used datasets:

HumanEval-X-Bugs: Paired solutions from HumanEval.
Buggy-MBPP: Constructed buggy versions of MBPP tasks.
QuixBugs: Paired defective and corrected algorithmic problems.
Total: Over 1,400 instances, each labeled with ground-truth correctness ( $y=1$ for correct, $y=0$ for buggy) and normalized bug types/failure symptoms.

Experimental Setup

Models: Five representative LLMs were evaluated:
- Closed-source: GPT-4o, Claude-4.5-sonnet, Gemini-2.0-flash.
- Open-source: Llama-3.1-8B, Mistral-Small-3.1-24B.
Prompting Strategies: Three distinct modes were tested to isolate the impact of prompt complexity:
1. Direct: Binary "Yes/No" verdict only.
2. Direct+Explain: Verdict + short rationale.
3. Full: Verdict + detailed rationale + proposed code fix (three-step process).

Evaluation Metrics

Confusion Matrix: Calculation of False Negative Rate (FNR, over-correction) and False Positive Rate (FPR, unsafe acceptance).
Rationale Reliability (A1): Self-Consistency measures whether the generated explanation logically supports the verdict (e.g., does a "No" verdict have a negative rationale?).
Fault-Awareness (A2): Evaluates if the rationale correctly identifies the specific bug type and failure symptoms for buggy code.
Taxonomy: A unified taxonomy was created to categorize the reasons for False Negatives (e.g., "Added Requirement," "Logic Error," "Boundary Error").

3. Key Contributions

Discovery of Systematic Overcorrection: The paper reveals that LLMs frequently reject correct code (high FNR), a bias that intensifies significantly when prompts require explanations and fixes.
Prompt-Induced Trade-offs: It demonstrates that increasing prompt complexity does not uniformly improve accuracy but rather shifts the error profile: it reduces False Positives (accepting bugs) at the cost of drastically increasing False Negatives (rejecting good code).
Rationale Analysis: The study shows that generated rationales are often internally inconsistent or "fault-agnostic" (correctly identifying symptoms but misdiagnosing root causes).
Mitigation Framework: The authors propose a Fix-guided Verification Filter, a novel mechanism that treats the model's proposed fix as executable counterfactual evidence to validate the original verdict.

4. Key Results

Judgment Performance & Bias

High False Negatives: Even under minimal prompts, LLMs exhibit high FNR. Under the Full prompt, FNR skyrocketed for some models (e.g., GPT-4o on MBPP rose from 35.9% to 87.9%; Llama-3.1-8B exceeded 91.9%).
The Trade-off: Richer prompts (Full) successfully reduced FPR (unsafe acceptance) in many cases (often to near 0%), but this came with a massive increase in FNR. For example, on MBPP, GPT-4o's False Negatives more than doubled when moving from Direct to Full prompts.
Model Differences:
- GPT-4o: Highly conservative under complex prompts, leading to extreme over-correction.
- Gemini-2.0-flash: Showed a weaker shift toward conservatism but higher baseline FPR.
- Open-source models: Exhibited the highest instability and error rates, particularly Llama-3.1-8B.

Rationale Reliability

Self-Consistency (A1): A significant portion of rationales contradicted their own verdicts. For instance, GPT-4o often output "No" (reject code) but provided rationales that argued the code was correct. Conversely, open-source models often said "Yes" while listing defects.
Fault-Awareness (A2): While models were excellent at describing symptoms (e.g., "runtime error"), they were poor at diagnosing root causes (Bug Match rates were often <60% even when Symptom Match was >95%).

Root Causes of Overcorrection

Analysis of False Negative rationales revealed four dominant patterns accounting for 87.2% of errors:

Logic Error: Claiming algorithmic flaws without evidence (48.2%).
Added Requirement: Hallucinating constraints not in the spec (14.1%).
Boundary Error: Incorrectly flagging off-by-one or boundary issues (13.2%).
Misread Spec: Misinterpreting the requirement text (11.7%).

Mitigation Effectiveness

The Fix-guided Verification Filter significantly reduced overcorrection:

Mechanism: If an LLM rejects code and proposes a fix, the system executes both the original and fixed code against benchmark and augmented tests. If the fix is unnecessary (both pass) or introduces regressions, the verdict is flipped back to "Yes."
Results:
- FNR Reduction: Reduced FNR by 30–67 percentage points across models and benchmarks (e.g., Llama-3.1-8B on MBPP dropped from 90.8% to 23.6%).
- FPR Impact: Caused only a modest increase in FPR (e.g., +1.8% to +2.5%), indicating the filter successfully recovers correct code without significantly accepting buggy code.

5. Significance and Implications

Challenging Common Assumptions: The paper refutes the intuition that "Chain-of-Thought" or detailed prompting (asking for explanations/fixes) inherently improves reasoning accuracy in code review. Instead, it often triggers over-critical behavior.
Operational Impact: In automated CI/CD pipelines, relying on LLMs with complex prompts could lead to a flood of false rejections, stalling development and wasting engineering resources.
New Paradigm for Verification: The study advocates for execution-based verification over purely textual reasoning. By using the model's own proposed fix as a testable hypothesis, the proposed filter grounds the decision in executable evidence rather than hallucinated rationales.
Guidance for Practitioners: Developers should be cautious about using LLMs as "judges" without test cases. If explanations are required, they must be validated against a "fix-guided" filter or similar execution-based checks to mitigate systematic bias.

In conclusion, the paper provides empirical evidence that current LLMs are not yet reliable standalone code reviewers for requirement conformance due to systematic overcorrection, but offers a practical, lightweight framework to mitigate these risks in automated workflows.