💻 computer science

When Policies Change Probabilities: Modular Decision-Making for LLM Code Review

This paper demonstrates that current LLM code reviewers conflate risk estimation with decision policies, causing reported probabilities to shift with cost assumptions, and proposes a modular pipeline that separates evidence-based risk elicitation from cost-driven actions to significantly improve accuracy and reduce decision loss.

Original authors: Rasvik Kudum, Max Corbett, Hitansh Paliwal, Romaisa Fatima, Thomas Jiralerspong, Sneheel Sarangi

Published 2026-08-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Rasvik Kudum, Max Corbett, Hitansh Paliwal, Romaisa Fatima, Thomas Jiralerspong, Sneheel Sarangi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a team of expert detectives to solve mysteries. You give them a clue, and they tell you, "There is a 20% chance this suspect is guilty." Now, imagine the rules of the game change. Suddenly, if you wrongly accuse an innocent person, it costs the city ten times more than if you let a guilty person go free. A smart detective should keep their 20% estimate the same—the suspect and the clue haven't changed—but they should change their action. They should be much more careful about arresting anyone now.

This is the world of Large Language Models (LLMs) acting as code reviewers. These AI "detectives" look at new computer code (patches) and try to guess if it will break things. For a long time, we hoped these AIs would act like perfect scientists: giving a stable probability of failure based only on the evidence, and letting a separate human (or computer) decide what to do based on how much a mistake would cost. But what if the AI's "probability" isn't a stable fact at all? What if the AI changes its mind about the numbers just because you told it the rules of the game changed? That is the big question this paper asks.

The Great AI Mood Swing

The researchers set up a massive experiment to see if AI code reviewers are actually reliable scientists or just mood-swinging actors. They gathered 720 pieces of code—half of which were known to work perfectly and half of which were known to fail. They then asked four different top-tier AI reviewers to look at these same pieces of code under different "cost" scenarios.

In one scenario, the AI was told: "It's equally bad to approve a broken patch or reject a good one." In another, the AI was told: "Oh no! If you approve a broken patch, it costs us 10 times more than if you reject a good one!"

Here is the shocking part: The AI changed its probability numbers. When the rules changed to make mistakes super expensive, the AI didn't just change its decision; it actually changed its estimate of how likely the code was to fail. On average, the reported failure probability shifted by about 13.6% to 16.9% just because the cost rules changed. It's as if the detective looked at the same clue and suddenly said, "Wait, I think the suspect is 15% more likely to be guilty now," even though the suspect hadn't moved an inch.

The "Bad Actor" Problem

The paper found that when these AIs were asked to make decisions under the "high cost" rules, they performed terribly. In fact, for every single AI tested, the decisions they made were worse than if you had just rejected every single patch automatically. It's like a security guard who, when told "don't let anyone in unless you're 100% sure," starts locking out the entire building, including the CEO.

Even worse, the researchers discovered that the AI's "probability" was the culprit. When they took the probability numbers the AI gave under the "safe" rules and applied the strict "high cost" decision logic to them, the system performed much better. This proves that the problem wasn't that the AI couldn't make a good decision; the problem was that the AI was lying about the numbers when it knew the stakes were high. It was letting the pressure of the rules warp its perception of reality.

The Modular Fix: A Team of Specialists

So, how do we fix a detective who changes their mind based on the rules? The researchers tried a new approach: Modular Decision-Making. Instead of asking one AI to do everything (look at the code, guess the risk, and decide what to do), they split the job up.

The Risk Reporter: One AI looks at the code and says, "Here is the risk," without knowing anything about the costs or the rules.
The Monitor: A second, independent AI gives a separate score on how risky the code is.
The Controller: A simple computer program (code) takes those two scores and applies the cost rules to make the final decision.

This "team of specialists" worked much better. When the costs were equal, this modular system was more accurate and made fewer mistakes than the single AI trying to do it all. However, the paper also found a limit: when the cost of a mistake became extremely high (10 times higher), even this smart modular system decided to reject everything. It turned out that the available tools just weren't good enough to safely approve anything when the penalty for failure was that severe.

The Takeaway

The main lesson here is that we can't trust an AI's "probability" if that AI knows what the consequences of its answer will be. If you want a reliable number, you have to ask for it in a vacuum, without telling the AI how much a mistake will cost. Then, you take that number and apply the rules yourself.

The paper shows that when we let the AI mix the "risk estimate" with the "decision policy," the numbers get messy and the decisions get worse. By separating the roles—having one part of the system just report the facts and another part handle the rules—we can build safer, more reliable code review systems. But, as the researchers found, even the best systems have their limits, and when the stakes are incredibly high, sometimes the safest choice is to just say "no" to everything.

Technical Summary: When Policies Change Probabilities

Problem Statement

Large Language Model (LLM) based code review systems increasingly rely on models to estimate patch risk (failure probability) and make approval decisions within a single prompt. This design assumes that a reported probability describes the patch and its evidence, independent of the downstream decision policy (e.g., the cost of false acceptance vs. false rejection).

The paper investigates a critical systems question: When the patch and evidence are fixed, can a downstream decision policy change the numeric failure probability itself? If a policy block (defining costs and thresholds) alters the reported probability, the score becomes "policy-dependent," rendering it unstable for reuse, calibration, or comparison across different organizational error costs.

Methodology

Experimental Setup

The study evaluates four deployed reviewer interfaces (DeepSeek-V4-Pro, Grok-4.3, Mistral-medium-3.5, and GPT-5.5/Codex) using 15,792 responses on 720 candidate patches.

Data Source: 360 repository issues from SWE-rebench. For each issue, the dataset includes exactly one patch that passed and one that failed an archived executable test harness, creating a balanced candidate-level failure rate of 50%.
Monitor: A separate model (Gemini 3.5 Flash) provides an independent continuous failure score for each patch.

System Designs Evaluated

One-Prompt Reviewer (Joint): The LLM receives the patch, code context, monitor score, and a policy block (defining costs and thresholds) in a single prompt, outputting both a probability and an action.
Modular Pipeline (Separated):
- Risk-Only: The reviewer reports a failure probability without any policy information (costs, thresholds, or priors).
- Monitor: An independent model supplies a failure score.
- Controller: Code combines the reviewer and monitor scores and applies the decision rule (thresholding) based on the specific cost policy.

Prompt Conditions

The study uses matched pairs of calls where the patch, context, and monitor evidence are identical, but the policy block varies:

Equal-Cost (Condition A): False-accept cost = 1, Threshold = 0.5.
Cost-10 (Condition C): False-accept cost = 10, Threshold ≈ 0.091.
Low-Prior (Condition B): Assumed failure rate changed to 0.10 (used as a positive control to verify sensitivity to evidence priors).
Cost-20 (Condition D): False-accept cost = 20.

Metrics

Brier Score: Mean squared error of probability forecasts (lower is better).
Decision Loss: Calculated using the costs printed in the prompt. Accepting a failing patch costs $C_{FA}$ ; rejecting a passing patch costs $C_{FR}=1$ .
Policy Dependence: Measured by the change in reported probability when only the policy block changes, compared against same-prompt repeat variation.

Key Results

1. Policy Prompts Change Reported Risk

Replacing an equal-cost policy block with a high-cost (10:1) block significantly altered the reported failure probabilities, even though the patch and evidence remained fixed.

Magnitude: The average change in reported probability was 13.6–16.9 percentage points across the four systems.
Stability: For three of the four systems (Grok, Mistral, GPT/Codex), this policy-induced movement exceeded the variation observed in same-prompt repeat calls.
Direction: The change was not uniform; some models lowered their risk estimates, while others (Grok) slightly increased them, indicating heterogeneous behavioral shifts rather than a single conservative bias.
Accuracy: The cost-10 prompt worsened the average Brier score by 0.0323 compared to the equal-cost prompt.

2. Decision Impact and Excess Loss

Model Actions: For every reviewer, the actions returned directly by the high-cost (Cost-10) prompt resulted in higher decision loss than simply rejecting all patches.
Source of Loss: When the same high-cost rule was applied in code to probabilities elicited under equal costs, the decision loss decreased for all four systems. This demonstrates that the act of eliciting the probability under a high-cost prompt contributes to the excess loss, independent of the model's ability to apply the threshold.
Threshold Adherence: While Grok, Mistral, and GPT/Codex largely followed the deterministic thresholding of their own reported probabilities (88–98% agreement), the probabilities themselves were degraded by the policy context.

3. Modular Pipeline Performance

Equal Costs: The modular pipeline (Risk-Only Reviewer + Monitor + Code Controller) improved average probability accuracy and reduced mean decision loss by 0.073 per issue compared to calibrated reviewer-only scores, accepting 58–68% of patches.
High Costs (10:1): At a 10:1 false-accept penalty, the modular pipeline accepted 0% of patches, matching the "reject-all" baseline. This indicates that while the modular design prevents policy-induced probability distortion, the available scores (even combined) are not accurate enough to support non-trivial approvals under extreme cost constraints.
Monitor Role: The monitor largely substituted for weaker reviewers (DeepSeek, Grok, Mistral) but provided only a marginal, uncertain gain for the strongest reviewer (GPT/Codex).

Significance and Claims

The paper argues that risk estimation and decision-making should be evaluated separately. The primary contribution is the demonstration that "downstream policy can change the probability it is meant to use."

Interface vs. Capability: The study distinguishes between an interface problem (policy-dependent probabilities that change when costs change) and a capability limit (scores that are not accurate enough to support approvals under high costs). The modular design solves the interface problem but exposes the capability limit.
Evaluation Protocol: The authors propose a three-part evaluation for reviewer systems:
1. Hold case and evidence fixed, vary downstream policy, and compare probability movement against repeat calls.
2. Report error for reviewer-only, monitor-only, and combined scores using matched evidence representations.
3. Apply decision rules outside the model and report loss across the relevant cost range.
Design Implications: A policy-dependent score is difficult to use as a software interface because it cannot be reliably cached, recalibrated, or consumed by teams with different cost structures. Separating the stages (risk elicitation, evidence combination, and cost application) creates a clearer record of where changes originate.

The authors explicitly state that these findings concern four fixed reviewer interfaces and an archived harness outcome; they do not imply universal behavior across all models or that separate calls are the only viable design, but rather that any design must pass the test of policy-independence to be robust.