🤖 AI

When Agents Disagree With Themselves: Behavioral Consistency as an Uncertainty Signal for LLM Agents

This paper demonstrates that behavioral variance in LLM agents' action sequences serves as a training-free uncertainty signal, enabling selective classification and distribution-free calibration that significantly improves accuracy and model ranking reliability without requiring held-out calibration data.

Original authors: Aman Mehta

Published 2026-07-16

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Aman Mehta

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Digital Double-Check: Why Asking an AI Twice Might Be Better Than Asking Once

Imagine you are trying to solve a tricky puzzle, like a complex riddle or a math problem. If you ask a friend for the answer, they might get it right, or they might guess and get it wrong. But what if you could ask that same friend the exact same question ten times in a row? If they give you the same answer every single time, you probably feel pretty confident they are right. If, however, they give you ten different answers, you know something is up—they are confused, guessing, or the question is too hard for them.

This is the core idea behind Large Language Models (LLMs), which are the super-smart computer programs powering today's AI agents. These agents are like digital detectives that can use tools (like searching the web or writing code) to solve multi-step problems. But just like humans, they aren't perfect. Sometimes they hallucinate (make things up) or get stuck in a loop. Scientists have long wondered: How can we tell when an AI is unsure of itself without just asking it, "Are you sure?" (Because AI can lie about being sure). This paper explores a clever trick: instead of trusting a single answer, we watch how the AI behaves when it tries the same task over and over again. If the AI's behavior is all over the place, it's a giant red flag that the answer might be wrong.

When AI Agents Can't Decide on Themselves

The researchers behind this study decided to test a simple but powerful idea: Behavioral Consistency. They treated the AI agent like a student taking a test. Instead of just grading the final answer, they watched how the student solved the problem. They ran the same AI agent on the exact same questions 10 times each, using different "random seeds" (which is like shuffling the deck of cards the AI draws from to make its decisions).

Here is what they found, and it's a bit surprising:

1. The AI is a Fickle Thinker
Even when you give an AI the exact same instructions and the same starting point, it doesn't always think the same way. In their tests, running the same agent 10 times produced between 2.3 and 4.2 different paths to the solution. Sometimes the AI would take a shortcut; other times, it would wander down a long, confusing alleyway. This "wandering" isn't just a glitch; it's a signal. The more the AI changes its mind about how to solve a problem, the less likely it is to get the right answer.

2. Consistency is a Crystal Ball for Accuracy
The team discovered a strong link between how consistent the AI is and how correct it is.

The "Steady" Group: When the AI stuck to the same path (2 or fewer unique paths), it got the answer right 82% to 87% of the time.
The "Wandering" Group: When the AI went in many different directions (4 or more unique paths), its accuracy dropped to between 41% and 65%.

This gap was huge. It means that if you see an AI taking a very different route every time you ask it a question, you should probably trust its answer less.

3. The "Step 2" Surprise
The researchers looked closely at when the AI started to get confused. They found that for some models (like Llama), the trouble started almost immediately. About 50.5% of the time, the AI would take a different first step or second step compared to its other attempts. It's as if the AI is standing at a fork in the road and can't decide which way to go right from the start. For other models (like Claude), they stayed on the same path much longer before diverging.

4. The "Filter" vs. The "Voter"
One of the most interesting findings was about how to use this information.

The "Voter" Approach: You might think, "If I ask the AI 3 times and take the majority vote, I'll get a better answer." The paper says this works, but only a little bit (improving accuracy by just 0–2%). Why? Because sometimes the AI is confidently wrong. If it makes a mistake early on, it might make the same mistake every single time, so the "vote" just confirms the error.
The "Filter" Approach: Instead of voting, the researchers tried a different strategy: Only answer if the AI agrees with itself. They said, "If the AI runs the task 3 times and all 3 runs give the exact same answer, then we accept it. If they disagree, we say 'I don't know' and skip the question."
- This "filter" worked amazingly well. By skipping the uncertain questions, the AI's accuracy on the questions it did answer jumped to 87–88%. That is a 6–14 percentage point improvement over just taking a single guess!

5. Better Than "Asking" the AI
The team also compared this "behavioral consistency" trick to asking the AI, "How confident are you?" (This is called "verbalized confidence"). The results were clear: the AI's actions were a much better lie detector than its words. The AI often said it was confident even when it was wandering around in circles. The behavioral signal (watching if it changes its mind) was a much more reliable way to spot a mistake.

6. It Works Everywhere
To make sure this wasn't just a fluke for one type of question, they tested the AI on a completely different kind of task: fixing bugs in computer code (using a benchmark called SWE-bench). Even though coding is very different from answering trivia questions, the same rule applied. The AI that was most consistent in its behavior was also the most accurate. The "consistency hierarchy" (which models were steady and which were wobbly) stayed exactly the same across both tests.

Why This Matters

The paper suggests that we don't need to change the AI's brain or teach it new things to make it more reliable. We just need to watch it. By running a task a few times and checking if the AI's "story" stays the same, we can act like a smart editor. We can choose to trust the answers that are consistent and ignore the ones where the AI seems confused.

This is a "training-free" method, meaning it works with any AI model right out of the box. It turns the AI's own uncertainty into a useful signal. Instead of blindly trusting a single answer, we can use this "consistency check" to know when to say, "I'm not sure about this one," which is a huge step forward for making AI agents safer and more trustworthy in the real world.

In short: If an AI can't decide on a plan for itself, it's probably not the best time to trust its answer. But if it sticks to the same plan every time, you can bet it's on the right track.

Technical Summary: Behavioral Consistency as an Uncertainty Signal for LLM Agents

Problem Statement

As Large Language Model (LLM) agents transition from prototypes to production systems, understanding their reliability is critical. A fundamental but understudied question is the behavioral consistency of these agents: given identical inputs, will an agent follow the same reasoning path and arrive at the same answer? Inconsistency complicates debugging, obscures the detection of uncertainty, and challenges standard benchmark evaluation methods. While existing work has quantified agent inconsistency (e.g., $\tau$ -bench), there is a lack of systematic analysis regarding where and why divergence occurs, and whether this variance can serve as an actionable, training-free uncertainty signal for selective prediction and failure detection.

Methodology

The authors present a systematic empirical study of behavioral consistency in ReAct-style agents across four models (Llama 3.1 70B, GPT-5, Claude Sonnet 4.5, and Gemini 3 Pro) and two benchmarks (HotpotQA and SWE-bench).

Experimental Setup: The study involved 8,000 total runs (200 HotpotQA questions $\times$ 4 models $\times$ 10 independent runs each) and 1,000 runs on SWE-bench (50 tasks $\times$ 5 runs). Runs were executed at temperature 0.7 (with specific adjustments for Gemini) to ensure stochastic sampling.
Metrics: The core metric is behavioral consistency, defined by the number of unique action sequences (trajectories) generated across $k$ runs. Additional metrics include step variance ratios, the point of first divergence, and answer correctness (via fuzzy string matching and token F1).
Statistical Frameworks: The paper instantiates three statistical frameworks using cross-run agreement as a single, black-box signal:
1. Selective Classification: Using agreement scores to filter uncertain inputs.
2. Distribution-Free Calibration: Validating risk-coverage curves without held-out calibration sets.
3. Sequential Monitoring: Analyzing step-level agreement to detect divergence early.
Theoretical Basis: The authors formalize the agreement score $s(x)$ as the fraction of runs producing the plurality answer. They prove a Hoeffding-style concentration bound (Proposition 1) showing that the empirical agreement score converges to the true plurality probability, assuming independent and identically distributed (i.i.d.) runs.

Key Contributions

1. Filter > Aggregator Asymmetry

The paper identifies a critical asymmetry in multi-step agents compared to single-turn Chain-of-Thought (CoT). While majority voting (aggregation) improves single-turn CoT accuracy by 5–17 percentage points (pp), it yields only +0–2pp for multi-step agents. This is because agentic errors are often systematic; an early wrong action commits the agent to an incorrect trajectory, resulting in "consistent-wrong" answers that voting cannot correct. Conversely, filtering based on disagreement (abstaining when runs do not agree) yields significant gains of 6–14pp for three of the four models.

2. Selective Prediction with Distribution-Free Validation

The authors demonstrate that behavioral consistency functions as a valid selection function within the Geifman–El-Yaniv selective classification framework. By answering only when $k=3$ runs agree, the system achieves 87–88% accuracy at 54–62% coverage. This approach matches the performance of split-conformal baselines (which require a held-out calibration set) without needing any separate calibration data, as the $k$ runs used for prediction inherently provide the signal.

3. Behavioral Consistency > Verbalized Confidence

In a head-to-head comparison on a common 100-question split, behavioral consistency (AUROC 0.65–0.74) significantly outperforms verbalized self-confidence (AUROC 0.48–0.55) as a failure detector across Claude, GPT-5, and Llama. Combining the two signals did not improve performance over behavioral consistency alone. Crucially, behavioral consistency requires no prompt engineering, no access to log-probabilities, and no model modification.

4. Cross-Benchmark Stability and Ranking Instability

The consistency hierarchy (Claude < GPT-5 < Gemini < Llama in terms of variance) is preserved exactly across HotpotQA (QA tasks) and SWE-bench (coding tasks), suggesting consistency is a stable, benchmark-independent model property. Furthermore, the paper reveals that single-run evaluations misrank models 29.3% of the time due to variance, undermining standard benchmark methodologies.

5. Divergence Mechanism

Divergence concentrates early in the reasoning process. For Llama, 50.5% of tasks diverge by step 2, compared to only 4–6% for Claude and GPT-5. The first search query largely determines the trajectory. Additionally, path length is strongly correlated with behavioral variance; longer paths indicate backtracking and increased opportunities for error.

Results

Consistency vs. Correctness: Consistent tasks (≤2 unique sequences) achieve 82–87% accuracy, while inconsistent tasks (≥4 sequences) drop to 41–65%. This gap persists even after controlling for task difficulty, though "consistent-wrong" tasks (where the agent confidently produces a wrong answer) set a hard ceiling on filtering gains (5.5–10% of tasks).
Failure Detection: Consistency metrics (e.g., unique sequence count >4) achieve AUROC scores of 0.62–0.78 for detecting failures, substantially outperforming random baselines.
Calibration: While behavioral consistency is a useful uncertainty signal, it is not perfectly calibrated (Expected Calibration Error ranges from 0.09 to 0.17), indicating that agreement levels do not perfectly map to correctness probabilities.
SWE-bench Validation: The consistency hierarchy holds across unrestricted bash coding tasks, with Llama showing the highest variance (CV 68.9%) and Claude the lowest (CV 17.7%).

Significance and Claims

The paper claims that behavioral consistency is a training-free, black-box uncertainty signal that emerges naturally from repeated execution. Its primary significance lies in enabling selective prediction (abstaining on uncertain inputs) and runtime failure detection without requiring model modification or internal state access.

The authors argue that this signal addresses a gap in agent reliability: unlike verbalized confidence or learned calibration, it is robust and requires no fine-tuning. By treating each independent run as an implicit ensemble member, the work connects agent behavior to established statistical frameworks for uncertainty estimation. The findings suggest that for high-stakes deployments, relying on single-run evaluations is insufficient; instead, monitoring cross-run agreement provides a reliable mechanism to filter out errors and improve system trustworthiness, albeit at the cost of reduced throughput (coverage). The paper concludes that consistency is not merely a reliability metric but a fundamental property that can be exploited to make agentic systems safer and more predictable.