🤖 AI

NeuroState-Bench: A Human-Calibrated Benchmark for Commitment Integrity in LLM Agent Profiles

NeuroState-Bench is a human-calibrated benchmark that evaluates LLM agent profiles using side-query probes to measure commitment integrity, revealing that task success and integrity often diverge and that integrity rankings are more stable under distractor perturbations than traditional success metrics.

Original authors: Jia Xiao

Published 2026-05-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jia Xiao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a personal assistant to plan a complex, multi-day trip. You give them a list of rules: "Book the hotel with the ocean view," "Don't spend more than $200 on dinner," and "Remember that I'm allergic to peanuts."

The Old Way (Outcome-Only Evaluation):
In the past, we only checked the final result. If the assistant came back with a perfect itinerary that included an ocean-view hotel, a cheap dinner, and no peanuts, we gave them an "A." We didn't care how they got there. Maybe they forgot the peanut allergy halfway through, panicked, and fixed it at the very last second. Maybe they accidentally booked the wrong hotel but swapped it out because they got lucky. As long as the final paper looked good, the job was done.

The New Problem:
The authors of this paper, NeuroState-Bench, argue that this "final grade" system is dangerous. In the real world, we need to know if the assistant consistently remembered the rules throughout the whole process. Did they keep the peanut allergy in mind while ordering lunch? Did they stick to the budget even when tempted by a fancy restaurant? If they forgot the rules in the middle and only fixed it at the end, they might make a mistake on a different trip where they don't get a second chance.

The New Solution: The "Side-Question" Test
The paper introduces a new way to test AI agents (smart computer programs) called NeuroState-Bench. Instead of just waiting for the final answer, this benchmark asks the AI "side questions" along the way.

Think of it like a teacher walking around a classroom during a test.

The Task: "Write an essay about the history of Rome."
The Side Question: "Hey, before you finish, what was the name of the first emperor you mentioned?"

If the student writes a great essay but can't remember the emperor's name when asked, they might have just guessed the ending or copied it from somewhere else. They didn't truly "hold the commitment" to the facts they were writing about.

How the Benchmark Works:

The Setup: The researchers created 144 different "scenarios" (like the trip planning example) with 8 different types of mental challenges (like remembering a rule, ignoring a distraction, or fixing a mistake).
The Probes: For every scenario, they added 306 specific "side questions" (probes) designed to check if the AI is still sticking to the original rules.
The Human Check: Humans reviewed these tests to make sure the questions were fair and the difficulty levels were accurate. They found that humans and the AI agreed very highly on what was "hard" and what was "easy."
The Score: They calculated a new score called HCCIS-CORE. This score measures "Commitment Integrity." It asks: Did the AI stay true to the rules it promised to follow, even when things got confusing?

The Surprising Results:
When they tested 32 different AI models (some small, some very large and powerful), they found something shocking:

The "Best" Student isn't the "Most Honest" Student: The AI that got the most correct final answers was not the one that stayed most consistent with the rules.
Ranking Flip: If you ranked the AI models by their final answers, the top spot went to one model. But if you ranked them by their "Commitment Integrity" (how well they stuck to the rules), that same model dropped down, and a different model took the top spot. In fact, 31 out of 32 models changed their ranking when you switched from grading the "final answer" to grading the "commitment."
Distractions: When the researchers added "distractors" (confusing extra information), the models that were good at getting the right final answer fell apart. However, the models with high "Commitment Integrity" stayed steady. They were better at ignoring the noise and sticking to the plan.

What This Means (According to the Paper):
The paper concludes that getting the right answer at the end is not enough proof that an AI is reliable. An AI can accidentally get the right answer while completely losing track of the rules in the middle.

The authors built this benchmark to be a "stress test" for AI honesty and consistency. They are not claiming to have built a new AI that is smarter; they have built a better ruler to measure if an AI is actually doing what it says it's doing.

What They Did NOT Claim:

They did not claim this is a medical tool or a way to diagnose human brain issues (despite the "Neuro" in the name, it's about AI "mental states," not human biology).
They did not claim their new scoring method is perfect or that it will always predict the future.
They explicitly stated that their "neural" (brain-like) add-ons didn't actually make the score much better, so they decided to stick with the simpler, human-calibrated version.

In a Nutshell:
NeuroState-Bench is a new report card for AI. It stops just looking at the final grade and starts checking the homework notes to see if the student was actually paying attention the whole time. It turns out that the AI with the best final grade often wasn't the one paying the most attention.

Technical Summary: NeuroState-Bench

Problem Statement
Current agent benchmarks predominantly rely on outcome-only evaluation, scoring agents based solely on whether a multi-turn task was eventually solved. This approach under-specifies whether an agent profile preserves the commitments (constraints, bindings, trusted sources, and belief updates) required to solve a task coherently. An agent may arrive at a correct final answer by accident, through later recovery, or by momentarily overwriting the wrong state, thereby masking a lack of operational commitment integrity. Existing benchmarks often fail to isolate whether an agent can maintain benchmark-defined commitments under explicit side-query elicitation, particularly in the presence of distractors or conflicting information.

Methodology
NeuroState-Bench operationalizes "commitment integrity" not as an inference of hidden latent activations, but as a measurable consistency between a task generator's gold commitment structure and the agent's elicited commitments via benchmark-defined side-query probes.

Benchmark Construction: The benchmark is deterministically generated, comprising 144 tasks across eight cognitively motivated failure families (e.g., constraint persistence, entity binding, conflict resolution, stale-state overwrite). Each task includes two variants (clean and distractor) and three difficulty bands (easy, medium, hard), with three deterministic instances per cell.
Side-Query Probes: Every task is paired with specific side-query probes (306 total) that query the agent's state at critical points. These probes are formed from the full task prompt plus a probe prompt. Crucially, probe responses are not appended to the task attempt used for final scoring; they serve as independent diagnostic checks.
Scoring Framework: The primary metric is HCCIS-CORE, a confidence-free, non-neural integrity score. It aggregates probe-level failure risks (state drift, trajectory risk, context-position risk) weighted by human-calibrated difficulty and construct validity.
- Legacy/Exploratory Scores: The paper retains HCCIS-FULL (a legacy heuristic including confidence/uncertainty terms) and HCCIS+N (an exploratory neural-augmented variant) as comparators, but explicitly positions them as secondary to the core score.
Human Calibration: A stratified sampling workflow involving 104 sampled task units and 216 raw annotations was adjudicated to produce a merged reporting scope of 108 task rows. This layer provides difficulty priors and construct-validity verification, achieving high inter-rater reliability (weighted $\kappa = 0.977$ , ICC = 0.977).
Evaluation Grid: The main experiment evaluates a 32-profile grid, combining a fixed 16-profile local subset (small models with various wrappers) and a matched 16-profile hosted large-model subset (including GPT-4.1, GPT-5-Chat, Claude Sonnet, and Qwen variants).

Key Results

Divergence of Success and Integrity: Task success and commitment integrity are empirically separable. In the 32-profile grid, the agent with the highest task success (Qwen3.5-27B memory profile, 0.9391) is not the integrity leader. The integrity leader is the GPT-5-Chat react profile (HCCIS-CORE 0.7772).
Ranking Instability: When integrity replaces task success as the ranking metric, 31 of 32 profiles change rank. The mean absolute rank shift is 6.8125, with a maximum shift of 20.
Stability Under Perturbation: Integrity-based rankings are more stable under distractor perturbation than success-based rankings. The mean clean-versus-distractor rank shift for task success is 4.3438, compared to 2.7500 for HCCIS-CORE.
Diagnostic Performance: HCCIS-CORE achieves an AUC of 0.8469 and PR-AUC of 0.6992 for post-probe discrimination of terminal task failure. While raw probe accuracy and state drift achieve slightly higher ROC-AUC (0.8587) and better calibration (Brier/ECE), HCCIS-CORE maintains higher point-estimate PR-AUC and remains more tightly tied to the benchmark's intended construct.
Neural and Mechanistic Signals: The exploratory neural-augmented variant (HCCIS+N) and mechanistic components (e.g., agreement-related components in small models) do not universally dominate simpler baselines. In fact, HCCIS+N performs worse than HCCIS-CORE on AUC, PR-AUC, Brier, and ECE. The paper treats these neural layers as auxiliary calibration context rather than central predictive claims.
Hidden Failures: Case studies demonstrate that agents can achieve correct final answers while exhibiting total state drift (e.g., HCCIS-CORE of 0.1192 with 100% state drift), a failure mode invisible to endpoint-only metrics.

Significance and Claims
The paper positions NeuroState-Bench as a calibrated evaluation resource and analysis framework rather than a new agent algorithm or a universal predictor of failure. Its primary contribution is methodological: it establishes that endpoint success is insufficient for assessing agent reliability when commitments, provenance, and constraints matter.

Construct Validity: The benchmark demonstrates that operational commitment integrity is a distinct, measurable construct that diverges systematically from task completion.
Evaluation Geometry: It reveals that different evaluation axes (success vs. integrity vs. confidence-sensitive heuristics) induce different ranking geometries and stability profiles.
Modest Scope: The authors explicitly state that the benchmark does not claim to infer unobservable mental states or that neural-augmented scores universally dominate. Instead, it offers a reproducible, human-calibrated "diagnostic stress test" to expose hidden commitment failures. The release includes full auditability, allowing researchers to trace ranking reversals back to specific task prompts, probes, and agent traces.

In summary, NeuroState-Bench contributes a calibrated axis for evaluating the consistency of agent behavior, arguing that integrity-aware evaluation materially changes the interpretation of agent capabilities, particularly in complex, multi-turn environments where distractors and state updates are prevalent.

Technical Summary: NeuroState-Bench

More like this