🤖 AI

Mind the Gap: How Elicitation Protocols Shape the Stated-Revealed Preference Gap in Language Models

This paper demonstrates that the stated-revealed preference gap in language models is highly dependent on elicitation protocols, showing that while allowing neutrality in stated preferences improves correlation with forced-choice revealed preferences, introducing abstention in revealed preferences or using system prompt steering fails to reliably resolve the mismatch.

Original authors: Pranav Mahajan, Ihor Kendiukhov, Syed Hussain, Lydia Nottingham

Published 2026-05-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Pranav Mahajan, Ihor Kendiukhov, Syed Hussain, Lydia Nottingham

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand the personality of a new friend. You have two ways to do this:

Ask them directly: "Do you value honesty more than kindness?" (This is their Stated Preference).
Watch them act: Put them in a tough situation where they have to choose between being honest or being kind, and see what they actually do (This is their Revealed Preference).

In the world of AI, researchers have noticed a problem: sometimes an AI says it loves "Honesty," but when put in a tricky story, it chooses "Kindness" instead. This mismatch is called the Stated-Revealed (SvR) Gap.

This paper, titled "Mind the Gap," investigates why this gap exists and, more importantly, how the way we ask the questions changes the answer. Think of the "elicitation protocol" as the rules of the game we play with the AI.

Here is what the researchers found, using simple analogies:

1. The "Forced Choice" Trap (The Old Way)

Previously, researchers played a game where the AI was forced to pick a side. Imagine a referee shouting, "You must choose: A or B! No talking, no hesitation!"

The Problem: If the AI is actually unsure, or thinks "It depends on the situation," it still has to pick A or B. It's like forcing a person to choose between "Pizza" and "Salad" even if they are full and don't want either.
The Result: The AI picks randomly or based on the order of the words, creating "fake" preferences. When researchers compared these fake choices to what the AI said it valued, the connection was weak and messy.

2. The "Opt-Out" Button (The New Way for Stated Preferences)

The researchers tried a new rule for the "Ask them directly" part. They said, "You can pick A, B, or you can say 'I don't care' or 'It depends'."

The Analogy: Imagine asking a friend, "Do you prefer coffee or tea?" but allowing them to say, "I'm not a drinker," or "It depends on the time of day."
The Result: This acted like a filter. It removed the "weak signals" (the AI's guesses or random choices). When the researchers only looked at the times the AI strongly picked a side, their stated preferences matched their actual behavior much better. The "gap" shrank significantly.

3. The "Too Many Opt-Outs" Problem (The New Way for Revealed Preferences)

Then, they tried letting the AI say "It depends" during the tough scenarios (the revealed preferences).

The Analogy: Now, imagine putting your friend in a real-life crisis and saying, "You can choose Action A, Action B, or just say 'I don't know'."
The Result: The AI started saying "It depends" or "Equal" almost all the time! It was like a student who, when given a hard test, just writes "I'm not sure" on every answer.
The Consequence: Because the AI refused to make a firm choice so often, the researchers couldn't build a clear ranking of what the AI actually valued. The connection between what they said and what they did vanished (dropped to near zero or even negative).

4. The "Instruction Manual" Experiment (Prompt Steering)

Finally, the researchers tried to "fix" the AI's behavior by giving it a cheat sheet. Before the tough scenarios, they told the AI: "Here is your own list of values you said you liked. Please follow this list strictly."

The Analogy: It's like telling a driver, "Remember, you said you love safety, so please drive safely!" right before a race.
The Result: It didn't work reliably. For some AIs, it helped a little. For others (like the "Claude" family), it actually made things worse. The AI seemed to ignore the cheat sheet or get confused by it. The researchers found that this "instruction manual" trick works for small lists of values but fails miserably when the list is long (16 different values).

The Big Takeaway

The main lesson of this paper is that how you ask the question changes the answer.

If you force an AI to choose when it's unsure, you get noisy, fake data.
If you let an AI say "I don't know" too much, you get no data at all.
Simply telling an AI to "follow its own rules" doesn't reliably fix the problem.

The authors conclude that to truly understand AI values, we need methods that acknowledge that sometimes, the AI (just like a human) genuinely doesn't have a clear preference, and we need to design our tests to handle that uncertainty rather than forcing a choice or ignoring it.

Technical Summary: Mind the Gap: How Elicitation Protocols Shape the Stated-Revealed Preference Gap in Language Models

1. Problem Statement

Recent literature has identified a stated–revealed (SvR) preference gap in large language models (LLMs): a discrepancy between the values models explicitly endorse in abstract settings and the choices they make in contextualized scenarios (Gu et al., 2025; Liu et al., 2025; Chiu et al., 2025). Existing evaluations of this gap rely heavily on binary forced-choice prompting. This methodology collapses preference strength, indifference, and uncertainty into a single binary outcome, potentially conflating genuine preferences with artifacts of the elicitation protocol (Khan et al., 2025; Balepur et al., 2025). The central problem addressed is whether the measured SvR correlation reflects true model alignment or is merely a byproduct of forcing models to make decisive choices when they may hold indeterminate preferences.

2. Methodology

The authors systematically evaluate how elicitation protocols affect SvR correlation across 24 language models. The study builds upon the LitmusValues framework and the AIRiskDilemmas dataset (Chiu et al., 2025), extending the methodology to explicitly allow for neutrality and abstention.

Elicitation Protocols:
- Forced-Choice: Models must select one of two values or actions.
- Expanded-Choice: Models may additionally respond with "Equal Preference" or "Depends", allowing expression of indifference or contextual uncertainty.
Elicitation Settings:
- Stated Preferences: Elicited via abstract pairwise value comparisons (16 Shared AI Values).
- Revealed Preferences: Elicited using 3,000 contextualized moral dilemmas from AIRiskDilemmas.
Ranking Construction:
- Stated rankings are derived from win rates over decisive binary comparisons.
- Revealed rankings use Elo ratings (converted to a 1–16 scale) based on wins/losses across dilemmas.
- Crucial Methodological Choice: Neutral responses (Equal/Depends) are excluded from the ranking construction to isolate strict directional priorities. This exclusion is necessary to create dense rankings required for Spearman's rank correlation but has significant implications for the results.
Intervention: The study tests system prompt steering, where a model's own stated value hierarchy (derived from expanded-choice elicitation) is prepended to the system prompt during revealed preference elicitation to see if it reduces the SvR gap.
Metrics: SvR correlation is measured using Spearman's rank correlation ( $\rho$ ) between the stated and revealed 1–16 rankings.

3. Key Results

A. Neutrality Rates are High and Variable
Under expanded-choice elicitation, neutrality rates (responses of "Equal Preference" or "Depends") vary significantly by model family and elicitation mode.

In stated preference elicitation, neutrality ranges from 48.2% to 100% (e.g., Qwen-3-8B predominantly selects "Depends").
In revealed preference elicitation, some models (e.g., Mistral-3-8B variants) select neutral responses in nearly all scenarios, while others (e.g., Gemma-3-4B) select "Equal Preference" in ~70% of cases.
This indicates that forced binary comparisons frequently mask underlying uncertainty or indifference.

B. Impact of Protocol on SvR Correlation
The study evaluates three configurations:

Forced–Forced (Baseline): Replicating Chiu et al. (2025), this shows substantial cross-model variance in SvR correlation, indicating sensitivity to model-specific decision patterns.
Expanded-Stated / Forced-Revealed: Allowing neutrality only in stated preferences (filtering out weak comparisons) produces a marked increase in SvR correlation. For example, LLaMA-3.1-405B-Instruct improves from $\rho \approx 0.2$ to $\rho \approx 0.7$ . Under this condition, SvR correlation is positively associated with model capability (Epoch Capabilities Index).
Expanded–Expanded: Allowing neutrality in both stated and revealed preferences causes SvR correlation to drop to near-zero or negative values for many models. This occurs because high neutrality rates in revealed preferences prevent the construction of a dense, stable ranking; the residual binary choices provide too weak a signal for correlation-based comparison.

C. System Prompt Steering is Unreliable
Applying system prompt steering (injecting the model's stated hierarchy into the context window) during revealed preference elicitation does not reliably improve SvR correlation on the 16-value domain.

Effects are inconsistent: while a small subset of models shows modest improvement, many (including the Claude family) exhibit reduced correlation after steering.
This aligns with prior findings that prompt steering is more effective for small value sets (e.g., 3 values) than larger ones (Liu et al., 2025). With 16 values, steering often introduces interference that degrades decision consistency.

4. Key Contributions

Protocol Sensitivity: Demonstrates that SvR correlation is highly protocol-dependent. The choice to allow or disallow neutrality fundamentally alters the measured alignment.
Filtering Weak Signals: Shows that allowing models to express neutrality in stated preferences filters out indeterminate comparisons, yielding rankings that correlate more strongly with forced-choice revealed behavior.
Limitations of Steering: Provides evidence that simple system prompt steering is insufficient to bridge the SvR gap in large value domains (16 values) and may be detrimental.
Neutrality as a Feature: Highlights that high neutrality rates in revealed preferences are not merely noise but indicate that many models lack a clean total ordering over values in contextualized scenarios.

5. Significance and Claims

The paper concludes that the SvR correlation is not a stable intrinsic property of a model but is heavily shaped by the elicitation protocol.

Methodological Implication: Preference evaluation must explicitly account for neutrality and indeterminacy rather than forcing binary choices that may create artificial decisiveness.
Alignment Implication: Bridging the SvR gap likely requires interventions stronger than stated-value system-prompting, especially when many values are in play.
Interpretation of Neutrality: The high rates of "Depends" and "Equal Preference" in revealed scenarios suggest that many models do not possess a rigid, context-independent value hierarchy, challenging the assumption of a single, stable "value system" that can be easily steered.

The authors emphasize that their findings suggest a need for evaluation methods that do not discard neutrality but rather model it, as forcing a total ordering on indeterminate preferences may yield misleading metrics of alignment.