Sensitivity to Subjective Expected Utility Maximization: A Methodological Study, with an Illustrative Application to LLM Decision-Making

Technical Summary: Sensitivity to Subjective Expected Utility Maximization

Problem Statement
Evaluating the quality of decisions made under uncertainty is notoriously difficult when labeled outcomes are scarce, costly, or confounded by luck. Traditional "label-based" assessment relies on an external key of correct choices or realized outcomes. However, such keys are often impractical to construct (e.g., in insurance triage or novel domains) and fundamentally flawed as measures of decision quality because a good outcome does not guarantee a good decision (outcomes depend on chance, not just the decision process).

This paper proposes a "consistency-based" evaluation framework. Instead of asking if an agent chose the "right" option, it asks if the agent's choices are coherent with a stated standard of rationality: Subjective Expected Utility (SEU) maximization. The goal is to measure the degree to which an agent's choices track the SEU ranking, separating this "sensitivity" from the specific content of the agent's beliefs and utilities, and from the outcomes they happen to realize.

Methodology
The paper develops a methodological pipeline centered on a softmax choice model applied to SEU-valued alternatives.

1. The Model: The core vehicle is a Luce–McFadden softmax rule where the probability of choosing alternative $r$ is proportional to $\exp(\alpha V(r))$. Here, $V(r)$ is the expected utility of the alternative, and $\alpha \ge 0$ is a scalar sensitivity parameter.

   • $\alpha = 0$: Uniform random choice (insensitive to utility).
   • $\alpha \to \infty$: Deterministic SEU maximization (perfectly sensitive).
   • Intermediate $\alpha$: A graded measure of how closely choices track expected utility.

2. Parameterization:

   • Beliefs ($\beta$): Subjective probabilities are generated via a linear-softmax map from feature vectors.
   • Utilities ($\delta$): Utilities are constructed from ordered increments with fixed endpoints ($0$ and $1$) to resolve affine indeterminacy.
   • Sensitivity ($\alpha$): The primary quantity of interest, representing the agent's disposition to act on their commitments.

3. Two Models:

   • Model $m_0$ (Uncertain Choices Only): Agents choose among alternatives with subjective probabilities.
   • Model $m_1$ (Extended with Risky Choices): Agents also choose among alternatives with objective probabilities (lotteries). This allows for a "risk-free" identification of utilities ($\delta$) since objective lotteries do not depend on the belief parameter $\beta$.

4. Computational Validation: The authors implement these models in Stan and validate them using a rigorous Bayesian workflow:

   • Prior Predictive Checks: Ensuring priors allow for a sensible range of behaviors.
   • Parameter Recovery: Simulating data from known parameters to test if the model can recover them.
   • Simulation-Based Calibration (SBC): Verifying that the posterior distributions are calibrated (ranks of true parameters are uniform).

Key Contributions and Findings

1. Identifiability vs. Estimability Gap
The paper provides a formal identifiability analysis that reveals a critical distinction between theoretical identifiability and practical estimability at realistic sample sizes.

• For $\alpha$ (Sensitivity): $\alpha$ is identifiable from the expected utility vector $\eta$ (given non-constant values). In practice, $\alpha$ is recovered with high precision and low bias, even when other parameters are poorly identified.
• For $\delta$ (Utilities): In the uncertain-only model ($m_0$), beliefs ($\beta$) and utilities ($\delta$) are only weakly informed by data; they trade off against each other, resulting in wide, correlated posteriors.
• The Extended Model ($m_1$) Limitation: While adding risky choices theoretically identifies $\delta$ (by breaking the $\beta$-$\delta$ coupling), the paper finds that at realistic sample sizes, this theoretical identifiability yields negligible practical gain. The reduction in confidence interval width for $\delta$ is less than 1%, and RMSE remains unchanged.
• For $\alpha$ Precision: Contrary to the intuition that "better" data types (risky vs. uncertain) improve precision, the study finds that finite-sample precision for $\alpha$ is driven by data quantity, not choice type. Doubling the number of uncertain choices significantly sharpens $\alpha$ estimates, whereas swapping in an equal number of risky choices provides no detectable precision gain.

2. The Marginal-SBC Demarcation
The authors introduce a crucial diagnostic insight: Marginal SBC passes do not imply joint posterior calibration.

• Standard SBC checks (rank uniformity) passed for all parameters ($\alpha, \beta, \delta$) in both models.
• However, the joint posterior for $(\beta, \delta)$ remained weakly contracted and highly correlated (weakly informed).
• Conclusion: A model can be perfectly calibrated marginally (each parameter looks correct in isolation) while the joint distribution remains unresolved. Marginal SBC is blind to weak joint informativeness.

3. Illustrative Application to LLMs
The framework was applied end-to-end to decision-making data from GPT-4o and Claude 3.5 Sonnet across two tasks: insurance-claims triage and Ellsberg-style urn gambles.

• Workflow: The pipeline successfully recovered $\alpha$ for real LLM choices, with diagnostics confirming model adequacy (except for the expected weak inference on nuisance parameters $\beta, \delta$).
• Results:
  • GPT-4o: Showed a structured, monotonic decline in sensitivity ($\alpha$) as sampling temperature increased across both tasks. The effect was statistically robust ($P(\text{slope} < 0) \approx 0.99$).
  • Claude 3.5: Did not show a monotonic pattern in either task.
  • Crucial Caveat: The paper explicitly states that the lack of a pattern for Claude is an inconclusive null, not a proven absence of effect. The study's resolution (minimum detectable effect) was insufficient to rule out a GPT-4o-sized effect for Claude. The instrument "declines to support" an effect rather than certifying its absence.
• Interpretation: The results support a within-design comparative claim (GPT-4o is more sensitive to SEU than Claude under these specific conditions) but do not support absolute claims about which model is "more rational" in a context-free sense.

Significance and Scope
The paper positions itself as a methodological contribution rather than a substantive breakthrough in LLM behavioral science or decision theory.

• Conceptual: It provides a principled explication of "sensitivity to SEU maximization" as a graded, measurable disposition, distinct from beliefs, utilities, and outcomes.
• Methodological: It demonstrates that identifiability analysis is necessary but insufficient for designing experiments; finite-sample precision depends on data quantity and design specifics, not just theoretical identification. It also clarifies the limitations of marginal SBC in diagnosing joint posterior geometry.
• Limitations: The paper explicitly avoids adjudicating normative questions (e.g., whether SEU is the correct standard) or testing for ambiguity aversion (the model absorbs ambiguity into beliefs or noise). It does not claim to measure "rationality" in an absolute sense, only conformity to a stated standard within a specific design.

In summary, the paper offers a transparent, reusable framework for evaluating decision-makers against a rationality standard, rigorously validating the measurement instrument while explicitly reporting its finite-sample caveats and resolution limits.