Technical Summary: BayesBench: Evaluating LLM Belief Trajectories Under Multi-Turn Evidence Accumulation

Problem Statement

Large Language Models (LLMs) are increasingly deployed in multi-turn conversational settings where evidence arrives sequentially. Rational decision-making in such environments requires the agent to infer unobserved quantities (latent variables) governing the environment and update beliefs about them as new evidence accumulates. However, standard evaluations typically collapse this process into a single-turn format, providing all information at once and scoring only the final answer. This approach leaves the dynamic process of belief updating unexamined. There is a lack of controlled benchmarks that directly measure how closely an LLM's belief trajectory matches that of a rational Bayesian reasoner as evidence accumulates over multiple turns.

Methodology

The authors introduce BayesBench, a benchmark suite designed to probe multi-turn belief updating across three progressively complex Bayesian tasks instantiated in four environments. The evaluation framework distinguishes between latent inference (estimating hidden quantities) and downstream prediction (using those estimates to forecast outcomes).

Experimental Setup

• Tasks:
  1. Bayesian Estimation: Inferring an unknown parameter from sequential i.i.d. evidence.
  2. Bayesian Prediction: Inferring a latent variable to guide a downstream prediction.
  3. Latent-Framed Bayesian Prediction: Observing evidence filtered through a user-persona framing (communication style), requiring joint inference over the latent state and the framing.
• Environments:
  1. Coin Flip (Task 1): Estimating the bias $\Theta$ of a coin from a sequence of flips. A closed-form Bayesian posterior is available for comparison.
  2. Recommender System (Task 2): Inferring a user's latent "type" from a sequence of movie ratings to predict a rating for a held-out movie. A closed-form Bayesian posterior is available.
  3. Social Judgment (Task 3): Predicting a Reddit community verdict (YTA/NTA) based on a social conflict post. The post is decomposed into storyboard segments and presented in different communication styles (Neutral, Conceding, Defending). No closed-form posterior exists; evaluation relies on invariance checks and behavioral audits.
  4. Medical Triage (Task 3): Assessing clinical urgency (Emergency, Urgent, Observation, Routine) based on patient symptom reports. The reports are styled by communication profiles (Accurate, Hypochondriac, Minimizer, Cyberchondriac). No closed-form posterior exists.
• Models: Seven open-weight instruction-tuned LLMs ranging from 3B to 70B parameters (LLaMA 3 and Qwen 2.5 families).
• Delivery Formats:
  • Passive Observation: Evidence is presented sequentially with neutral acknowledgments.
  • Active Engagement: A simulated user (powered by an LLM) interacts with the model, revealing evidence in a conversational style dictated by a specific persona.
• Metrics:
  • Total Variation Distance (TVD): Measures the divergence between the model's elicited distribution and the closed-form Bayesian reference (for Tasks 1 & 2).
  • Mean Absolute Error (MAE): Measures the error between the model's posterior mean and the reference/ground truth.
  • Behavioral Audits: In Task 3, a third-party grader audits model responses for specific behaviors (e.g., sycophancy, counter-concession) relative to the evidence.

Key Contributions

1. BayesBench Suite: A novel benchmark suite organized along three structured Bayesian tasks, instantiated in four environments. Two environments provide tractable, closed-form Bayesian references, while the others utilize multi-turn user-simulation pipelines to control delivery formats and communication styles.
2. Empirical Evaluation of Belief Trajectories: An evaluation of seven LLMs across these environments, identifying recurring dynamics in how models accumulate, calibrate, and utilize evidence over multiple turns.
3. Identification of the Latent-Prediction Gap: The demonstration that intermediate belief trajectories expose departures from rational belief updating that final-turn accuracy alone can miss. Specifically, the paper highlights a gap between a model's ability to infer latent structure and its ability to use that inference to rationally update beliefs about target outcomes.

Key Results

1. Latent Inference Improves with Scale

Models demonstrate the ability to infer latent structure from accumulating evidence, and this capability strengthens with model size.

• Coin Flip: Larger models separate coin biases more clearly than smaller models, though they often overshoot the Bayesian posterior, becoming overconfident (pushing predictions toward 0 or 1) even when the reference remains moderate.
• Recommender System: Larger models recover user types more accurately from rating patterns alone, even when semantic features (movie titles/genres) are anonymized.
• Medical Triage: Larger models become more confident in inferring patient communication styles (e.g., distinguishing Hypochondriac from Minimizer) as evidence accumulates, reaching ~0.80 probability on the true profile by mid-conversation.

2. The Gap Between Inference and Prediction

Better latent inference does not reliably translate into calibrated downstream predictions.

• Over-Update and Extremes: In the coin task, larger models over-update, pushing predictions to extremes. In medical triage, models sharpen predictions for extreme labels (Emergency, Routine) but struggle with middle labels (Urgent, Observation), often pushing ambiguous cases toward the extremes.
• Conditional Performance: Explicitly conditioning downstream predictions on the inferred latent (e.g., "Given the user is Type X, what is the rating?") improves performance for some models in the recommender and medical tasks. However, models still fail to reliably translate their inferred latent structure into accurate unconditional predictions.

3. Active Engagement and Bias

Active engagement with a user in a simulated conversation introduces persistent biases not present in passive observation.

• Social Judgment: Active engagement reveals a persistent pro-user bias. Models assign lower probabilities that the poster is at fault compared to passive observation. While larger models reduce this bias as evidence accumulates, it does not disappear.
• Behavioral Shifts: Communication styles elicit different response patterns. Conceding users elicit more reassurance (counter-concession); defending users elicit more sympathy for the poster. These behavioral shifts occur even when the final verdict remains statistically invariant to style.
• Medical Triage: Active engagement degrades urgency assessment calibration for most models, with the gap closing only for the largest models (Qwen-32B, LLaMA-70B).

Significance and Claims

The paper claims that BayesBench provides a controlled way to study multi-turn LLM interaction as a belief-updating problem. The primary finding is an alignment gap: while scaling improves a model's ability to identify relevant hidden structure (latent inference), it does not yet reliably ensure that this inference is translated into calibrated, rational downstream predictions.

The authors emphasize that intermediate belief trajectories are critical for diagnosis; final-turn accuracy alone can mask systematic failures in how models accumulate and use evidence. The work motivates future research into multi-turn reasoning that accumulates evidence more reliably and post-training methods that make implicit latent integration more robust, ensuring that improvements from explicit conditioning carry over to ordinary conversational interactions. The authors remain modest, noting that their evaluation is tied to specific probe formats and controlled latent variables, and that real-world users may exhibit more complex, continuous, or mixed latent structures.