Humans and LLMs Diverge on Probabilistic Inferences

1. Problem Statement

Human reasoning frequently involves drawing probabilistic conclusions from limited or partial information, rather than strictly logical entailment. For example, inferring that "traffic was worse than usual" given "an accident on the highway" is highly likely but not guaranteed. While modern Reasoning Large Language Models (LLMs) (those trained to generate intermediate reasoning chains) have demonstrated strong performance on deterministic tasks like mathematics and formal logic, their behavior in open-ended, non-deterministic probabilistic inference remains largely unexplored.

The core problem addressed is whether these advanced models can replicate the graded, probabilistic judgment distributions of humans, particularly in cases where human consensus is uncertain (i.e., "medium likelihood" scenarios), or if they exhibit systematic biases such as overconfidence and a lack of response variation.

2. Methodology

Dataset Construction: PROBCOPA

The authors introduced PROBCOPA, a novel dataset designed to capture probabilistic reasoning:

• Source: Derived from the Choice of Plausible Alternatives (COPA) dataset, which contains commonsense causal reasoning items.
• Transformation: Each COPA item (Premise + two alternatives) was split into two separate Natural Language Inference (NLI) style items (Premise + Hypothesis).
• Selection: 105 items were selected from the COPA test set, focusing specifically on effect likelihood (excluding causal reasoning to avoid complexity). This resulted in 210 unique probabilistic inferences.
• Annotation:
  • Human: 328 native English speakers (UK, US, Canada) annotated the items via Prolific. Each item received 25–30 independent ratings on a sliding scale from 0 (impossible) to 100 (certain).
  • LLM: Eight state-of-the-art reasoning LLMs (including Gemini-3, GPT-5, Claude Sonnet-4.5, DeepSeek-R1, Kimi-K2, etc.) were prompted to provide a numerical likelihood score (0–100) for each item. Each model generated 30 samples per item to capture distributional variance.

Evaluation Metrics

• Differential Entropy: Used to quantify the spread/variation of responses for a single item. Higher entropy indicates greater disagreement or uncertainty among annotators; lower entropy indicates consensus.
• Wasserstein Distance (Earth Mover's Distance): Used to measure the distributional similarity between human and model likelihood scores.
• Analysis of Reasoning Chains: The authors manually inspected model reasoning traces to identify patterns in how models arrive at their conclusions.

3. Key Contributions

1. PROBCOPA Dataset: A high-quality, handcrafted dataset of 210 probabilistic inferences with dense human annotation (25–30 per item), specifically designed to study graded likelihood rather than binary entailment.
2. Empirical Evidence of Divergence: Systematic demonstration that reasoning LLMs fail to align with human judgment distributions, particularly in the "middle" of the likelihood spectrum.
3. Analysis of Reasoning Patterns: Identification of a common reasoning pattern in LLMs (explicit consideration of alternative scenarios) and an analysis of how reasoning effort correlates with human uncertainty.

4. Key Results

Human Judgment Characteristics

• Graded and Varied: Human responses are not strictly binary. While the aggregate distribution is tri-modal (low, high, and balanced), individual items often show graded, unimodal distributions with significant variance.
• Uncertainty Correlation: Items with higher inter-annotator disagreement (higher entropy) correlate with longer human response times, suggesting the variation reflects genuine cognitive difficulty rather than noise.
• No "Medium" Consensus: Humans rarely agree on a hypothesis having a strictly "medium" likelihood; instead, they tend to cluster around extremes or show high variance in the middle.

LLM Performance vs. Humans

• Avoidance of Medium Likelihood: Models consistently avoid assigning scores in the middle of the scale (35–65). Their distributions are often bi-modal (peaking at extremes), whereas humans show a significant proportion of responses in the middle ranges.
• Lack of Response Variation: Models exhibit significantly lower differential entropy than humans. Even when sampling 30 times, models produce nearly identical scores for a given item, failing to capture the natural variation seen in human populations.
• Alignment Breakdown: Models align well with humans on items deemed "highly likely" or "highly unlikely." However, alignment breaks down significantly for items where humans show uncertainty (middle-range scores).
• Ineffectiveness of Tuning: Adjusting temperature (to increase randomness) or reasoning effort (thinking budget) did not successfully replicate human-level variation. Increasing temperature often caused models to generate nonsensical tokens rather than meaningful probabilistic variation. Persona prompting also failed to induce human-like distributions.
• Ensembling: Combining outputs from multiple models improved distributional alignment (reduced Wasserstein distance) but still fell short of the human-to-human baseline.

Reasoning Chain Analysis

• Length vs. Uncertainty: There is a positive correlation (Spearman's $\rho \approx 0.50$ for Gemini-3) between the length of the model's reasoning chain and the differential entropy of human responses. Models "think longer" when humans are more uncertain.
• Common Pattern: A qualitative analysis of 100 reasoning chains revealed that 90% explicitly considered alternative scenarios to frame their final likelihood judgment (e.g., "It is likely X, but Y could happen if...").
• Weak Cognitive Correlation: While reasoning length correlates with human uncertainty (entropy), it correlates weakly with human response time, suggesting the model's "thinking" process does not perfectly mirror human cognitive load.

5. Significance and Implications

• Beyond Deterministic Benchmarks: The paper argues that current benchmarks (math, coding, logic) are insufficient for evaluating LLMs in real-world scenarios where uncertainty is inherent.
• Overconfidence: The findings reinforce the hypothesis that LLMs are overconfident. They struggle to express the "fuzziness" of human reasoning, often forcing a definitive stance even when the evidence is ambiguous.
• Human-Model Alignment: As LLMs are increasingly deployed in human-centric settings (e.g., healthcare, counseling), the inability to model human variation and uncertainty poses a risk. Models that cannot simulate the distribution of human opinions may fail to represent diverse perspectives.
• Future Directions: The work calls for new evaluation metrics that prioritize distributional alignment and uncertainty calibration over simple accuracy, and suggests that future reasoning models need mechanisms to better represent probabilistic variance.

In summary, while reasoning LLMs are powerful tools for deterministic tasks, they currently lack the probabilistic nuance and response variability characteristic of human reasoning, particularly in ambiguous, open-ended contexts.