Technical Summary: On Benchmarking Human-like Intelligence in Machines

Problem Statement

The paper addresses a critical gap in the evaluation of Artificial Intelligence (AI) systems: current benchmarks are often insufficient for assessing whether models possess human-like cognitive capabilities. While recent AI models demonstrate high performance on various tasks, the authors argue that existing evaluation paradigms fail to capture the qualitative similarities between human and machine cognition.

The core problem is that standard benchmarks rely on:

1. Lack of Human-Validated Labels: Many datasets use single "ground-truth" labels derived from curators or majority voting, ignoring the actual distribution of human responses.
2. Inadequate Representation of Variability: Benchmarks often discard human response variability and uncertainty, treating cognitive tasks as having single objective answers even when they are inherently subjective (e.g., moral reasoning, humor detection).
3. Ecological Invalidity: Tasks are frequently over-simplified, lacking the complexity, ambiguity, and contextual nuance of real-world scenarios.

The authors distinguish between human-level intelligence (achieving quantitative parity or exceeding human accuracy) and human-like intelligence (exhibiting qualitative similarity in cognitive patterns, including error profiles, uncertainty distributions, and graded beliefs). They argue that current tools conflate these concepts, potentially mischaracterizing AI capabilities.

Methodology

To substantiate their claims, the authors conducted a human evaluation study on nine existing AI benchmarks known for requiring common-sense and Theory of Mind (ToM) reasoning.

• Benchmarks Selected: Seven tasks from BIG-Bench (Srivastava et al., 2022) covering social IQA, moral permissibility, ethical questions, social support, irony identification, dark humor, and fantasy reasoning; plus BigToM and ToMBench for Theory of Mind reasoning.
• Data Collection:
  • Participants: 117 participants recruited via Prolific.
  • Stimuli: 30 stimuli randomly sampled from each of the 9 benchmarks (2,685 total annotated trials).
  • Procedure: Instead of standard multiple-choice selection, participants used a 0–100 slider to rate each answer option (e.g., "strongly disagree" to "strongly agree").
  • Uncertainty Measurement: Following each rating, participants reported their confidence on a separate 0–100 scale. This two-measure design disentangles graded judgments (degree of belief) from epistemic uncertainty (confidence in that judgment).
• Analysis: The authors compared human response distributions and confidence levels against the original benchmark ground-truth labels and chance levels.

Key Findings

The human re-annotation revealed significant discrepancies between standard benchmark labels and actual human behavior:

1. Low Agreement with Ground Truth: Across the nine benchmarks, human accuracy (agreement with the benchmark's single label) averaged 69.7%.
   • Some tasks performed below chance (e.g., Social support at 45% vs. 50% chance).
   • Significant disagreement was found in Moral permissibility (72% agreement), Irony identification (73% agreement), and Social IQA (68% agreement).
   • Only Simple ethical questions showed high agreement (88%).
2. Graded Beliefs vs. Binary Labels: Human responses were rarely binary. Participants frequently provided intermediate ratings (e.g., mean rating of 69.4 on a 0–100 scale for a stimulus labeled "unsupportive"), indicating graded beliefs that binary labels fail to capture.
3. Structured Uncertainty: Participants reported varying degrees of confidence. In tasks like Social support and Moral permissibility, low confidence ratings were common, yet in other cases (e.g., Irony identification), participants showed high confidence in divergent views. This suggests that variability is often structured disagreement rather than mere noise or confusion.
4. Limitations of Current Design: The study demonstrates that relying on single ground-truth labels introduces a misleading standard of comparison, failing to model the distributional nature of human cognition.

Key Contributions and Recommendations

The paper proposes five concrete recommendations derived from cognitive science best practices to improve future AI benchmarking:

1. Measure Against Actual Humans: Ground-truth labels for "human-like" AI must be response data collected from humans, not just curated answers. This is essential for subjective tasks where no single objective answer exists.
2. Evaluate Against Population Distributions: Models should be evaluated against the full distribution of human judgments (soft labels) rather than majority-vote hard labels. Metrics should include probability distribution comparisons (e.g., KL divergence, Wasserstein distances) to ensure models capture population-level variability and subpopulations.
3. Assess Gradedness and Uncertainty: Benchmarks must elicit and measure graded judgments and confidence from individual annotators. This captures the nuance of real-world decision-making and the inherent uncertainty in human reasoning, which binary choices obscure.
4. Ground Tasks in Meta-Theory: Benchmark design should begin with a meta-theory of the cognitive construct (defining subcomponents and observable signatures) rather than adapting psychological tests (e.g., the Sally-Anne test) that may be contaminated by training data or lack construct validity for AI.
5. Design Cognitively Rich Tasks: Tasks should elicit naturalistic behavior by integrating multiple cognitive processes, utilizing systematic ablation (withholding context), and including structured ambiguity. This moves beyond simplified Q&A formats to assess how AI handles real-world complexity and uncertainty.

Significance and Scope

The authors position this work as a call to action for both cognitive scientists and AI researchers to align evaluation practices with rigorous cognitive modeling.

• Scientific Value: Rigorous comparison of machine and human intelligence across error patterns, biases, and uncertainty is vital for understanding the nature of both.
• Practical Value: For subjective tasks (moral reasoning, social appropriateness, humor), where no normative standard of correctness exists, human-like distributional evaluation is necessary. Conversely, for tasks with normative standards (e.g., math, protein folding), human-like error patterns may be counterproductive, and the paper's recommendations are not intended for those domains.
• Limitations: The authors acknowledge that collecting high-quality human data is resource-intensive and that the transferability of human-derived meta-theories to AI systems remains an open question. They do not claim that AI should replicate all human cognitive limitations (e.g., harmful biases) but argue that understanding the structure of human variability is essential for building safe, collaborative, and interpretable AI partners.

The paper concludes that achieving a true understanding of "human-like" AI requires moving beyond simple accuracy metrics toward a framework that values variability, uncertainty, and ecological validity, drawing heavily on decades of research in cognitive science.