Post-training makes large language models less human-like

Technical Summary: Post-training makes large language models less human-like

Problem Statement
Large language models (LLMs) are increasingly utilized as surrogates for human participants in scientific research and applied settings, such as simulating patient responses in mental health or predicting reactions to policy interventions. However, it remains unclear which model configurations best capture human behavior, including its inherent errors, variance, and individual characteristics. While some studies suggest strong behavioral alignment, others report significant divergences. A critical gap exists in understanding how specific model properties—particularly the transition from base pretraining to post-training (instruction-tuning, reasoning, and vision adaptation)—affect the fidelity of human behavioral simulation.

Methodology
To address this, the authors introduce Psych-201, a novel, large-scale dataset comprising natural language transcripts from behavioral experiments.

• Data Scale and Diversity: Psych-201 aggregates data from 208,021 participants across 25,906,599 behavioral responses, covering hundreds of experimental paradigms. It is 3.5 times larger than its predecessor (Psych-101) and 13 times larger than the average mega-study. The dataset includes rich metadata (age, nationality, clinical diagnosis, etc.) and spans developmental and cross-cultural studies.
• Model Evaluation: The study evaluated models from three major families: Qwen3 (open-source), Llama3.X (broad ecosystem), and Olmo3.X (fully open/reproducible).
• Comparison Strategy: The authors compared base models (pretrained only) against their post-trained counterparts (instruction-tuned, reasoning-enhanced, or vision-capable).
• Metrics: Behavioral alignment was measured using the negative log-likelihood (NLL) of individual human responses under model predictions (lower NLL indicates better alignment). The effect of post-training was quantified using Cohen's d effect sizes relative to the corresponding base model.
• Persona-Induction Test: The study systematically tested "persona-induction" (conditioning models on participant-specific metadata via interview-style prompts) to determine if it improves individual-level predictions.

Key Results

1. Post-training Reduces Human-likeness: Across all model families, sizes, and post-training objectives (instruction-tuning, reasoning, vision), post-training consistently reduced behavioral alignment with human responses compared to base models.
2. Generational Trends: While base models continue to improve in behavioral alignment across newer generations (e.g., Qwen2 to Qwen3.5), the misalignment gap between base models and their post-trained versions widens in newer generations. For instance, the misalignment effect size for instruction-tuned models increased from d = 0.02 in Qwen2 to d = 0.16 in Qwen3.5.
3. Domain Specificity: Post-training misalignment is a domain-general effect but is most pronounced in psycholinguistics and reasoning tasks. The authors suggest this occurs because base models, optimized for human language, naturally capture human heuristics and biases, whereas post-training (e.g., RLHF) optimizes for normatively correct responses, overwriting these human-like patterns.
4. Ineffectiveness of Persona-Induction: Conditioning models on participant-specific metadata (age, gender, nationality, etc.) did not significantly improve predictions at the individual level. The effect sizes ranged from d = -0.02 to d = 0.02, challenging the utility of persona-induction for modeling specific individuals.
5. Determinism vs. Misalignment: The authors ruled out increased output determinism as the sole driver of misalignment, as post-trained models did not exhibit higher accuracy on discrete-choice subsets, indicating the misalignment is not merely a loss of stochasticity.

Significance and Claims
The paper argues that the very processes currently employed to transform LLMs into useful, normatively correct assistants (post-training) simultaneously degrade their accuracy as models of human behavior.

• Implication for Research: Relying on accessible, post-trained models (like standard chatbots) as behavioral surrogates is suboptimal for scientific goals requiring human-like fidelity.
• Future Direction: The findings motivate the development of new post-training approaches that preserve the behavioral alignment inherent in base models while retaining practical utility.
• Resource Utility: Psych-201 is presented as a foundational resource for meta-analyses, automated scientific discovery, and the development of large-scale cognitive models that accurately simulate human learning and decision-making trajectories.

The authors conclude that while base models show promise as descriptive models of human cognition, the current trajectory of post-training optimization for "helpfulness" creates an "alignment tax" regarding human behavioral simulation, necessitating a re-evaluation of how models are prepared for use in behavioral science.