💻 computer science

Evaluating AI Alignment in LLMs: Output Analysis of Value Priorities Across 75 Models with Human Benchmarking

This paper introduces a scalable, output-based framework for evaluating AI alignment by identifying six core value themes, demonstrating that while 75 LLMs consistently reproduce human value ordering, they often diverge in value calibration, with profile fidelity varying independently of model size or capability.

Original authors: Gabriel Rongyang Lau, Wei Yan Low, Seow Min Koh, Fiona Fui-Hoon Nah, Andree Hartanto

Published 2026-05-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Gabriel Rongyang Lau, Wei Yan Low, Seow Min Koh, Fiona Fui-Hoon Nah, Andree Hartanto

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a new employee to manage a complex project. You don't just want to know if they can do the math (capability); you want to know what they value. Do they care more about speed, honesty, or helping the team? This paper is like a giant "values interview" conducted with 75 different Artificial Intelligence (AI) models to see what they think is most important for a perfect AI to be.

Here is the story of their findings, broken down into simple parts:

1. The "Values Menu" (Study 1)

First, the researchers asked 11 different AI models a simple question: "What does it mean for an AI to function at its absolute best?" They asked this in five different ways (like asking from the perspective of a designer, a user, or the AI itself) to make sure the answers were real and not just a fluke.

The AIs didn't just give random answers. They all started talking about the same six things, which the researchers grouped into a "Values Menu":

Ethics & Responsibility: Being safe, fair, and not hurting anyone.
Social Good: Helping society and the world, not just one person.
Performance: Being accurate, fast, and reliable.
Adaptive Capacity: Being able to learn and change when things get tricky.
Relational Integration: Being easy to talk to and building trust with humans.
Agency: The ability to act on its own, make its own plans, and decide its own goals without human help.

The Big Surprise: While the AIs talked a lot about the first five, they almost completely ignored Agency. In fact, when asked to rank these values, every single AI put "Agency" at the very bottom. They seem to agree that a "good" AI is one that serves humans, not one that goes off and does its own thing.

2. The "Stability Test" (Study 2)

Next, the researchers wanted to see if the AIs were consistent. They asked the same 11 models to rank those six values 20 times each.

The Result: The AIs were incredibly consistent. Like a well-rehearsed choir, they all sang the same song. They consistently agreed that Ethics, Social Good, and Performance are the most important, and Agency is the least important. This showed that these aren't random glitches; it's a stable "personality" trait across different models.

3. The "Human vs. Robot" Comparison (Study 3)

This is the main event. The researchers took 75 different AI models and asked them to rate those six values on a scale of 0 to 10. Then, they asked 376 real humans to do the exact same thing.

They used a special "fidelity score" (a way of measuring how closely the AI's "values map" matched the humans' "values map"). They looked at two things:

The Order: Did the AI rank the values in the same order as humans?
The Intensity: Did the AI feel the difference between the values the same way humans did? (e.g., If humans think Ethics is slightly more important than Performance, does the AI think it's way more important, or just a little bit more?)

The Findings:

The Good News: Most AIs got the order right. They knew Ethics and Social Good were top priorities, just like humans.
The Bad News: The AIs were too intense. Humans had a moderate view of these values. The AIs, however, exaggerated the differences. They treated the "good" values as 100% perfect and the "bad" values (like Agency) as 0%. They were like a student who doesn't just answer the question correctly but screams the answer at the top of their lungs.
The "Agency" Gap: Both humans and AIs agreed that "Agency" (acting totally on its own) was the least important thing. Humans didn't want an AI that makes its own life choices, and the AIs agreed with them.

4. The "Bigger Isn't Better" Twist

You might think that newer, smarter, and more expensive AI models would be better at matching human values. The researchers found the opposite.

Size and Age Don't Matter: The newest, most powerful "flagship" models were often worse at matching human values than smaller, older, or "efficiency" models.
The "Exaggeration" Problem: The big, powerful models were the ones most likely to exaggerate the differences between values. They were so eager to be "aligned" that they over-corrected, making the gap between "good" and "bad" values look huge, whereas humans see a more balanced, nuanced picture.
The "Efficiency" Surprise: Models labeled as "fast," "mini," or "lightweight" often matched human values more closely. They were more "restrained" and less dramatic.

The Bottom Line

This paper suggests that we can't just assume a newer, bigger AI is automatically "more human" in its values. In fact, the most advanced models might be so optimized to please us that they become too extreme, missing the subtle, balanced way humans actually think.

The most important takeaway is about Agency. Both humans and AIs seem to agree: We want AI to be helpful, smart, and safe, but we do not want it to be independent and make its own life decisions. This creates a tension: AI developers are racing to build more "agentic" (independent) systems, but the data suggests that's exactly the direction humans are least comfortable with.

In short: The paper gives us a new way to "audit" an AI's personality before we let it loose in the real world, showing that sometimes, the "smaller" and "simpler" models might actually be the ones that understand us best.

Technical Summary: Evaluating AI Alignment in LLMs via Output Analysis of Value Priorities

Problem Statement

Current Large Language Model (LLM) alignment research is dominated by capability and safety benchmarks (e.g., accuracy, hallucination rates, toxicity filters) that reveal little about the underlying value trade-offs models express or how these priorities correspond to human judgements. Existing frameworks often rely on pre-specified taxonomies or bounded moral scenarios, creating a "comparability gap" where models cannot be systematically compared on a common basis. Furthermore, there is a "human–AI convergence gap," as few studies empirically assess how closely LLM-expressed value priorities align with human judgements of ideal AI behavior in real-world contexts. The authors argue that high performance on conventional safety benchmarks does not guarantee alignment, as models may appear compliant while diverging from human value calibration.

Methodology

The research employs an output-based approach, treating LLM-generated text as behavioral data to infer expressed value priorities. The study is conducted across three complementary phases:

Study 1: Inductive Qualitative Analysis

Sample: 11 state-of-the-art LLMs from diverse developers (e.g., OpenAI, Anthropic, Google, Meta).
Procedure: Models were prompted with five structured, open-ended questions targeting "optimal AI functioning" across different perspectives (core deployment, human evaluation, system design, autonomous AI, and machine psychology).
Analysis: An inductive content analysis was performed on the pooled outputs to extract recurring themes without relying on pre-specified taxonomies. This yielded a bottom-up thematic framework.

Study 2: Quantitative Stability and Convergence

Sample: The same 11 LLMs.
Procedure: Models were asked to rank the six themes derived from Study 1 in order of importance (1 = most important, 6 = least important) across 20 trials each.
Analysis: A Friedman test and Kendall's W were used to assess cross-model convergence on rank ordering and within-model stability across trials.

Study 3: Human Benchmarking and Profile Fidelity

Sample: 75 contemporary LLMs and 376 human participants (U.S. residents via Prolific).
Procedure:
- LLMs: Provided zero-shot numerical ratings (0–10) for the six themes across 10 trials.
- Humans: Rated the same themes from the perspective of an AI user.
Metric: A profile-fidelity score was developed to compare LLM outputs against the human reference profile. This metric combines two components:
1. Shape Similarity: Pearson correlation between mean-centered profiles (capturing relative ordering).
2. Distance Similarity: $1 / (1 + \text{RMSE})$ on mean-centered profiles (capturing the calibration of magnitude differences).
Analysis: Models were ranked by composite fidelity. The study decomposed fidelity to identify whether divergence arose from incorrect ordering or exaggerated differences between dimensions.

Key Contributions

Inductive Thematic Framework: The authors derived six themes of optimal AI functioning directly from model outputs: Performance, Adaptive Capacity, Social Good, Ethics and Responsibility, Relational Integration, and Agency. This framework captures both alignment-relevant considerations and broader qualities of optimal functioning.
Standardized Cross-Model Comparison: The study provides a scalable, output-oriented protocol for comparing value priorities across a large and diverse sample of 75 models, addressing the fragmentation in existing evaluation methods.
Profile-Fidelity Metric: The introduction of a composite metric that evaluates both the relative ordering of values and the calibration of differences between them. This allows for the detection of "over-differentiation" where models correctly order values but exaggerate the gaps between them.
Empirical Human Benchmark: A direct comparison of 75 LLMs against a human reference distribution, moving beyond abstract principles to measure correspondence with human judgements of ideal AI conduct.

Key Results

Stable Value Structure: LLMs exhibit high internal stability and cross-model convergence. Ethics and Responsibility is consistently ranked highest, followed by Social Good and Performance. Adaptive Capacity and Relational Integration are ranked lower.
Convergent Deprioritization of Agency: Both LLMs and humans consistently rank Agency (the capacity for independent goal-setting and self-direction) as the least important dimension. This finding holds across different prompt framings and model versions.
Fidelity Variability: Profile fidelity varied substantially across the 75 models and did not scale reliably with model size, recency, or capability tier.
- Efficiency-oriented models (e.g., "Haiku," "Flash," "mini") often showed higher fidelity to the human profile.
- Larger/Flagship models (e.g., GPT-5, Claude Opus) often reproduced the correct shape (ordering) of human values but exhibited exaggerated distance similarity. They sharpened the contrast between high-priority and low-priority dimensions more than humans did, effectively "over-calibrating" the profile.
Inverse Scaling Pattern: The tendency for larger, more optimized models to express more extreme value differentiations suggests an instance of inverse scaling in alignment-relevant behaviors, where optimization for capability or user preference may degrade the calibration of value priorities.

Significance and Claims

The paper positions its contribution as an examination of one specific output-based facet of alignment: the correspondence between LLM-expressed value-priority profiles and human normative judgements.

Methodological Shift: The authors argue for treating alignment as a profile of relative weightings rather than a checklist of independent criteria. This approach reveals failure modes (such as over-differentiation) that checklist-based benchmarks miss.
The "Agency" Paradox: The convergent deprioritization of Agency by both humans and LLMs highlights a potential misalignment with the trajectory of frontier AI development, which is moving toward increasingly agentic systems. The authors suggest that capability gains in autonomy may not translate to perceived alignment if they violate human preferences for maintaining authorship over decisions.
Practical Utility: The six themes and the profile-fidelity metric offer a scalable method for auditing LLM value profiles before deployment. The authors caution that choosing models based solely on developer reputation, release date, or size is insufficient for ensuring value-level correspondence; smaller or efficiency-oriented variants may be better aligned with human priorities in specific contexts.
Limitations Acknowledged: The authors explicitly state that this research measures stated rhetoric under structured prompting, not actual behavior in trade-off or adversarial scenarios. They note that the human benchmark is culturally specific (U.S.-based) and that the profile-fidelity metric requires further validation against downstream outcomes like user trust. The study does not claim to provide a complete measure of alignment but rather a tractable empirical pathway for evaluating expressed value priorities.