Evaluating LLM Alignment With Human Trust Models

This paper presents a white-box analysis of the EleutherAI/gpt-j-6B model, demonstrating through contrastive prompting that its internal representation of trust aligns most closely with the Castelfranchi socio-cognitive model, thereby validating the feasibility of using LLM activation spaces to analyze socio-cognitive constructs and inform human-AI collaboration.

The Gist

Imagine you have a very smart, but slightly mysterious, robot brain (a Large Language Model, or LLM). You know it can write stories, answer questions, and chat like a human. But here's the big question: Does it actually understand what "trust" means, or is it just guessing the right words?

This paper is like a "brain scan" for that robot. The researchers wanted to peek inside the robot's mind to see how it organizes the concept of trust, and whether that organization matches how humans think about it.

Here is the story of their discovery, broken down with some simple analogies.

1. The Problem: The "Black Box" Mystery

Usually, when we use AI, we treat it like a Black Box. We put a question in one side, and an answer comes out the other. We don't know what happens inside.

• The Old Way: Researchers would ask the AI, "Do you trust this person?" and see what it said. This is like judging a chef only by the taste of the food, without seeing the kitchen.
• The New Way (This Paper): The researchers decided to open the box. They looked at the internal wiring (the mathematical "activation space") of the AI to see how it actually feels about trust.

2. The Method: The "Contrastive Prompting" Game

To see how the AI thinks about trust, the researchers played a game of Contrastive Prompting.

Think of the AI's mind as a giant library of feelings. If you ask it to write a story about "Happiness," it pulls out a specific book. If you ask for "Sadness," it pulls a different one.

• The Trick: They asked the AI to write two stories for every concept: one where the concept is present (e.g., "Katherine helps Alice") and one where it is absent or opposite (e.g., "Katherine ignores Alice").
• The Result: By subtracting the "ignoring" story from the "helping" story, they isolated the pure "mathematical fingerprint" of Helpfulness or Trust inside the AI's brain. It's like taking a photo of a room with the lights on, then with the lights off, and subtracting the two to see exactly where the lightbulb is.

3. The Map: Five Different "Trust Theories"

Humans have written many books on how trust works. The researchers picked five famous theories (like the Marsh Model, Mayer Model, and Castelfranchi Model) to see which one the AI's brain looked most like.

Imagine these theories are different maps of a city:

• Map A (Marsh): Says trust is just a probability score based on past behavior.
• Map B (Mayer): Says trust is about ability, kindness, and honesty.
• Map C (Castelfranchi): Says trust is a complex mental state involving beliefs, goals, and predictions.

The researchers took the AI's "Trust Fingerprint" and tried to overlay it onto these five maps. They asked: "Does the AI's internal map look like Map A, Map B, or Map C?"

4. The Discovery: The "Castelfranchi" Match

Here is the exciting part. They measured how close the AI's internal map was to the human maps using a "similarity score" (like a dating app match percentage).

• The Winner: The AI's brain matched the Castelfranchi Model the best!

  • What does this mean? The Castelfranchi model is very "psychological." It says trust isn't just about "did they do the job?" It's about "do I believe they want to do the job, and do I believe they can do it?"
  • The AI seems to understand that trust is a mental attitude, not just a scorecard. It connects trust with concepts like "willingness," "commitment," and "predictability" in a way that feels very human.

• The Runner-Up: The Marsh Model came in second. This is the more "mathy" model based on past performance. The AI gets this too, but it prefers the deeper, psychological understanding.

5. The Surprise: Where the AI Got It Wrong

The researchers also found some funny mismatches.

• The "Risk" Confusion: In the Mayer Model, "Risk" is a positive thing. The theory says, "You can only trust someone if you are willing to take a risk."
• The AI's View: When the researchers checked the AI's brain, it thought Risk and Trust were opposites! To the AI, "Risk" felt more like "Danger" or "Fear," not "Vulnerability."
• The Lesson: The AI hasn't fully learned the human nuance that sometimes you have to be brave (take a risk) to trust someone. It sees risk as a bad thing, period.

Why Does This Matter?

Think of the AI as a new employee joining your team.

• Before: You didn't know if they understood your company culture.
• Now: You know they have a very sophisticated internal map of how relationships work. They understand that trust is about beliefs and goals, not just checking boxes.

The Takeaway:
This paper proves that AI isn't just a parrot repeating words. It has built a structured, internal understanding of human social concepts like trust. By knowing how the AI thinks about trust, we can:

1. Build better AI: We can tweak the AI's "brain" to make it act more trustworthy.
2. Understand ourselves: It helps us see where human logic and machine logic differ (like the "Risk" example).

In short, the researchers opened the robot's head, found a very human-like map of trust inside, and realized that while the robot is smart, it still needs a little help understanding that sometimes, taking a risk is the first step to trusting someone.

Technical Summary

Here is a detailed technical summary of the paper "Evaluating LLM Alignment With Human Trust Models" by Debnath et al.

1. Problem Statement

While Large Language Models (LLMs) are increasingly used in human-AI collaboration and multi-agent systems, there is a significant gap in understanding how these models internally conceptualize and reason about trust.

• The Gap: Existing research largely adopts a "black-box" approach, analyzing LLM inputs and outputs (e.g., conversation analysis) without probing the internal activation patterns that underpin trust-related cognition.
• The Challenge: Trust is a complex socio-cognitive construct defined differently across various theoretical frameworks (e.g., computational vs. socio-cognitive models). It remains unclear whether LLMs encode these specific theoretical structures in their latent spaces or if they merely mimic surface-level linguistic patterns.
• Objective: The authors aim to perform a white-box analysis to determine if and how the internal embeddings of an LLM align with established human trust models.

2. Methodology

The study employs a contrastive prompting technique to extract embedding vectors from the activation space of the open-source model EleutherAI/gpt-j-6B. The methodology is divided into three phases:

A. Contrastive Prompting & Vector Extraction

• Concept Generation: The authors generated 100 positive and 100 negative one-line stories for 30 distinct emotional/interpersonal concepts (e.g., Trust, Competence, Risk, Benevolence).
• Directionality: To capture the dyadic nature of trust, concepts were treated bidirectionally (e.g., Katherine trusts Alice vs. Alice trusts Katherine), expanding the set to 60 distinct directional concepts.
• Prompt Design: Prompts maintained a constant background (two colleagues, Katherine and Alice) while varying the specific behavior to elicit the target concept (positive) or its opposite (negative).
• Vector Calculation:
  1. The model processed the stories, and hidden representations (embeddings) were extracted from all 28 transformer layers.
  2. Token-level vectors were averaged to create statement-level vectors.
  3. Mean vectors for positive and negative stories were computed per layer.
  4. Persona Vectors: The difference between the mean positive and mean negative vectors was calculated for each layer. These layer-wise difference vectors were stacked and then averaged across all layers to produce a single embedding vector representing the concept's direction in the model's latent space.

B. Establishing a Similarity Threshold

• Baseline Analysis: Pairwise cosine similarities were computed across all 60 concept vectors.
• Threshold Derivation: A histogram of these similarities was plotted. The authors identified a threshold corresponding to the 80th percentile of the distribution.
• Purpose: This threshold (empirically set at 0.6) distinguishes "significantly aligned" concepts from general associations. Only concept pairs exceeding this threshold are considered to have strong internal semantic alignment within the LLM.

C. Alignment with Trust Models

• Target Models: The study evaluated five established trust models:
  1. Marsh Model (Computational/Probabilistic)
  2. Mayer Model (Organizational: Ability, Benevolence, Integrity)
  3. McAllister Model (Cognition-based vs. Affect-based)
  4. McKnight Model (Initial trust formation)
  5. Castelfranchi Model (Socio-cognitive: Beliefs, Goals, Willingness)
• Evaluation Metrics: For each model, the authors:
  1. Identified the specific trust-related concepts associated with the model (e.g., Competence, Reliability, Risk).
  2. Computed the cosine similarity between the Trust vector and each associated concept vector.
  3. Calculated two metrics:
     • Average Cosine Similarity: The mean similarity score of all concepts in the model relative to Trust.
     • Threshold Count: The number of concept-trust pairs exceeding the 0.6 similarity threshold.

3. Key Results

A. Internal Structure of the LLM

• The LLM's latent space effectively separates opposing emotional concepts (e.g., Happiness vs. Anger) while clustering related concepts, validating the use of contrastive prompting for extracting semantic directions.

B. Model Alignment Rankings

The study quantified how closely the LLM's internal representation of trust aligns with each theoretical framework:

Trust Model	Average Cosine Similarity	Concepts Above Threshold (0.6)	Alignment Rank
Castelfranchi	0.7303	8	1st (Highest)
Marsh	0.6973	7	2nd
McAllister	0.6704	4	3rd
McKnight	0.6640	5	4th
Mayer	0.4530	5	5th (Lowest)

C. Specific Findings & Anomalies

• Castelfranchi Dominance: The LLM's internal representation of trust most strongly aligns with the Castelfranchi socio-cognitive model, which views trust as a mental state grounded in beliefs about competence, willingness, and predictability.
• Mayer Model Discrepancy: The Mayer model showed the lowest alignment. Notably, concepts theoretically linked to trust in this model, such as Risk and Benevolence, exhibited negative cosine similarities with the Trust vector in the LLM.
  • Interpretation: While human theory posits that "willingness to take risk" is a prerequisite for trust, the LLM's internal embeddings associate "Risk" with negative or unrelated concepts, failing to capture this specific theoretical nuance.

4. Key Contributions

1. White-Box Trust Analysis: This is the first study to move beyond black-box evaluation, providing a mechanistic interpretation of how trust is encoded in the activation space of an LLM.
2. Contrastive Prompting Framework: The authors demonstrate a robust method for extracting bidirectional, concept-specific embedding vectors (persona vectors) to analyze complex social constructs.
3. Theoretical Validation: The work empirically validates that LLMs encode socio-cognitive constructs in a structured manner, with a specific preference for the Castelfranchi model over purely computational or organizational models.
4. Identification of Misalignment: The study highlights specific areas where LLM reasoning diverges from human theoretical models (e.g., the handling of "Risk"), offering a target for future alignment research.

5. Significance and Implications

• Human-AI Collaboration: Understanding that LLMs naturally align with socio-cognitive models (Castelfranchi) suggests they are better suited for dynamic, context-aware interactions than purely statistical trust models.
• Steering AI Behavior: The findings suggest that "trust vectors" derived from specific theoretical models can be injected into an LLM's activations during generation. This could allow developers to steer AI behavior toward specific trustworthiness traits (e.g., increasing "competence" or "reliability" embeddings) to create more trustworthy agents.
• Trust Monitoring: The ability to quantify latent trust representations enables the development of software that monitors trust levels between interacting entities (e.g., coach-athlete, user-agent) in real-time.
• Future Research: The paper lays the groundwork for studying the evolution of trust reasoning in multi-turn conversations and comparing these structures across different LLM architectures.

Limitations

• Model Specificity: The analysis is limited to a single model (EleutherAI/gpt-j-6B). Results may vary with larger or differently trained models.
• Static vs. Dynamic: The study uses static embeddings and does not yet capture how trust representations evolve dynamically during multi-turn interactions.