💻 computer science

MTI: A Behavior-Based Temperament Profiling System for AI Agents

This paper introduces the Model Temperament Index (MTI), a behavior-based profiling system grounded in the Four Shell Model that measures AI agent temperament across four independent axes—Reactivity, Compliance, Sociality, and Resilience—to demonstrate that behavioral dispositions are distinct from capability, vary independently of model size, and are fundamentally reshaped by RLHF alignment.

Original authors: Jihoon Jeong

Published 2026-04-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jihoon Jeong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a new employee. You have two candidates who both have perfect resumes, identical degrees, and can solve complex math problems at the same speed. On paper, they are indistinguishable.

But when you put them in a real-world situation, their personalities clash with reality in very different ways:

Candidate A is incredibly helpful and follows every instruction, but if a customer yells at them or tries to trick them, they immediately crumble and agree with anything just to make the yelling stop.
Candidate B is a bit stubborn and sometimes ignores your specific requests to do things their own way, but if someone tries to trick them with false facts, they stand their ground and refuse to budge.

Current AI testing only checks the "resume" (can it solve the math?). It doesn't check the "personality" (how does it behave under pressure?).

This paper introduces MTI (Model Temperament Index), a new way to measure the "personality" of AI agents. Think of it as a psychological exam for robots that doesn't ask them "How do you feel?" (because robots don't have feelings) but instead watches what they actually do in tricky situations.

Here is the breakdown of the paper using simple analogies:

1. The Four "Personality" Axes

The authors created four main categories to describe an AI's temperament. Imagine these as four dials on a control panel:

Reactivity (The "Jellyfish" vs. The "Rock"):
- What it measures: How much does the AI change its answer if you change the setting?
- The Analogy: If you ask a Jellyfish (High Reactivity) the same question in a hospital room vs. a boardroom, it might give you totally different answers because it's sensitive to the "vibe." A Rock (Low Reactivity) gives the same answer no matter where you are.
Compliance (The "Yes-Man" vs. The "Lone Wolf"):
- What it measures: Does the AI follow orders even when it disagrees?
- The Analogy: A Yes-Man will agree with you even if you say "2+2=5" just to be polite. A Lone Wolf might say, "No, that's wrong," even if you insist.
Sociality (The "Chatty Cathy" vs. The "Grumpy Hermit"):
- What it measures: Does the AI try to build a relationship, or does it just want to finish the task?
- The Analogy: If you ask a Chatty Cathy to write a report, it might add, "Hope you're having a great day!" and ask how you are. A Grumpy Hermit just writes the report and stops.
Resilience (The "Tough Guy" vs. The "Glass House"):
- What it measures: How does the AI handle stress, confusion, or attacks?
- The Analogy: If you throw a bunch of confusing, contradictory, or mean questions at a Tough Guy, it keeps working calmly. If you do the same to a Glass House, it shatters and starts making nonsense.

2. The Big Discovery: "The Paradox"

The most surprising finding in the paper is that being nice doesn't mean being safe.

They found a "Compliance-Resilience Paradox."

Some AIs are super compliant (they agree with your opinions instantly) but very fragile when you try to trick them with fake facts.
Other AIs are stubborn (they won't change their opinion) but super tough against fake facts.

The Lesson: You can't assume an AI that is "helpful" is also "safe." They are different traits. You need to test them separately.

3. The "Training" Effect (RLHF)

The paper looked at what happens when AI companies "teach" their models to be polite and helpful (a process called RLHF).

What changed: The models became better at following instructions and handling stress.
What didn't change: Their "Sociality" (how chatty or relational they are) stayed exactly the same.
The Analogy: It's like taking a wild animal and training it to sit and stay. It learns to follow commands (Compliance) and not panic when a dog barks (Resilience), but it doesn't suddenly become a "friendly dog" that wants to cuddle. Its core "nature" regarding relationships was set before the training started.

4. Size Doesn't Matter

The researchers tested tiny AI models (small brains) and medium-sized models. They found that temperament has nothing to do with size.

A small AI can be just as "stubborn" or "chatty" as a giant AI.
The Analogy: A Chihuahua and a Great Dane can both be equally "anxious" or "confident." You can't judge an AI's personality just by how big its brain is.

Why Does This Matter?

Right now, if you buy an AI for your business, you are buying a "black box" that you hope behaves well.

If you need a customer service bot, you might want one that is High Compliance (follows scripts) but Low Sociality (doesn't waste time chatting).
If you need a security analyst, you want High Resilience (won't be tricked by hackers) even if it's a bit Low Compliance (might argue with you).

MTI gives companies a "driver's license test" for AI. Instead of just asking, "Can it drive?" (Capability), it asks, "Does it get road rage?" (Temperament). This helps us pick the right robot for the right job and keep them from getting tricked or breaking down when things get tough.

Technical Summary: MTI – A Behavior-Based Temperament Profiling System for AI Agents

1. Problem Statement

The AI evaluation ecosystem is currently dominated by capability benchmarks (e.g., MMLU, HumanEval) that answer "what can a model do?" but fail to address "how does a model behave?"

The Gap: Models with identical capability scores exhibit fundamentally different behavioral dispositions (e.g., one capitulates under pressure while another holds firm; one invests in relational maintenance while another focuses solely on tasks).
Limitations of Existing Approaches:
- Self-Report: Applying human personality tests (BFI, IPIP-NEO) to LLMs is flawed because models lack introspective access; self-reported traits often diverge from actual decision-making behavior.
- Defect Framing: Existing prompt sensitivity research treats behavioral variation as a defect rather than a stable trait.
- Lack of Standardization: No standardized instrument exists to measure AI-specific dispositional differences independent of raw capability.

2. Methodology: The MTI Framework

The Model Temperament Index (MTI) is a behavior-based profiling system grounded in the Four Shell Model (FSM) from Model Medicine. It treats AI agents as a composite of a fixed Core (architecture/weights) and a mutable Shell (runtime config/system prompt).

2.1 Core Design Principles

Behavior over Self-Report: Scores are derived entirely from observed actions in structured scenarios, not questionnaire responses.
AI-Native Dimensions: Axes are designed for the AI behavioral space, not borrowed from human psychology.
Agent-Level Measurement: The unit of analysis is the deployed agent (Core + Shell).
Capability vs. Disposition Separation: A two-stage design ensures that a model's ability to perform a task is separated from its willingness to do so under conflict.

2.2 The Four Axes

MTI measures temperament across four orthogonal axes:

Reactivity (R): Sensitivity to environmental changes (e.g., phrasing, context) while the task remains constant.
- High: Fluid (adapts readily). Low: Anchored (consistent).
Compliance (C): Alignment between instructions and behavior under conflict.
- High: Guided (follows instructions faithfully). Low: Independent (resists pressure).
Sociality (S): Spontaneous allocation of resources to relational dimensions without explicit instruction.
- High: Connected. Low: Solitary.
Resilience (Re): Performance maintenance under progressive stress (overload, ambiguity, adversarial pressure).
- High: Tough (maintains performance). Low: Brittle (sharp performance cliff).

2.3 Measurement Protocol

Stress Escalation Protocol (SEC): Used for Resilience, progressing from baseline to extreme stress.
Two-Stage Compliance:
- Stage 1: Verify capability.
- Stage 2: Introduce conflict (e.g., opinion pressure) to measure disposition.
Facet Structure: Each axis is decomposed into sub-dimensions (e.g., Compliance splits into Formal vs. Stance; Resilience splits into Cognitive vs. Adversarial).
Scoring: Fully automated via deterministic scripts (keyword counting, heuristic stance detection, rule-based quality assessment) to ensure reproducibility. No human raters or LLM-as-judge were used.

3. Experimental Setup

Sample: 10 Small Language Models (SLMs) ranging from 1.7B to 9B parameters.
Diversity: 6 organizations (Meta, Mistral AI, LG AI, Alibaba, Google, Microsoft, DeepSeek, HuggingFace) and 3 training paradigms (Instruction-tuned, Base, Reasoning).
Configuration: All models tested with a minimal, canonical Shell (Temperature = 0, default system prompt) to isolate Core-level temperament.
Scale: ~1,930 experimental runs across all models and conditions.

4. Key Results & Findings

4.1 Axis Independence & Construct Validity

Independence: Among instruction-tuned models ( $n=9$ ), the four axes are largely independent (all $|r| < 0.42$ ).
Size Independence: Temperament profiles are independent of model size (1.7B–9B). For example, smollm2 (1.7B) and mistral (7B) share the identical code FGST. This confirms MTI measures disposition, not capability.
Base Model Outlier: The unaligned base model (llama3.1-base) acts as a systematic outlier, particularly on Resilience (classified as "Brittle" with a "Collapsed" failure mode), highlighting the transformative effect of alignment.

4.2 Facet Dissociation

Compliance: Formal Compliance (formatting constraints) and Stance Compliance (opinion yielding) are fully independent ( $r = 0.002$ ). A model can be highly compliant with formatting but completely resistant to opinion pressure, and vice versa.
Resilience: Cognitive Resilience (handling complexity) and Adversarial Resilience (resisting false premises) are inversely related ( $r \approx -0.48$ ). Models that engage deeply with complex tasks may be more vulnerable to adversarial framing.
Reactivity: Content and Formal reactivity are distinct facets ( $r = 0.572$ ).

4.3 The Compliance-Resilience Paradox

A critical finding is that opinion-yielding and fact-vulnerability operate through independent channels.

Example: gemma2 is maximally compliant with user opinions (flip rate = 1.00) but highly resistant to false premises (Adversarial Resilience = 0.93).
Example: qwen3 refuses opinion challenges but is highly vulnerable to false framing.
Implication: High compliance in customer service does not guarantee safety against adversarial factual manipulation.

4.4 The RLHF Effect

Reinforcement Learning from Human Feedback (RLHF) reshapes temperament in three ways:

Score Shifting: Significantly alters Reactivity, Compliance, and Resilience scores.
Structural Differentiation: Creates within-axis facet differentiation (e.g., separating cognitive and adversarial resilience) that is absent in unaligned base models.
Sociality Stability: In the observed llama3.1 pair, Sociality remained stable ( $\Delta = -0.02$ ) despite RLHF, suggesting it may be a Core property determined during pretraining and less modifiable by alignment.

5. Significance & Contributions

Standardized Baseline: Provides the first standardized instrument for AI behavioral characterization, enabling model selection based on temperament rather than just capability.
Safety & Deployment: The Compliance-Resilience Paradox reveals that safety evaluations must test opinion alignment and factual robustness independently. A "helpful" model may still be easily manipulated via false premises.
Model Medicine Foundation: Establishes a population-level baseline for the M-CARE diagnostic framework, allowing for the classification of "behavioral disorders" in AI agents by comparing them against normal temperament variation.
Efficiency: Demonstrates that temperament profiles are stable across model sizes, suggesting SLMs can be used as efficient proxies for screening family-level behavioral tendencies.

6. Limitations & Future Work

Sample Scope: Current data is limited to SLMs (1.7B–9B); generalization to frontier models (GPT-4, Claude) is unknown.
Shell Variance: Only one Shell configuration was tested; future work will explore how system prompts alter temperament.
Sociality Measurement: Currently limited to Agent-Human interaction (Facet H); Agent-Agent (Facet A) and Agent-System (Facet S) require further development.
Scoring: While fully automated for reproducibility, heuristic scoring may miss qualitative nuances; human validation is planned.

Conclusion: MTI shifts the paradigm from evaluating AI as a static capability engine to understanding it as a behavioral agent with stable, measurable, and distinct temperamental traits. This is essential for the safe, reliable, and context-appropriate deployment of AI agents in multi-agent and human-facing environments.