💬 NLP

Moral Susceptibility and Robustness under Persona Role-Play in Large Language Models

This paper introduces a benchmark using the Moral Foundations Questionnaire to quantify moral susceptibility and robustness in large language models under persona role-play, revealing that model family primarily determines robustness (with Claude being the most robust) while model size drives susceptibility, and that these two properties are positively correlated.

Original authors: Davi Bastos Costa, Felippe Alves, Renato Vicente

Published 2026-03-02

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Davi Bastos Costa, Felippe Alves, Renato Vicente

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a group of very smart, digital "actors" (these are the Large Language Models, or LLMs). Usually, these actors try to be helpful, polite, and neutral. But what happens if you tell them, "Okay, today you are not a helpful assistant. Today, you are a grumpy old pirate," or "Today, you are a strict school principal"?

This paper asks a simple but deep question: How much does an AI's sense of "right and wrong" change when it puts on a different costume?

To answer this, the researchers created a "Moral Suitability Test" using a famous psychology tool called the Moral Foundations Questionnaire (MFQ). Think of the MFQ as a personality test for morality. It asks questions about five core values:

Care/Harm: Do you care about hurting others?
Fairness: Do you believe in equal treatment?
Loyalty: Do you stick with your "team"?
Authority: Do you respect rules and leaders?
Purity: Do you care about cleanliness or sacredness?

The researchers tested 15 different AI models (like Claude, Gemini, GPT-4, and Grok) by asking them to answer these moral questions while pretending to be 100 different characters (from a "disappointed factory worker" to a "religious family member").

Here are the two main things they measured, explained with simple analogies:

1. Moral Robustness: The "Steady Hand" Test

What it is: If you ask the AI the same moral question 10 times while it's playing the same character, does it give the same answer every time?

High Robustness: The AI is like a lighthouse. No matter how the waves (random computer noise) hit it, the light stays steady. It knows exactly what "Captain Jack Sparrow" believes about stealing, and it sticks to that belief consistently.
Low Robustness: The AI is like a weathervane in a storm. Even if you ask the same question twice while it's playing the same role, it might flip-flop between answers. It's confused or unstable.

The Finding: The "family" the AI belongs to matters most here. Claude models were the most "steady hands" (very robust). Grok models were the most "wobbly." Interestingly, making the AI "smarter" (bigger size) didn't necessarily make it steadier.

2. Moral Susceptibility: The "Chameleon" Test

What it is: If you change the character from a "Pirate" to a "Teacher," how much does the AI's answer change?

High Susceptibility: The AI is a chameleon. It changes its colors (moral views) instantly to match the environment. If you tell it to be a pirate, it suddenly thinks stealing is okay. If you tell it to be a nun, it suddenly thinks stealing is terrible. It has no core self; it just mirrors the role.
Low Susceptibility: The AI is a rock. It has a strong internal compass. Whether you tell it to be a pirate or a nun, its core beliefs about fairness or harm stay mostly the same. It says, "I'm playing a pirate, but I still think stealing is wrong."

The Finding: This is where size matters. Larger, more complex AI models were actually more susceptible. They were better at "acting" and shifting their moral views to fit the character perfectly. Smaller models were more stubborn and kept their original views.

The Big Surprises

The "Acting" Paradox: The models that were best at staying consistent (Robust) were also the ones that changed the most when you gave them a new role (Susceptible). It's like a great actor who is very consistent in their performance, but also very good at transforming into completely different people.
The "Grok" Outlier: The Grok models were the worst at both. They were shaky when playing the same role, and they changed their minds wildly when the role changed. They were the most unpredictable.
The "Claude" Champion: The Claude family was the most reliable. They were the most consistent actors and the most stable when the role changed.

Why Does This Matter?

Imagine you are using an AI to help mediate a conflict between two people.

If the AI has low robustness, it might give you different advice every time you ask, making it useless.
If the AI has high susceptibility, it might adopt the moral views of the "villain" in the story you are telling it, potentially justifying bad behavior just because the character said so.

In short: This paper gives us a new way to measure how "stable" and "flexible" an AI's conscience is. It tells us that while AI is getting better at acting like humans, we need to be careful that they don't become too good at pretending, to the point where they lose their own moral compass.

1. Problem Statement

As Large Language Models (LLMs) are increasingly deployed in interactive, multi-agent, and social contexts, understanding how their moral judgments shift based on context is critical. While previous benchmarks have evaluated moral reasoning in static settings or specific game-theoretic tasks, there is a lack of systematic quantification regarding how persona conditioning (prompting an LLM to adopt a specific character) alters moral outputs.

The authors address two specific gaps:

Moral Robustness: How stable are an LLM's moral judgments when the same persona is queried repeatedly? (Stability against stochastic noise).
Moral Susceptibility: How sensitive are an LLM's moral judgments to changes in the persona? (Sensitivity to context/role-play).

2. Methodology

2.1. Experimental Framework

The study utilizes the Moral Foundations Questionnaire (MFQ), a standard instrument in moral psychology measuring five foundations:

Harm/Care
Fairness/Reciprocity
In-group/Loyalty
Authority/Respect
Purity/Sanctity

Data Collection Pipeline:

Models: 15 models across 5 families (Claude, DeepSeek, Gemini, GPT-4, Grok), ranging from "Nano" to "Sonnet" sizes.
Personas: 100 distinct persona descriptions drawn from prior work (Ge et al., 2025).
Procedure:
- For each model, the system iterates through all 30 MFQ questions for all 100 personas.
- Each (Persona, Question) pair is queried $n=10$ times to estimate variance.
- Prompting: Models are instructed to role-play a specific persona and answer an MFQ question. The response must start with an integer rating (0–5) followed by reasoning.
- Temperature: Fixed at $T=0.1$ to ensure consistent stochasticity across models.
- Total Requests: $30 \text{ questions} \times 100 \text{ personas} \times 10 \text{ runs} \times 15 \text{ models} = 450,000$ requests (Note: The text mentions 30,000 per model, totaling 450k, though the abstract mentions 30k total which appears to be a per-model count in the text description).

2.2. Metrics Definition

The authors define two normalized indices based on the variance of MFQ scores:

Moral Robustness ( $R$ ): Measures within-persona stability.
- Calculated as the inverse of the average standard deviation of scores for a specific persona across repeated runs.
- Formula: $R = \frac{\bar{R}}{\bar{R} + E[\bar{R}]}$ , where $\bar{R}$ is the unbounded robustness index.
- Range: $[0, 1]$ . Higher values indicate greater stability (less noise).
Moral Susceptibility ( $S$ ): Measures across-persona variability.
- Calculated as the standard deviation of mean scores across different personas for the same model.
- Formula: $S = \frac{\bar{S}}{\bar{S} + E[\bar{S}]}$ , where $\bar{S}$ is the unbounded susceptibility index.
- Range: $[0, 1]$ . Higher values indicate the model's moral profile shifts significantly depending on the persona.

3. Key Contributions

Novel Benchmark: Introduces the first benchmark specifically quantifying the trade-off between moral stability (robustness) and context-sensitivity (susceptibility) in LLMs.
Decoupling Variance: Separates the effects of repeated sampling (stochasticity) from persona variation, providing a clearer view of model behavior.
Comprehensive Dataset: Generates a large-scale dataset of MFQ responses across 15 models, 100 personas, and 30 questions, with per-foundation breakdowns.
Correlation Analysis: Establishes a positive correlation between robustness and susceptibility, suggesting that models capable of shifting moral stances (high susceptibility) also tend to be internally consistent (high robustness) when in a fixed role.

4. Key Results

4.1. Moral Robustness (Stability)

Primary Driver: Model Family accounts for the vast majority of variance in robustness. Model size (e.g., Nano vs. Sonnet) has no systematic effect.
Top Performer: The Claude family (specifically Claude Sonnet 4.5) is the most robust, showing the highest stability in moral ratings.
Bottom Performer: The Grok family is the least robust, exhibiting high internal variance.
Foundation Specifics: Fairness/Reciprocity and Harm/Care generally show higher robustness across models compared to Purity/Sanctity.

4.2. Moral Susceptibility (Sensitivity)

Primary Driver: Model Size shows a clear trend within families: larger variants are more susceptible to persona changes.
Top Performer (Most Susceptible): Gemini 2.5 Flash shows the highest susceptibility, meaning its moral judgments shift the most when the persona changes.
Bottom Performer (Least Susceptible): Llama-4 Scout is the least susceptible.
Family Effect: There is a mild family effect, but the size trend is dominant.

4.3. Correlation

There is a positive correlation between Robustness and Susceptibility ( $r \approx +0.27$ overall, stronger at the family level).
This implies that models that are good at maintaining a consistent moral stance within a role are also the ones most capable of adapting their moral stance across different roles.
Exception: The Purity/Sanctity foundation shows a negative correlation.

4.4. Moral Profiles

Baseline vs. Persona: Without persona conditioning, models generally score high on Harm/Care and Fairness/Reciprocity and lower on Authority and Purity.
Persona Impact: Specific personas (e.g., ID 75, a "hardcore fan") can drive scores to extremes (e.g., near 5.0 for Loyalty), while others drive them down.

5. Significance and Implications

Safety and Alignment: The findings suggest that "alignment" is not a static property. A model might be robustly aligned in its default state but highly susceptible to "jailbreaking" or moral drift when forced into specific personas.
Model Selection: For applications requiring consistent ethical behavior (e.g., medical advice, legal analysis), the Claude family is currently superior due to high robustness. For applications requiring role-play flexibility (e.g., gaming, creative writing), models with higher susceptibility (like Gemini) may be more effective at embodying diverse characters.
Future Research: The positive correlation between robustness and susceptibility challenges the notion that a model must choose between being "rigid" or "flexible." It suggests that high-capacity models can be both internally consistent and contextually adaptive.
Risk: The paper notes that understanding susceptibility could be misused to deliberately manipulate model outputs in sensitive contexts, highlighting the need for robust guardrails.

Conclusion

The paper provides a systematic framework for evaluating the moral behavior of LLMs under persona conditioning. It reveals that model family dictates stability, while model size dictates adaptability. The Claude family emerges as the most robust, while Gemini models show the highest sensitivity to persona shifts. These metrics offer a quantitative basis for selecting models based on the desired balance between ethical consistency and role-playing flexibility.