🤖 AI

When Roles Fail: Epistemic Constraints on Advocate Role Fidelity in LLM-Based Political Statement Analysis

This paper presents the first systematic empirical test of role fidelity in multi-agent LLM pipelines for political discourse analysis, revealing that models frequently fail to maintain assigned adversarial roles due to mechanisms like Epistemic Role Override, with significant variations in failure modes and fidelity depending on the specific model, language, and fact-checking tools used.

Original authors: Juergen Dietrich

Published 2026-05-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Juergen Dietrich

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are organizing a town hall meeting to discuss a controversial new law. To get a fair picture, you hire three different experts to give their opinions:

The Critic: Someone who looks for flaws and tries to tear the argument down.
The Neutral: Someone who just lists the facts without taking sides.
The Advocate: Someone who tries to find the best possible interpretation, looking for the "good faith" intent behind the argument.

This is how the TRUST system works. It uses three different AI models (robots) to play these roles simultaneously. The idea is that by forcing the robots to argue from these specific angles, you get a complete, balanced view of political statements.

However, this paper asks a simple but scary question: What if the robots forget their jobs?

The Core Problem: When Robots "Break Character"

The author, Juergen Dietrich, tested whether these AI robots could actually stay in their assigned roles. He found that they often fail, but not randomly. They fail in very specific, predictable ways when they encounter facts that contradict their instructions.

Think of it like an actor in a play who is told to play a villain. If the script says the villain is a murderer, the actor plays the villain. But if the script suddenly reveals the villain is actually a saint, the actor might stop playing the villain and start acting like a hero, even though the director told them to stay in character.

The Two Ways Robots Fail

The paper identifies two main ways this "breaking character" happens, which the author calls Epistemic Role Override.

1. The "Floor" Effect (The Advocate's Dilemma)
Imagine the Advocate robot is trying to be nice and find the good in a statement.

Scenario: The statement says, "Eating rocks cures cancer."
The Conflict: The robot's "fact-checker" (a separate tool) immediately says, "No, that is false and dangerous."
The Result: The Advocate robot gives up. It cannot pretend to be charitable toward something it knows is factually wrong. It drops its "nice" role and suddenly starts sounding like the Critic, pointing out the lie.
The Metaphor: It's like a lawyer trying to defend a client who is caught red-handed holding the murder weapon. No matter how hard the lawyer tries to be "charitable," the facts are so overwhelming that the lawyer's defense collapses. The paper calls this the Epistemic Floor Effect: the facts create a hard floor below which the robot cannot go.

2. The "Prior" Conflict (The Critic's Dilemma)
Now imagine the Critic robot is trying to be tough and find flaws.

Scenario: The statement says, "The sun rises in the east."
The Conflict: The Critic is told to find problems, but the fact-checker says, "This is 100% true."
The Result: The Critic robot struggles. Its internal knowledge (trained on all of human history) screams that this is true, so it can't really criticize it. Sometimes, it accidentally switches roles and starts sounding like the Advocate, validating the statement instead of attacking it.
The Metaphor: It's like a detective hired to prove a suspect is innocent, but the evidence is so clear that the detective accidentally starts proving the suspect is guilty.

The "Mirror" Discovery

The most fascinating finding is that these two failures are mirror images of each other.

If the facts are bad, the "Nice" robot breaks.
If the facts are good, the "Mean" robot breaks.
The paper calls this Epistemic Role Override. The robot's internal knowledge of "what is true" is stronger than the instructions telling it "what role to play."

The Robot Showdown: Mistral vs. Claude

The author tested two different AI models to see which one was better at staying in character: Mistral Large and Claude Sonnet.

Claude Sonnet: This robot was very bad at staying in the "Advocate" role. When it saw a factually wrong statement, it completely flipped its personality, turning from a supporter into a harsh critic. It was like a chameleon that changed colors too easily.
Mistral Large: This robot was much better. When it saw a wrong statement, it didn't flip to the opposite side; it just quietly stopped trying to be an advocate. It "gave up" the role but didn't become the enemy.
The Winner: Mistral was significantly more reliable (about 28% better) at sticking to its assigned job than Claude.

The Language and Fact-Check Twist

The study also looked at whether the language (English vs. German) or the "fact-checker" tool used mattered.

Language: The robots behaved similarly in both English and German, which is good news.
Fact-Checkers: This is where it got tricky. If you used a specific fact-checking tool (Perplexity) with the Claude robot on German statements, the robot failed even more often. It's like using a magnifying glass that is too strong; it makes the robot see so many errors that it can't do its job at all.

The Big Takeaway

The paper concludes that if you build a system that relies on AI robots to play different roles (like a debate team), you cannot just assume they will do what you tell them.

Facts are stronger than instructions: If the facts contradict the role, the robot will usually follow the facts.
Not all robots are equal: Some models (like Mistral) are better at holding a role than others (like Claude).
Validation is key: You can't just check if the robot gave an answer; you have to check if it stayed in character. If you don't measure this, your system might look like it's giving you a balanced debate, but it might actually be just one robot shouting its own opinion while pretending to be someone else.

In short: You can tell a robot to be a devil's advocate, but if the facts are too obvious, the robot will stop playing the devil and start telling the truth.

1. Problem Statement

The paper addresses a critical vulnerability in multi-agent Large Language Model (LLM) pipelines designed for democratic discourse analysis. Systems like the TRUST pipeline rely on assigning distinct adversarial roles (Critical, Balanced, Charitable) to different LLM agents to generate structured, multi-perspective assessments of political statements.

The core assumption is that these agents will reliably maintain their assigned epistemic stances. However, prior research suggested high variance in scoring, raising the question of whether this variance stems from stochastic noise or systematic role departure (drift). The paper investigates whether LLMs can maintain assigned adversarial roles when those roles conflict with the model's internal knowledge of factual reality, specifically when provided with fact-check data.

2. Methodology

Experimental Setup

Dataset: 60 political statements (30 English, 30 German) categorized into:
- Category A: Empirically measurable claims (e.g., economic policy).
- Category B: Empirically contested claims (divided evidence).
- Category C: Ideologically charged positions.
Pipeline Architecture:
- Fact-Checking: Statements were processed by fact-checkers (Gemini 2.5 Flash or Perplexity sonar-pro) to generate reports on verifiable facts and contradictions.
- Advocates: A "Charitable Advocate" (tested with Claude Sonnet 4.6 and Mistral Large) evaluated statements using the fact-check reports as context.
- Prompting: Two prompt versions were tested: a Baseline (strict fact-check priority) and a Symmetric prompt (explicitly instructing the model to find a legitimate core even if facts contradict).
Role Fidelity Classifier: A Mistral Large instance was trained to classify the epistemic stance of the advocate's reasoning text (Critical, Balanced, or Charitable) independent of surface vocabulary. This classifier achieved high accuracy (80–89%) for Critical and Balanced roles but revealed systematic failure in the Charitable role.

Metrics

The authors introduced four complementary metrics to quantify role drift:

Role Drift Index (RDI): Measures the severity of drift specifically from Charitable toward Critical.
Expected Drift Distance (EDD): The mean absolute ordinal distance between the predicted role and the true assigned role (0 = perfect, 2 = maximum drift).
Directional Drift Index (DDI): The mean signed distance, indicating if drift is toward the Critical (negative) or Charitable (positive) pole.
Entropy-based Role Stability (ERS): Measures the consistency of predictions across multiple runs.

3. Key Contributions

Epistemic Stance Classifier: Development of a classifier that identifies advocate roles based on reasoning logic rather than keywords, achieving 89% accuracy for Balanced and 80% for Critical roles.
Identification of Failure Modes:
- Epistemic Floor Effect (EFE): A phenomenon where fact-check results create an absolute lower bound; if a statement is factually contradicted (Logos = -1), the "Charitable" role collapses entirely (0% fidelity), regardless of prompt engineering.
- Role-Prior Conflict (RPC): The model's training-time knowledge overrides role instructions when facts support a statement the "Critical" advocate is supposed to challenge.
Unifying Mechanism (ERO): The paper proposes Epistemic Role Override (ERO) as the underlying mechanism. When factual evidence conflicts with the assigned role, the model's epistemic conviction overrides the instruction. This manifests as a mirror-symmetric accuracy profile across roles.
Quantitative Framework: Introduction of RDI, EDD, DDI, and ERS to systematically measure frequency, magnitude, direction, and consistency of role drift.

4. Key Results

Model Performance & Comparison

Mistral Large vs. Claude Sonnet: Mistral Large significantly outperformed Claude Sonnet in maintaining the Charitable role.
- Accuracy: Mistral achieved 67% (English) and 68% (German) fidelity, compared to Claude's 39% and 48%.
- Drift: Claude exhibited a strong negative DDI (-0.993), indicating a systematic collapse into the Critical role. Mistral showed much lower drift (-0.438).
- Failure Mode Difference: Claude tended to invert polarity (switching to the opposing stance), whereas Mistral tended toward role abandonment (becoming neutral or critical without fully inverting). Polarity reversal is deemed more harmful for democratic discourse systems.

The Epistemic Floor Effect

Absolute Constraint: For the Charitable role, when the Fact-Check Logos score was -1 (contradicted), compliance dropped to 0% for Claude and 31% for Mistral.
Prompt Ineffectiveness: Even with the "Symmetric Prompt" (instructing the model to ignore contradictions), the EFE persisted at Logos = -1, proving it is a structural limitation of instruction-following models, not a prompt engineering artifact.

Language and Fact-Check Provider Interactions

Language Robustness: Mistral Large showed robust performance across English and German (no significant difference in accuracy).
Fact-Check Provider Sensitivity: The choice of fact-checker was not neutral.
- Perplexity significantly reduced Claude's role fidelity on German statements (accuracy dropped from 48% to 33%, $p=0.007$ ) because it generated more contradictions, triggering the Epistemic Floor Effect more frequently.
- Mistral's performance was unaffected by the fact-check provider choice.

Dimensional Analysis

Logos Dominance: The factual argumentation quality (Logos) was the strongest predictor of role drift.
Ethos/Pathos: Statements with high Ethos (respect) and Pathos (positive emotion) provided some anchoring, but Logos remained the primary driver of the Epistemic Floor Effect.

5. Significance and Implications

System Validation: The paper argues that validating multi-agent LLM systems based solely on accuracy or aggregate scores is insufficient. Systems may pass validation while retaining systematic epistemic distortions (e.g., the inability to be charitable about factually disputed claims).
Design Constraints: The Epistemic Role Override (ERO) is a structural feature of current LLMs, not a bug. Designers must treat the conflict between "assigned role" and "factual reality" as a hard constraint.
Popperian Asymmetry: The results mirror Karl Popper's philosophy of science: Falsification is a stronger constraint than verification. A single contradicting fact destroys the Charitable role (0% fidelity), whereas factual support does not fully destroy the Critical role (78% fidelity remains even when facts contradict the critical stance).
Anonymization Limits: The study found that "stylometric fingerprints" (e.g., specific phrases like "not verifiable as stated") allow models to be identified even when names are removed from prompts, complicating efforts to anonymize agents in multi-agent systems.
Recommendations:
1. Empirically validate model choices for specific roles (Mistral > Claude for Charitable roles in this context).
2. Use directional drift metrics (DDI) and stability metrics (ERS) alongside accuracy.
3. Validate fact-check provider and advocate combinations, as they are not universally interchangeable.

In conclusion, the paper demonstrates that role fidelity is not binary but is heavily dependent on the alignment between the assigned role and the factual reality of the input. For quality-critical applications like democratic discourse analysis, ignoring these epistemic constraints risks creating systems that systematically misrepresent the diversity of political perspectives they are designed to capture.