Trust Me, I Can Convince You: The Contextualized Argument Appraisal Framework

This paper introduces the Contextualized Argument Appraisal Framework, which integrates psychological appraisal models to analyze how subjective factors like familiarity and emotional response influence argument convincingness, validated through a novel role-playing annotation setup that produced the 4,000-entry ContArgA corpus.

The Gist

Imagine you are sitting in a crowded town hall meeting. Someone stands up and says, "We should ban plastic bottles to save the environment."

In a perfect world, you would just look at the logic of that statement and decide, "Yes, that makes sense," or "No, that's silly." But in the real world, your brain doesn't work like a calculator. It works like a security guard combined with a personal historian.

This paper, titled "Trust Me, I Can Convince You," is about understanding exactly how that security guard and historian work together when you hear an argument.

The Big Idea: It's Not Just About the Words

The authors argue that whether an argument convinces you isn't just about the quality of the speech (the "logos"). It's about how that speech hits your personal life.

Think of an argument like a package delivered to your door.

• The Argument: The package itself.
• The Sender: The person who mailed it (a politician? a friend? a stranger?).
• The Receiver (You): You, standing on your porch.

If you work at a plastic bottle factory, that "save the environment" package might feel like a threat. Your security guard screams, "Danger! I might lose my job!" You feel fear or anger. Because you feel threatened, the argument feels less convincing, even if the environmental facts are 100% true.

If you are a parent worried about your child's health, that same package feels like a gift. Your security guard says, "Safe! This helps my family!" You feel relief or trust. Now, the argument feels very convincing.

The New Tool: The "Contextualized Argument Appraisal Framework"

The researchers built a new map (a framework) to track this process. They call it the Contextualized Argument Appraisal Framework (CAAF).

Instead of just asking, "Is this argument good?", they ask a series of deeper questions, like a detective interviewing a witness:

1. Familiarity: "Have I heard this before, or is it a shock?"
2. Pleasantness: "Does this feel good or bad to my gut?"
3. Consequences: "If I believe this, what happens to my life? Do I lose money? Do I gain safety?"
4. Effort: "Is this argument hard to think about, or does it flow easily?"

They realized that emotions (like anger, joy, or fear) are the result of these questions being answered. And the answer to "How convincing is this?" depends entirely on those emotional results.

The Experiment: A Role-Playing Game

To test this, the researchers didn't just ask people to read a text. They turned it into a role-playing game.

Imagine you are an actor in a play.

• The Setup: You are told, "You are at a town hall meeting. A speaker is about to talk."
• The Twist: They tell you who the speaker is (e.g., "A young woman" or "An older man").
• The Action: The speaker says an argument.
• The Reaction: You have to pause and report:
  • "How do I feel right now?" (Angry? Trusting? Scared?)
  • "Why do I feel this way?" (Because of the speaker? Because of the topic? Because of my own life?)
  • "How much do I believe this?"

They did this with 4,000 different reactions from real people on 800 different arguments.

What They Found (The "Aha!" Moments)

After analyzing all this data, they found some very clear patterns:

1. Positive Emotions = High Persuasion: When people felt trust, relief, or joy, they were much more likely to be convinced. It's like when a friend gives you advice; if you feel good, you listen.
2. Negative Emotions = Low Persuasion: When people felt anger, disgust, or fear, they were much less likely to be convinced. It's like when a stranger yells at you; you immediately put up a wall.
3. The "Familiarity" Factor: The most important thing wasn't the speaker's age or gender; it was whether the argument felt familiar. If an argument felt like something you already knew or agreed with, it was much more convincing.
4. The Argument Itself is King: While the speaker matters, the content of the argument was the biggest driver of emotion. However, how you interpreted that content depended on your own personal history.

Why This Matters

For a long time, computers (AI) have tried to judge arguments by looking only at the text, like a robot reading a dictionary. They often miss the human element.

This paper says: "Stop looking at just the text. Look at the person reading it."

If we want to build better AI that understands persuasion, or if we want to understand why people believe fake news or political slogans, we need to model this inner security guard. We need to understand that an argument isn't just a string of words; it's a key that either unlocks a feeling of safety or triggers a feeling of danger.

In short: You can't convince someone just by having the best facts. You have to understand their fears, their hopes, and their history. If you don't, the argument will bounce right off them.

Technical Summary

Here is a detailed technical summary of the paper "Trust Me, I Can Convince You: The Contextualized Argument Appraisal Framework and the ContArgA Corpus."

1. Problem Statement

The paper addresses a significant gap in Natural Language Processing (NLP) regarding argument mining and emotion analysis. While substantial research exists on the logical structure (logos) and overall convincingness of arguments, the emotional appeal (pathos) and the subjective cognitive appraisal of arguments remain underexplored.

Key challenges identified include:

• Implicit Nature of Emotion: Arguments rarely explicitly state the emotions they intend to evoke; emotional responses are often implicit and context-dependent.
• Subjectivity: The convincingness of an argument depends not just on the text, but on the interplay between the sender, the receiver (annotator), and the context.
• Lack of Appraisal Models in NLP: While psychological appraisal theories (which model how cognitive evaluation leads to emotion) exist, they have not been adapted to the specific domain of argumentation. Existing work often treats emotion as binary or categorical without considering the cognitive evaluation process (e.g., perceived consequences, familiarity, effort).

2. Methodology

A. Theoretical Framework: Contextualized Argument Appraisal Framework (CAAF)

The authors propose the CAAF, a model based on perspectivism that simulates the cognitive process of a receiver evaluating an argument. The framework posits that:

1. Contextual Inputs: The receiver evaluates the argument text, the sender's traits, and the situational context.
2. Appraisal: The receiver cognitively appraises the argument based on specific dimensions (adapted from psychology).
3. Outcome: This appraisal triggers a discrete emotion and a judgment of convincingness.

Key Appraisal Dimensions (15 total):

• Adapted from Troiano et al. (2023): Suddenness, Familiarity, Pleasantness/Unpleasantness, Internal/External Check.
• Novel Dimensions for Arguments: Suppression, Consequential Importance, Positive/Negative Consequentiality, Consequence Manageability, Response Urgency, Cognitive Effort, Argument Internal/External Check.

B. Corpus Creation: ContArgA

To validate the framework, the authors created the ContArgA corpus using a novel role-playing annotation setup.

• Data Source: 800 arguments sampled from the UKPConvArgv1 and IBM-Rank-30k datasets, covering 39 diverse topics (e.g., abortion, plastic bottles, retirement).
• Annotation Setup:
  • Role-Playing: Participants were immersed in a simulated "town-hall meeting" scenario to mimic real-world exposure.
  • Sender Manipulation: The age (old/young) and gender (male/female) of the argument's speaker were systematically manipulated to test their influence.
  • Receiver Data: Participants provided demographics, personality traits (Big Five via TIPI), initial stance, and current emotional state.
  • Task: For each argument, participants reported:
    • Dominant Emotion: From a set of 12 categories (Ekman's basic + shame, guilt, pride, trust, relief).
    • Emotion Intensity: 1–5 scale.
    • Appraisal Scores: Ratings for all 15 appraisal dimensions.
    • Convincingness: Perceived persuasiveness (1–5 scale).
    • Stance Change: Measured before and after exposure.
    • Reasoning: Free-text explanation for the emotion.
• Scale: 4,000 individual annotations (5 per argument) from ~3,900 unique participants (filtered for quality via attention checks).

C. Data Analysis

The authors analyzed correlations between:

• Discrete emotions and convincingness.
• Appraisal dimensions and convincingness.
• Contextual factors (sender/receiver traits) and emotional response.

3. Key Contributions

1. The CAAF Framework: The first psychological framework specifically designed to model the interplay between sender, receiver, and argument in NLP, bridging the gap between argument mining and appraisal theory.
2. The ContArgA Corpus: A large-scale, high-quality dataset (4,000 annotations) that includes not just argument text, but rich metadata on the receiver (personality, stance), the sender (demographics), and fine-grained cognitive appraisals.
3. Novel Annotation Paradigm: A role-playing-based approach that moves beyond static text labeling to simulate dynamic, contextual argument exposure.
4. Empirical Validation: The first NLP study to empirically demonstrate how specific appraisal variables (e.g., familiarity, pleasantness) drive emotional responses and convincingness in arguments.

4. Key Results

A. Correlation with Convincingness

• Positive Emotions: Strong positive correlation with convincingness. Trust ($r=0.58$), Relief ($r=0.51$), Pride ($r=0.47$), and Joy ($r=0.45$) are the strongest predictors of high convincingness.
• Negative Emotions: Strong negative correlation. Anger ($r=-0.22$) and Disgust ($r=-0.23$) significantly lower perceived convincingness. Fear showed no significant correlation, contradicting some prior psychological studies suggesting fear increases persuasion.
• Appraisals:
  • Pleasantness ($r=0.58$) and Positive Consequentiality ($r=0.40$) are highly correlated with convincingness.
  • Familiarity ($r=0.34$) is a crucial factor; arguments perceived as familiar are more convincing.
  • Negative Appraisals: "Argument External Check" (violating norms) and "Unpleasantness" show strong negative correlations ($r \approx -0.50$).

B. Drivers of Emotional Response

• Primary Driver: The argument text itself is the most significant factor driving emotional response (avg. importance score 3.6), followed by the receiver's own characteristics (3.1), and finally the sender (1.9).
• Sender Influence: While the sender's demographics (age/gender) were manipulated, they had a minimal impact on the type of emotion evoked compared to the argument content. However, specific emotions like Guilt were rarely attributed to the sender.

C. Inter-Annotator Agreement

• Agreement was intentionally low (Krippendorff's $\alpha = 0.06$ for convincingness), which the authors argue validates the perspectivist nature of the data. It confirms that convincingness and emotion are highly subjective and individual, rather than objective properties of the text.

5. Significance and Future Work

• Theoretical Impact: The paper establishes that affective responses to arguments are not superficial but are deeply tied to cognitive appraisal processes (e.g., "Does this argument threaten my goals?").
• NLP Applications:
  • Improved Convincingness Prediction: Integrating appraisal variables and discrete emotions can improve automated argument quality assessment.
  • Content Moderation: Understanding how specific appraisals (e.g., "Negative Consequentiality") trigger anger or disgust can help identify harmful or polarizing content.
  • Personalized Persuasion: Models can be tailored to individual user profiles (personality, stance) to predict how they will react to specific arguments.
• Limitations: The study is limited to English and a specific demographic (UK/Ireland). Future work aims to adapt the framework for multi-turn conversations and other languages.

In conclusion, this work provides a foundational shift from analyzing arguments as static text to analyzing them as dynamic cognitive events, offering a robust framework and dataset for the next generation of persuasive AI systems.