PRISM of Opinions: A Persona-Reasoned Multimodal Framework for User-centric Conversational Stance Detection

This paper introduces PRISM, a persona-reasoned multimodal framework and the accompanying U-MStance dataset, to address the limitations of pseudo-multimodality and user homogeneity in conversational stance detection by leveraging longitudinal user personas and chain-of-thought reasoning for more realistic, user-centric attitude interpretation.

The Gist

Imagine you are walking through a bustling, chaotic town square (the internet). People are shouting opinions about everything from politics to the latest car models. Some are holding signs (text), while others are holding up photos or memes (images) to make their points.

The goal of this research paper is to build a super-smart "Town Square Observer" who can listen to these arguments, look at the pictures, and figure out exactly what each person really thinks about a specific topic.

Here is the story of how the authors built this observer, called PRISM, and why the old way of doing things wasn't good enough.

The Problem: The "Blindfolded" Observer

The researchers found that previous attempts to build these observers had two major flaws:

1. The "One-Sided" View (Pseudo-multimodality):
   Imagine the observer could see the pictures only on the main stage (the original post), but when people in the crowd shouted back (comments), the observer had to pretend those people were just speaking text. In real life, people often reply with memes or photos too! The old observers were blind to this, missing half the story.
2. The "Cookie-Cutter" Crowd (User Homogeneity):
   The old observers treated everyone in the crowd as if they were the same person. They didn't realize that "Grumpy Grandpa" is naturally more critical than "Optimistic Teen." If you don't know who is speaking, you might misinterpret a sarcastic joke as a serious attack, or a genuine compliment as a fake one.

The Solution: A New Map and a New Detective

To fix this, the team did two big things:

1. The New Map: U-MStance Dataset

They created a massive new library of real conversations called U-MStance.

• What's special? It contains over 40,000 comments where both the original posts and the replies can have pictures.
• The Twist: They also included the "history books" of the people speaking. They know what these users have posted for years, so they can build a profile of who they are.

2. The New Detective: PRISM

They built a new AI framework named PRISM (Persona-Reasoned multImodal Stance Model). Think of PRISM as a detective with three special tools:

• Tool 1: The "Personality X-Ray" (Longitudinal User Persona)
  Before the detective listens to the current argument, they look at the speaker's history. They use a psychological framework (the "Big Five" personality traits) to build a profile.

  • Analogy: If the speaker is usually "Neurotic" (emotional and anxious), the detective knows to look for emotional triggers. If they are "Open" (adventurous), they might be more willing to try new ideas. This helps the AI understand why someone is saying what they are saying.

• Tool 2: The "Translator" (Rationalized Cross-Modal Grounding)
  When someone posts a picture, it's not just a picture; it's a joke, a threat, or a compliment wrapped in an image.

  • Analogy: Imagine someone posts a picture of a burning house with the caption "Great weather!" A normal AI might just see "House" and "Weather." PRISM uses a "Chain of Thought" (a step-by-step reasoning process) to say: "Wait, the house is burning, but they are smiling. They are being sarcastic. This picture is actually an insult to the situation." It translates the visual joke into a clear meaning.

• Tool 3: The "Practice Run" (Mutual Task Reinforcement)
  To get really good at guessing what people think, PRISM practices a second game: writing the next comment.

  • Analogy: It's like a student trying to understand a math problem. If they can also teach the problem to someone else (generate a response), they understand the logic much better. By training the AI to both guess the stance and write a reply, it learns the nuances of human conversation much faster.

The Results: Why It Matters

When they tested PRISM against other smart AI models:

• It won. It was much better at figuring out if someone was for or against a topic, especially when sarcasm or complex images were involved.
• It was adaptable. Even when the AI was tested on topics it had never seen before (like switching from talking about "Cars" to "Cryptocurrency"), it still performed well because it understood the people behind the words, not just the words themselves.

The Bottom Line

This paper teaches us that to truly understand human opinion online, we can't just look at the text or the pictures. We have to look at who is speaking, what they usually stand for, and how they use images to hide or reveal their true feelings. PRISM is the first tool that brings all these pieces together to give us a clear, human-like understanding of the digital crowd.

Technical Summary

Here is a detailed technical summary of the paper "PRISM: A Persona-Reasoned Multimodal Framework for User-centric Conversational Stance Detection."

1. Problem Statement

The paper addresses two critical limitations in existing Multimodal Conversational Stance Detection (MCSD) research:

• Pseudo-multimodality: Current datasets and models typically treat source posts as multimodal (text + image) but restrict conversational replies (comments) to text-only. This misaligns with real-world social media interactions where users frequently reply with images, memes, or videos.
• User Homogeneity: Existing approaches treat all users as a uniform group, ignoring individual personality traits, historical behavior, and discourse tendencies that fundamentally shape how a user expresses their stance.

These limitations lead to models that fail to capture the nuances of conflicting viewpoints and the pragmatic intent behind multimodal cues in complex conversations.

2. Key Contributions

The authors propose two major contributions to bridge these gaps:

1. U-MStance Dataset: The first user-centric MCSD dataset containing over 40,000 annotated comments across six real-world targets (Trump, Biden, Tesla, BMW, Costco, Bitcoin).
   • Uniqueness: Unlike previous datasets, it incorporates multimodality in both source posts and conversational replies and includes rich user information (historical posts/comments) to enable persona modeling.
2. PRISM Framework: A Persona-Reasoned multImodal Stance Model designed to detect stances by integrating user personality and cross-modal reasoning.

3. Methodology: The PRISM Framework

PRISM operates through three core components, leveraging a Multi-Modal Large Language Model (MLLM):

A. Longitudinal User Persona Distillation

• Goal: To capture stable individual traits that influence stance expression.
• Mechanism: The model aggregates a user's complete historical activity (posts and comments) and uses an MLLM to infer their personality based on the Big Five (OCEAN) model (Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism).
• Output: A structured persona vector ($p_{u_N}$) consisting of numerical scores (1–5) for each trait. This persona is injected into the final stance detection prompt to condition the prediction on the user's identity.

B. Rationalized Cross-Modal Grounding (RCMG)

• Goal: To bridge the semantic and pragmatic gap between text and images in a conversational context.
• Mechanism: Utilizing Chain-of-Thought (CoT) reasoning, the module performs a two-stage process for every image in the conversation:
  1. Objective Description: Generates a factual description of the visual content.
  2. Intent-aware Interpretation: Combines the objective description, the current conversation text, and the image to infer the speaker's rhetorical intent (e.g., sarcasm, criticism, support).
• Output: An "intent-aware caption" ($\hat{x}_i$) that replaces raw visual features, explicitly grounding the image's meaning within the dialogue.

C. Mutual Task Reinforcement

• Goal: To jointly optimize stance detection and user communication style modeling.
• Mechanism: A multitask learning objective that trains the model on two correlated tasks:
  1. Primary Task (Stance Detection): Predicting the stance label ($y_N$) of the final comment.
  2. Auxiliary Task (Response Generation): Generating the text of the final comment based on the context and the predicted stance.
• Optimization: The total loss is a weighted sum ($L_{total} = \lambda L_{cls} + (1-\lambda)L_{gen}$). This forces the model to learn a holistic representation where understanding the user's style (generation) improves stance prediction, and vice versa.

4. Experimental Results

Experiments were conducted on the U-MStance dataset using Qwen2.5-VL-7B as the backbone, compared against strong baselines (BERT, RoBERTa, LLaMA2, GPT-4, LLaVA, MiMo, etc.).

• In-Target Performance: PRISM achieved the highest F1-avg of 68.49%, significantly outperforming the best baseline (GPT4-1 at 66.24%) and text-only models (GPT-4 at 60.74%).
• Cross-Target Generalization: In scenarios where training and testing targets differ (e.g., training on Trump, testing on Biden), PRISM maintained robust performance (55.45% F1-avg), whereas smaller models suffered significant degradation. This highlights the model's ability to learn target-invariant stance representations via persona and intent modeling.
• Ablation Studies: Removing any of the three core components (Persona, Intent-aware grounding, or Mutual Reinforcement) resulted in consistent performance drops, confirming the necessity of each module.
• Backbone Adaptability: PRISM showed consistent gains when integrated with different MLLM backbones (LLaVA, MiMo), demonstrating its generalizability.

5. Significance and Impact

• Paradigm Shift: The paper moves the field from "content-centric" analysis to "user-centric" analysis, acknowledging that who is speaking is as important as what they are saying.
• Realistic Benchmarking: By introducing U-MStance, the authors provide a benchmark that reflects the true multimodal nature of social media, where images are active participants in arguments, not just static backgrounds.
• Interpretability: The use of CoT for image grounding and explicit persona modeling offers a more interpretable path to understanding why a model predicts a specific stance, particularly in cases involving sarcasm or implicit criticism.
• Future Direction: The work establishes a foundation for more nuanced social media analysis, potentially applicable to polarization detection, misinformation spread analysis, and personalized content recommendation.

Limitations

The authors note that performance declines as conversational depth increases (beyond 5 turns), suggesting challenges in capturing long-range logical transitions in extremely deep threads. Additionally, the dataset currently lacks coverage of highly specialized professional sectors (e.g., legal or scientific controversies).