💻 computer science

ProCrit: Self-Elicited Multi-Perspective Reasoning with Critic-Guided Revision for Multimodal Sarcasm Detection

This paper proposes ProCrit, a novel proposal-critic framework that enables autonomous, self-elicited multi-perspective reasoning and targeted revision guided by an external critic to enhance multimodal sarcasm detection by overcoming the limitations of fixed, predefined analytical perspectives.

Original authors: Yingjia Xu, Jiulong Wu, Bowen Zhang, Baokui Guo, Siyuan Chai, Min Cao

Published 2026-05-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yingjia Xu, Jiulong Wu, Bowen Zhang, Baokui Guo, Siyuan Chai, Min Cao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out if a friend is being sarcastic. They post a picture of a traffic jam with the caption, "What a perfect day!"

Detecting sarcasm in images and text is tricky because the "joke" can come from many different angles. Sometimes it's about a cultural reference, sometimes it's about how the text contradicts the image, and sometimes it's about an exaggerated tone. The problem is that every single sarcastic post is different, so you can't use the same checklist for every one.

The paper introduces a new system called ProCrit to solve this. Think of ProCrit not as a single robot, but as a two-person detective team working together to solve a mystery.

The Two Detectives: The "Proposal" and The "Critic"

The Proposal Agent (The Detective):
This is the thinker. When it sees a picture and text, it doesn't just guess "Yes" or "No." Instead, it acts like a detective who says, "Okay, let's look at this from a few different angles."
- The Old Way: Previous systems were like detectives who were forced to use a fixed checklist (e.g., "Check tone," "Check image," "Check grammar") for every single case, even if the case didn't need all those checks.
- The ProCrit Way: This detective is smart enough to invent its own checklist for every specific case. If a joke relies on a historical reference, it invents a "History Check." If it relies on a visual contradiction, it invents a "Visual Check." It builds its reasoning step-by-step, using the clues from the first step to inform the second.
The Critic Agent (The Editor/Coach):
This detective is the "second pair of eyes." Once the Proposal Agent writes down its reasoning and makes a guess, the Critic steps in.
- The Critic doesn't just say "Right" or "Wrong." It acts like a strict editor who reads the detective's notes and says, "Hey, you missed the fact that the car in the background is on fire! That changes everything," or "You focused too much on the text and ignored the sad face in the photo."
- The Critic gives specific, natural language feedback on what was missed or misunderstood.

The "Draft–Critique–Revise" Loop

Here is how they work together, like a writer and an editor:

Draft: The Proposal Agent writes its first draft of the reasoning and makes a guess.
Critique: The Critic Agent reads it, finds the holes, and writes a note saying, "You missed this specific clue."
Revise: The Proposal Agent takes that feedback, throws away the old draft, and writes a brand new analysis from scratch, making sure to fix the specific mistakes the Critic pointed out.

How They Learned to Do This (The "Training" Part)

The paper notes that real-world data (like social media posts) usually just has a "Yes/No" label, not a step-by-step explanation of how to find the sarcasm. It's like having a test with an answer key but no explanation of the math.

To fix this, the researchers created a special training method:

The "Dynamic Role" Simulation: They used a super-smart AI to simulate a team of experts. One expert would look at the text, the next would look at the image, the next would check the tone, and so on. They passed notes back and forth until they solved the puzzle.
Flattening the Notes: They took all those back-and-forth notes and turned them into a single, long story. This taught the Proposal Agent how to "think" in a chain of logic without needing a human to tell it what to do next.
Mutual Refinement: They trained the two detectives to get better together. The Proposal Agent gets better at fixing mistakes because the Critic gives better feedback. The Critic gets better at giving feedback because it sees which of its notes actually helped the Proposal Agent fix the problem. It's a feedback loop where they both level up.

The Results

When they tested this team on three different sets of social media data, ProCrit outperformed all other methods.

It was better at catching the "tricky" sarcasm that other systems missed (improving its ability to find the "Yes" cases).
It showed that having a "Critic" who gives specific feedback is much better than just letting the AI try to fix its own mistakes alone (which the paper says is often unreliable).

In short: ProCrit is a system that teaches an AI to be a flexible detective that invents its own investigation strategy for every case, and then has a strict editor review its work to ensure it didn't miss any clues.

Technical Summary: ProCrit – Self-Elicited Multi-Perspective Reasoning with Critic-Guided Revision

Problem Statement

Multimodal sarcasm detection requires identifying incongruities between literal expressions and intended meanings across text and image modalities. Existing approaches face two primary limitations:

Rigid Perspective Selection: Current methods often rely on fixed, predefined sets of analytical perspectives (e.g., specific reasoning templates) governed by hand-crafted routing rules. These static frameworks fail to adapt to the diverse mechanisms of sarcasm (e.g., verbal irony, hyperbole, cultural allusion), which vary significantly across samples.
Lack of Process-Level Supervision and Reliability: Existing datasets typically provide only binary labels or brief explanations, lacking annotations that capture the step-by-step analytical process. Furthermore, models attempting self-correction often fail to reliably detect their own reasoning flaws, particularly when dealing with subtle, implicit cross-modal cues.

Methodology: ProCrit Framework

The authors propose ProCrit, a Proposal–Critic two-agent framework designed to enable self-elicited multi-perspective reasoning. The system autonomously generates the necessary analytical perspectives for each sample and progressively integrates them into a coherent analysis, guided by an external critic.

1. Process-Level Reasoning Annotation Synthesis

To address the absence of process-level supervision, ProCrit synthesizes reasoning annotations using a dynamic-role agentic rollout:

Dynamic-Role Rollout: A strong vision-language model (Teacher, $T$ ) sequentially spawns analytical roles within a shared context. Each role contributes a distinct perspective based on preceding analyses and determines if the current evidence is sufficient for a final prediction.
Trajectory Flattening: The resulting multi-turn trajectory is flattened into a single autoregressive sequence. This preserves cross-perspective dependencies while eliminating the need to re-encode prior context at every step, enabling efficient training of a student model to generate sample-adaptive perspectives in one pass.

2. Draft–Critique–Revise Paradigm

To enhance reasoning reliability, ProCrit decouples generation from evaluation:

Draft: The Proposal Agent generates an initial multi-perspective reasoning draft and a binary prediction.
Critique: An independent Critic Agent evaluates the draft against the original multimodal evidence. It provides targeted natural-language feedback identifying specific deficiencies and a quality score.
Revise: The Proposal Agent uses this feedback to generate a completely new reasoning analysis from scratch, rather than merely editing the draft.

3. Mutual-Refinement Training Strategy

The framework employs a dual-stage training process where the agents alternately optimize each other:

Proposal Agent Optimization: Trained via Dual-Stage Group Relative Policy Optimization (GRPO).
- Stage 1 (Draft): Optimized using rewards for accuracy, format adherence, and the critic's quality score.
- Stage 2 (Revision): Optimized using rewards for accuracy and an "improvement reward" that specifically credits corrections that fix previous errors while penalizing harmful changes.
Critic Agent Optimization: Trained via GRPO to improve scoring calibration and feedback utility. Rewards are based on format, score alignment with ground truth, and actionability (whether the feedback successfully leads the Proposal Agent to a correct revision).
Initialization: Both agents are first initialized via Supervised Fine-Tuning (SFT) on synthesized data before entering the reinforcement learning loop.

Key Contributions

Self-Elicited Multi-Perspective Reasoning: ProCrit shifts multimodal sarcasm detection from manually prescribed perspectives to a dynamic, sample-adaptive approach where the model autonomously determines which analytical angles are needed.
Process-Level Annotation Synthesis: The introduction of a dynamic-role agentic rollout strategy to generate progressive, sample-specific reasoning trajectories, providing the necessary process-level supervision for training.
Critic-Guided Revision with Mutual Refinement: A novel draft–critique–revise paradigm that utilizes an independent critic for external evaluation. The framework employs a mutual-refinement strategy where the proposal agent's revision outcomes reciprocally refine the critic's feedback quality, optimizing both reasoning generation and critique utility.

Experimental Results

ProCrit was evaluated on three widely used benchmarks: MMSD, MMSD2.0, and RedEval.

Performance: ProCrit achieved the strongest overall performance across all benchmarks under both zero-shot and fine-tuned settings.
- On MMSD2.0, the fine-tuned ProCrit achieved an F1 score of 83.1, outperforming the best predefined-perspective baseline (Fixed-role Reasoner, 79.8) and the plain Chain-of-Thought baseline (79.4).
- On MMSD, ProCrit reached an F1 of 80.7 (fine-tuned), surpassing the draft-only version (78.4).
- On RedEval, ProCrit achieved the highest recall (93.7), significantly improving the detection of subtle sarcastic instances missed by draft-only reasoning.
Ablation Studies:
- Adaptive vs. Fixed: Adaptive perspective generation consistently outperformed both "None" (plain CoT) and "Predefined" (fixed roles) settings.
- Critic Impact: The inclusion of the critic-guided revision step provided substantial gains, particularly in recall, demonstrating its ability to recover missed cues.
- Training Stages: Both SFT and Reinforcement Learning (RL) stages were shown to be complementary; removing either led to performance degradation.
- Revision Efficiency: A single revision round captured the dominant performance improvement, with diminishing returns observed in subsequent rounds.

Significance and Claims

The paper claims that ProCrit represents a significant step forward by reformulating multimodal sarcasm detection to be sample-adaptive rather than relying on static, hand-crafted reasoning templates. By synthesizing process-level supervision and employing a mutual-refinement training strategy, the framework addresses the inherent difficulty of detecting subtle, cross-modal incongruities.

The authors emphasize that the critic-guided revision mechanism is crucial for reliability, as it provides external signals to correct reasoning flaws that models cannot reliably self-correct. The framework demonstrates that explicitly modeling the analytical process and allowing the model to dynamically select perspectives leads to more coherent and accurate sarcasm detection compared to existing state-of-the-art methods. The work also highlights the efficiency of the approach, noting that the draft-only version already achieves competitive performance, offering a practical trade-off between accuracy and inference cost.