SEVADE: Self-Evolving Multi-Agent Analysis with Decoupled Evaluation for Hallucination-Resistant Irony Detection

The paper introduces SEVADE, a novel self-evolving multi-agent framework featuring a Dynamic Agentive Reasoning Engine and a decoupled adjudicator to enhance sarcasm detection accuracy and reliability by mitigating hallucinations through structured, multifaceted reasoning.

The Gist

Imagine you are trying to figure out if someone is being sarcastic. You know that sarcasm is tricky because people often say the opposite of what they mean, usually to be funny or to mock something. If you ask a standard computer program (or a basic AI) to detect this, it often gets confused. It might take the words literally, or it might just guess wrong because it's trying to do too much at once.

The paper you shared introduces a new system called SEVADE. Think of SEVADE not as a single super-smart robot, but as a highly organized detective agency working together to solve a mystery.

Here is how it works, broken down into simple concepts:

1. The Problem: The "One-Brain" Trap

Most current AI models try to read a sentence, think about it, and give an answer all in one go.

• The Analogy: Imagine asking a single person to act as a lawyer, a psychologist, a linguist, and a judge all at the same time. They might get overwhelmed, miss a subtle clue, or just make up a reason for their answer (which experts call "hallucination").
• The Result: The AI says, "This is sarcastic!" when it's actually just a serious argument, or vice versa.

2. The Solution: The Detective Agency (SEVADE)

SEVADE solves this by splitting the job into two distinct teams.

Team A: The Investigators (The "DARE" Engine)

Instead of one brain, SEVADE uses a team of specialized agents. Think of them as a squad of detectives, each with a specific superpower based on how humans use language:

• The Logic Detective: Checks if what the person said makes sense with real-world facts.
• The Tone Detective: Looks at the emotional vibe. Is the person angry when they should be happy?
• The "Common Sense" Detective: Checks if the statement violates basic rules of how we live.
• The Web Searcher: If the text is confusing, this agent goes online to find background info (like checking if a famous person actually said something).

How they work together:
They don't just give a final answer immediately. They have a dynamic meeting:

1. They all analyze the text.
2. If one detective is confused or unsure, the team leader asks them to rethink their opinion based on what the others said.
3. If the team is still stuck, the leader calls in a new specialist from the waiting room to offer a fresh perspective.
4. They keep refining their thoughts until they agree on a clear, step-by-step story of why they think the text is sarcastic or not.

Team B: The Judge (The "Rationale Adjudicator")

This is the most important part of the new design.

• The Analogy: Imagine a courtroom. The Investigators (Team A) present their evidence and their reasoning story to the Judge (Team B).
• The Twist: The Judge is not allowed to look at the original text again. They can only read the Investigators' written report.
• Why do this? This forces the Judge to make a decision based purely on the logic of the argument, not on a gut feeling or a random guess. It stops the AI from "hallucinating" (making things up) because the Judge has to stick to the facts provided by the team.

3. Why is this better?

The paper tested this system on four different sets of difficult text data. Here is why it won:

• No "One-Size-Fits-All": Because the team can add new detectives or change their strategy based on the specific text, they are flexible. If a joke is subtle, they dig deeper. If it's obvious, they wrap it up quickly.
• Fewer Mistakes: By separating the "thinking" (the team) from the "deciding" (the judge), the system is much less likely to make up reasons for its answers.
• Better at Hard Cases: On the hardest tests, SEVADE was about 7% more accurate than the best previous methods. That's a huge jump in the world of AI.

Summary

Think of SEVADE as a company that doesn't rely on one genius employee to do everything. Instead, it hires a team of experts to debate and refine an idea, writes down their final conclusion, and then hands that report to a strict judge who makes the final call based only on that report.

This "teamwork + strict judge" approach makes the AI much better at understanding human sarcasm, which is one of the hardest things for computers to figure out.

Technical Summary

Here is a detailed technical summary of the paper "SEVADE: Self-Evolving Multi-Agent Analysis with Decoupled Evaluation for Hallucination-Resistant Sarcasm Detection."

1. Problem Definition

Sarcasm detection is a critical yet difficult Natural Language Processing (NLP) task due to the semantic incongruity between literal and intended meanings, reliance on contextual cues, and pragmatic contrast. While Large Language Models (LLMs) have advanced this field, the authors identify three persistent limitations in existing LLM-based approaches:

• C1: Single-Perspective Reasoning: Standard LLMs act as monolithic predictors, lacking the ability to systematically deconstruct text from multiple linguistic dimensions.
• C2: Hallucination Risk: LLMs often synthesize conflicting signals into unreliable final judgments, leading to "hallucinated" conclusions that do not faithfully reflect the analytical process.
• C3: Static Reasoning Pathways: Current models rely on fixed prompts or architectures, failing to dynamically adapt their analytical strategies to the specific complexity of the input text.

2. Methodology: The SEVADE Framework

The authors propose SEVADE, a novel framework featuring a Self-Evolving Multi-Agent architecture with Decoupled Evaluation. The system consists of two primary stages:

A. Dynamic Agentive Reasoning Engine (DARE)

This is the core reasoning module designed to generate a structured, transparent reasoning chain. It operates through an iterative, self-evolving process:

1. Agent Pool: The system utilizes a pool of specialized agents grounded in linguistic theory:
   • Core Analysis Agents (6 types): Semantic Incongruity (SIA), Pragmatic Contrast (PCA), Rhetorical Device (RDA), Emotion Polarity Inverter (EPIA), Common Sense Violation (CSVA), and Persona Conflict (PeCA).
   • Support Agents: A Web Search Agent (WSA) for external context retrieval and a Summarization Agent (SA) for final synthesis.
2. Iterative Workflow:
   • Instantiation: A Controller Agent selects an initial team of agents relevant to the input text.
   • Targeted Refinement: In each cycle, the Controller identifies the "most ambivalent" agent (highest uncertainty in sarcasm intensity score) and prompts it to refine its analysis based on peer conclusions.
   • Adaptive Expansion: If the analysis is deemed incomplete or contradictory, the Controller recruits a new agent from an inactive pool to introduce a complementary perspective.
   • Termination: The loop continues until consensus is reached or the agent pool is exhausted.
3. Output: The Summarization Agent synthesizes all findings into a structured Reasoning Chain ($R$).

B. Rationale Adjudicator (RA)

To address the hallucination risk (C2), SEVADE employs a decoupled architecture:

• The final classification is not performed by the large LLM generating the reasoning.
• Instead, a lightweight, fine-tuned classifier (based on BERT) takes the Reasoning Chain ($R$) as its sole input.
• This forces the final decision to be grounded strictly in the logical coherence of the generated rationale, separating complex reasoning from the final judgment.

3. Key Contributions

1. Decoupled Multi-Agent Architecture: A novel framework that separates the generation of multi-perspective reasoning (via DARE) from the final decision-making (via RA), significantly reducing hallucination risks.
2. Self-Evolving Reasoning: The DARE module dynamically adapts its agent team composition and reasoning depth based on input complexity, moving beyond static prompt engineering.
3. Linguistic Grounding: The system integrates six specialized agents based on established linguistic theories (e.g., Gricean maxims, pragmatic contrast) to ensure a multi-faceted deconstruction of sarcasm.
4. State-of-the-Art Performance: The framework achieves new benchmarks across four datasets with superior robustness and generalization.

4. Experimental Results

The framework was evaluated on four benchmark datasets: IAC-V1, IAC-V2, MuSTARD, and SemEval-2018.

• Performance: SEVADE achieved a new State-of-the-Art (SOTA) across all datasets.
  • Average Accuracy: 78.14% (an improvement of 7.01% over the strongest baseline, DC-Net).
  • Average Macro-F1: 77.90% (an improvement of 6.55%).
• Comparison: It outperformed strong reasoning models like GPT-5 and various fine-tuned PLMs (BERT, RoBERTa) and deep learning baselines (MIARN, SAWS, DC-Net).
• Generalization: Cross-dataset testing showed SEVADE significantly outperformed BERT and RoBERTa (e.g., >27% improvement in Macro-F1 when transferring from IAC-V1 to SemEval), demonstrating robustness against domain shifts.
• Ablation Studies:
  • Removing any single Core Agent caused performance drops, confirming the necessity of multi-perspective analysis.
  • Disabling the "Evolving" mechanism (static analysis) resulted in substantial performance degradation, validating the need for dynamic adaptation.
  • Replacing the lightweight RA with the base LLM reduced performance, proving the value of the decoupled adjudication in mitigating hallucinations.

5. Significance and Conclusion

SEVADE represents a paradigm shift in sarcasm detection by moving away from monolithic "black-box" LLM predictions toward a transparent, interpretable, and adaptive multi-agent system.

• Interpretability: The framework provides a clear "reasoning chain" that explains why a text is deemed sarcastic, making the decision process auditable.
• Hallucination Resistance: By decoupling the reasoning generation from the final classification, the system ensures that judgments are based on logical consistency rather than the LLM's tendency to hallucinate when synthesizing complex signals.
• Adaptability: The self-evolving nature of DARE allows the system to handle varying levels of sarcasm complexity, from obvious rhetorical devices to subtle pragmatic contrasts, without requiring manual prompt engineering for each case.

The authors conclude that SEVADE effectively bridges the gap between the reasoning capabilities of LLMs and the rigorous, multi-dimensional requirements of sarcasm detection, offering a robust solution for real-world applications in sentiment analysis and content moderation.