Collaborative Evaluation of Deepfake Text with Deliberation-Enhancing Dialogue Systems

1. Problem Statement

The proliferation of Large Language Models (LLMs) has made distinguishing between human-authored and machine-generated (deepfake) text increasingly difficult. While automated detectors exist, they are often brittle, easily fooled by paraphrasing, and struggle with unseen models. Conversely, human evaluators alone perform only slightly better than random guessing, even with training.

The paper addresses the gap in collaborative deepfake detection. While prior work suggests group collaboration improves detection accuracy, the mechanisms that make group deliberation effective in adversarial settings are underexplored. Specifically, the authors investigate whether an AI-driven deliberation-enhancing bot can improve group dynamics and detection performance without providing direct answers or task-specific knowledge.

2. Methodology

A. Dataset Curation

• Structure: The authors curated 14 articles, each containing three paragraphs: two written by humans and one generated by an LLM.
• Models Used: To balance difficulty and reflect current capabilities, the dataset utilized GPT-2 (older, easier to detect) and GPT-3.5 (newer, harder to detect). Half the dataset was regenerated using GPT-3.5 to ensure a mix of easy and challenging tasks.
• Domain: Political news articles were selected as they represent high-stakes scenarios where deepfakes cause significant societal harm and elicit strong reasoning.
• Validation: The dataset was filtered for coherence using T5-based cosine similarity and evaluated by three SOTA LLMs (GPT-3.5, LLAMA2-70B, Claude-2) to ensure a balanced difficulty distribution.

B. System: DeepFakeDeLiBot

The study introduces DeepFakeDeLiBot, a deliberation-enhancing dialogue agent adapted from the original DeLiBot (Karadzhov et al., 2023).

• Goal: To foster constructive group discussion, encourage balanced participation, and prompt reflection without providing the correct answer or specific domain knowledge.
• Architecture:
  • Retrieval: A retrieval-based agent that tracks dialogue history (last 5 utterances) and retrieves probing utterances from a database using Sentence-T5 embeddings and cosine similarity.
  • Selection: A Tactics-Strategy classifier estimates which retrieved utterance will yield the highest performance gain.
  • Generation: A fine-tuned Flan-T5 model refines the selected utterance to ensure accurate references to specific users and choices (contextual adaptation).
• Training Data: The bot was trained on a custom dataset created by manually annotating deepfake-related conversations and using GPT-3.5-Turbo for in-context learning to generate synthetic probing pairs.

C. Experimental Design

• Participants: 49 freelancers recruited via Upwork (fluent in English, diverse backgrounds).
• Phases:
  1. Individual Phase: Participants solved 14 detection tasks individually.
  2. Group Phase: Participants were grouped into 20 teams (2–3 members). They discussed the same 14 tasks.
• Conditions: A between-subjects design where 10 groups interacted with DeepFakeDeLiBot, and 10 groups did not (control).
• Metrics:
  • Performance: Accuracy of identifying the machine-generated paragraph.
  • Dynamics: Participant engagement, even participation, consensus formation, frequency of probing utterances (moderation, solution, reasoning), and diversity of solutions/reasoning.

3. Key Contributions

1. First Deliberation Agent for Deepfakes: Introduced the first conversational agent specifically designed to enhance group deliberation for deepfake text detection.
2. Empirical Validation of Collaboration: Provided statistical evidence that group problem-solving significantly outperforms individual efforts in deepfake detection.
3. Insight into Bot Impact: Demonstrated that while the bot does not significantly boost accuracy directly, it substantially improves collaboration dynamics (engagement, consensus, reasoning diversity).
4. Conditional Effectiveness: Identified that the bot's value is contingent on participants' perception of group collaboration effectiveness; it adds value primarily when the group already perceives collaboration as effective.

4. Results

RQ1: Detection Performance

• Group vs. Individual: Group collaboration improved detection accuracy by approximately 8–9% compared to individual efforts (statistically significant, $p < 0.001$).
• Bot vs. No Bot: Groups using DeepFakeDeLiBot achieved slightly higher accuracy (57.43%) than those without (54.76%), but this difference was not statistically significant ($p = 0.482$).

RQ2: Collaboration Dynamics

Groups interacting with DeepFakeDeLiBot exhibited significantly better group dynamics:

• Engagement: Higher participation rates ($p=0.006$).
• Consensus: Higher likelihood of reaching a consensus ($p=0.005$).
• Reasoning: Significantly higher frequency of reasoning-driven probing utterances ($p=0.001$).
• Diversity: Greater diversity in discussed solutions ($p=0.02$).
• Predictors: Linear regression showed that consensus formation, solution diversity, and reasoning diversity were strong positive predictors of performance gain.

RQ3: Conditions for Effectiveness

• User Perception: The bot significantly improved performance for individuals who perceived group collaboration as effective (interaction term coefficient = 9.64, $p=0.034$). For those who found collaboration unhelpful, the bot offered no benefit.
• Interaction Patterns: Groups with performance gains showed higher bot engagement rates (8% vs. 5%) and more frequent moderation-probing utterances from the bot. However, these metrics did not strongly predict performance in regression models.
• Qualitative Findings: The bot failed in some groups due to:
  1. Temporal Misalignment: Intervening after consensus was reached.
  2. Lack of Receptiveness: Participants ignoring the bot.
  3. Accelerated Task Completion: Groups moving too fast for the bot to intervene.

5. Significance and Conclusion

The study highlights that human-in-the-loop collaboration is a robust strategy for deepfake detection, outperforming individual efforts. While the deliberation-enhancing bot did not directly increase detection accuracy in a statistically significant way, it successfully acted as a catalyst for productive group dynamics.

The findings suggest that AI agents in high-stakes decision-making should focus on facilitating the deliberation process (encouraging diverse reasoning and consensus) rather than providing answers. The effectiveness of such agents is highly context-dependent, relying on the group's existing willingness to collaborate and the timing of the agent's interventions. Future work should explore larger sample sizes and a broader range of LLMs to generalize these findings.