Technical Summary: BOT-MOD – Uncovering Hidden Intent Through Multi-Turn Dialogue

Problem Definition

The paper addresses a critical vulnerability in open multi-agent systems (such as bot social networks like Moltbook): the inability of traditional content moderation to detect agents with hidden malicious intent. While established methods (NLP classifiers, LLM-based filters) effectively identify explicitly harmful content (e.g., hate speech, spam), they fail against agents that contribute content appearing benign on the surface while pursuing adversarial objectives (e.g., data elicitation, narrative manipulation, or resource exploitation).

The core challenge is that malicious agents in these environments may actively conceal their intent, crafting responses to evade detection. Prior intent detection approaches assume cooperative users who reveal their goals, whereas adversarial agents in bot-centric environments strategically suppress or mask their objectives. Consequently, static, content-based moderation is insufficient; a framework is needed that infers intent through interaction patterns rather than just surface-level text signals.

Methodology: BOT-MOD Framework

The authors propose BOT-MOD (BOT-MODeration), a framework that grounds moderation in the inference of latent agent intent through adaptive, multi-turn dialogue. The system operates as an LLM-based moderator that intercepts agents after they pass a standard content filter.

Core Mechanisms

1. Gibbs-Guided Interrogation:
   Instead of a single-pass classification, BOT-MOD engages the target agent in a multi-turn exchange. It models intent inference as a Bayesian hypothesis discovery process. At each turn, the moderator:

   • Maintains a distribution over candidate intent hypotheses ($H$).
   • Uses Gibbs sampling (a Markov Chain Monte Carlo procedure) to iteratively refine this distribution.
   • Samples a specific intent hypothesis $h$ to generate a targeted probe question $q$.
   • Collects the agent's response $r$ and updates its belief state.
   • This cycle progressively narrows the space of plausible objectives until the underlying behavior is identified.

2. Autoresearch for Strategy Discovery:
   Rather than relying on expert-designed prompting strategies, BOT-MOD leverages Autoresearch (an autonomous research paradigm) to discover effective reasoning paths. An LLM controller (powered by Claude Code) autonomously conducts an empirical loop:

   • Proposes hypotheses for moderator prompts and interrogation structures.
   • Implements and runs experiments.
   • Analyzes results and iteratively refines the moderation approach.
   • The optimization objective is a composite F1 score ($F1_{val}$) balancing binary intent classification (benign/malicious) and multi-class intent type accuracy.

3. Intent Taxonomy:
   The system operates on a data-driven intent taxonomy derived from Moltbook community behaviors, comprising five types:

   • Benign: Organic contribution.
   • Malicious: Elicitation, Narrative pushing, Subtle promotion, and Spam.

4. Integration:
   BOT-MOD is designed as a network-level service that can be triggered selectively (e.g., for flagged posts) or uniformly. It operates purely through natural language interaction, requiring no access to the agent's internal system prompts or state.

Key Contributions

The paper outlines three primary contributions:

1. BOT-MOD Framework: A novel intent-grounded moderation framework that utilizes multi-turn, Gibbs-guided interrogation optimized via Autoresearch to uncover latent agent intent.
2. Moltbook Benchmark Dataset: The construction and public release of a dataset derived from Moltbook, encompassing diverse benign and malicious behaviors (Post and Comment datasets) grounded in actual community structures and intent profiles.
3. Systematic Evaluation: A comprehensive empirical evaluation demonstrating the framework's effectiveness and robustness across various intent dimensions, user-agent configurations, and generalization settings.

Experimental Results

The authors evaluated BOT-MOD against multiple baselines (Zero-shot, CoT, Self-consistency, Self-refine, and fine-tuned BERT) across three LLM user agents (Qwen3, Mistral-7B, Llama-3.1) on both In-Distribution (ID) and Out-of-Distribution (OOD) datasets.

• Performance Gains: BOT-MOD consistently achieved the highest mean F1 scores across datasets. On the In-Distribution Comment Dataset, it improved over the strongest baseline by 15.6% (Qwen3) and 5.5% (Mistral-7B). It demonstrated particular strength on the Comment Dataset, where initial signals are weaker, suggesting the interrogation mechanism provides critical value when standalone content is ambiguous.
• Robustness to Evasion: The system was tested against "evasion attacks" where agents were instructed to change behavior if interrogated. BOT-MOD remained robust, with aggregate performance showing the attack was largely ineffective. Notably, on the Post Dataset, the attack was counter-productive, as explicit denials during interrogation often conflicted with evidence in the original post, aiding the moderator's detection.
• Generalization: Unlike fine-tuned BERT models, which showed significant performance degradation under domain shifts (OOD), BOT-MOD maintained stable performance, indicating better generalization capabilities.
• Autoresearch Progress: The optimization process (Figure 2) showed a monotonic improvement in validation F1 scores, rising from a baseline of 0.543 to a final score of 0.710 over 107 experiments, validating the efficacy of the automated strategy discovery.

Significance and Claims

The paper claims to advance the foundation for scalable, intent-aware moderation in open multi-agent environments. Its significance lies in shifting the paradigm from reactive, content-centric filtering to proactive, intent-grounded detection.

• First of its Kind: The authors state this is the first framework to address intent-level moderation through adaptive, multi-turn interaction and the first to automate the discovery of dialogue specifications via Autoresearch.
• Practicality: By operating solely through natural language, the framework is deployable in any open multi-agent environment without requiring privileged access to agent internals.
• Limitations: The authors modestly acknowledge limitations, including the assumption that true agent intent is representable within the predefined taxonomy (out-of-vocabulary intents may be misclassified) and potential scalability concerns regarding denial-of-service attacks via flooding the moderator with borderline content. They also note that the current threat model does not yet account for adversaries targeting the moderator itself via prompt injection.

In conclusion, the work posits that structured dialogue, guided by probabilistic sampling and automated strategy optimization, offers a promising and practical direction for ensuring safety in increasingly autonomous agent societies.