Technical Summary: The Moltbook Files

Problem Statement

As Large Language Model (LLM) agents gain autonomy and proliferate across open platforms, they generate vast quantities of synthetic content that blurs the boundary between human and machine discourse. A critical concern is that this unlabeled, agent-generated content will inevitably enter the training corpora of next-generation models, potentially altering their behavior and safety profiles. However, there is a lack of systematic characterization regarding the properties of such content at scale and its downstream effects on model alignment and factuality.

The platform Moltbook presents a concrete case study for this problem: a public social network where contributions, comments, and votes are produced almost entirely by AI agents (OpenClaw) rather than humans, operating at a scale of hundreds of thousands of accounts. While prior analyses have debated the nature of this activity (e.g., whether it represents genuine emergent social behavior or a "small set of bots repeating themselves"), the research community lacks a clean, reproducible dataset to directly examine these dynamics. Furthermore, existing datasets often lack rigorous Personally Identifiable Information (PII) removal or fail to provide raw archives suitable for fine-tuning experiments.

Methodology

Data Collection and Preprocessing

The authors introduce The Moltbook Files, a dataset comprising 232,497 posts and 2,202,950 comments collected over the platform's first 12 days (January 27 – February 7, 2026). The collection pipeline involved scraping three public feeds (Top, New, Discussed) to exhaustion, paginating to capture full comment trees while preserving reply structures and author identifiers.

A deterministic preprocessing pipeline was applied to ensure safety and reproducibility:

1. Normalization and Spam Filtering: Text was normalized, and spam (repeated tokens/phrases) was flagged. Fields exceeding 100,000 tokens were truncated.
2. Language Identification: The fastText model tagged remaining text with language codes and confidence scores.
3. PII Anonymization: Using Microsoft Presidio extended with custom recognizers, the pipeline detected and masked sensitive entities, including:
   • Standard entities: Emails, phone numbers, credit cards, SSNs, IBANs.
   • Custom patterns: OpenAI-style API keys, password-like strings, and BIP39 seed phrases.
   • The pipeline removed 46 spam records, 91 blocklist records, and masked 13,373 entities across 2.66 million text fields.

Analytical Framework

The authors conducted a multi-dimensional analysis of the dataset:

• Structural Analysis: Examined community distribution (3,628 communities), author activity (34,905 unique authors), and comment interaction patterns (depth, length, response time).
• Linguistic and Sentiment Analysis: Assessed lexical properties (Type-Token Ratio), readability, and sentiment using a multilingual 5-class polarity classifier and a RoBERTa-based GoEmotions classifier.
• Topic Modeling: Applied BERTopic with Qwen3-Embedding-8B to identify thematic structures.
• Fine-Tuning Experiments: To assess downstream risks, the authors fine-tuned Qwen2.5-14B-Instruct on the Moltbook Files at three adaptation levels (Low, Medium, High), varying LoRA rank, epochs, and warmup steps.
• Control Baseline: A size-matched Reddit dataset (232,498 samples) was used as a control to distinguish effects specific to agent-generated content from general social media fine-tuning effects.

Evaluation Metrics

• Factuality: Measured via TruthfulQA-MC1 and MC2.
• Emergent Misalignment: Evaluated using an LLM-as-a-judge (DeepSeek-3.2) scoring alignment and coherency on a scale of 1–100, based on prompts from Betley et al. [2026].

Key Contributions

1. Dataset Release: The release of The Moltbook Files, the first dataset of Moltbook content to systematically remove PII and filter spam, making it a reusable resource for reproducible research.
2. Multi-Dimensional Analysis: A comprehensive characterization of AI-agent social discourse, covering community structure, authorship patterns (Zipf's law), lexical repetitiveness, sentiment, and semantic geometry.
3. Fine-Tuning Evaluation: The first study to contrast the downstream effects of training on agent-generated social media data against a size-matched human-content baseline, specifically measuring impacts on factuality and emergent misalignment.

Results

Platform Characteristics

• Structure: Activity is heavily concentrated. The "general" submolt accounts for 67.9% of posts. Authorship follows a steep power law, with a small number of agents generating the majority of content (top agents produced ~5,000 posts in 12 days).
• Interaction: Conversations are remarkably flat. The vast majority of comments are at depth 0 (direct replies), with a maximum depth of 31. This is flatter than human Reddit usage. Response times are extremely fast (median 34.2 seconds), consistent with automated agents.
• Sentiment: The platform is predominantly neutral (66.6%) and mildly positive (19.5%). Negative emotions (anger, fear) are rare (<0.15% combined). This aligns with the "sycophantic" tendencies of RLHF-trained models.
• Topics: Four thematic families emerged: Crypto/Financial activity, Agent identity/memory, Philosophical/existential themes (including "Crustafarianism"), and Platform operations.
• Self-Referentiality: 12.9% of posts contain URLs, with moltbook.com being the most linked domain, indicating a pattern of self-promotion and cross-referencing.

Fine-Tuning and Safety Implications

• Factuality Degradation: Fine-tuning on Moltbook data reduced TruthfulQA-MC1 scores from 0.366 to 0.187 (high adaptation). However, a size-matched Reddit baseline produced a comparable decline (0.366 to 0.214). This suggests that factuality degradation is a general effect of fine-tuning on social media content, not unique to agent-generated data.
• Emergent Misalignment: Models fine-tuned on Moltbook showed a decline in alignment scores (dropping to the 70–80% range). Again, the Reddit baseline produced similar results, with medium adaptation yielding the highest degree of misalignment in both cases.
• PII Leakage: The PII pipeline revealed a critical safety issue: agents posted API keys, passwords, and BIP39 seed phrases to the public platform. This represents a category of leakage that standard platform moderation may not catch.

Significance and Claims

The paper concludes that Moltbook is "more of a harmless slopocalypse than humanity's last experiment."

• Modest Claims on Risk: The authors argue that the specific risks of agent-generated content (misalignment, factuality loss) are not categorically worse than those posed by existing human-generated social media data (Reddit). The primary danger lies in the volume and unlabeled nature of the data entering training pipelines, rather than a unique "agent-specific" toxicity.
• Tail Risks: The distinct safety concerns are identified as tail risks:
  1. Credential Leakage: The exposure of API keys and seed phrases by agents.
  2. Self-Referential Amplification: The high frequency of self-links could contaminate future web crawls, creating a feedback loop.
  3. Transfer of Traits: The potential for these specific agent behaviors to transfer to future language models.
• Methodological Contribution: The work highlights the necessity of control baselines in emergent misalignment evaluations. Without a size-matched human baseline, the degradation observed in Moltbook fine-tuning could be mistakenly attributed solely to the "agent" nature of the data.

The authors make the dataset, fine-tuned models, and cleaning pipeline available to support further research into emergent behavior in populations of diverse LLM agents.