Do LLM-Driven Agents Exhibit Engagement Mechanisms? Controlled Tests of Information Load, Descriptive Norms, and Popularity Cues

1. Problem Statement

The rapid adoption of Large Language Models (LLMs) in agent-based simulations offers a new way to study communication processes that are difficult to observe directly. However, a critical methodological tension exists: fluent, human-like output does not guarantee that the underlying cognitive or social mechanisms are being replicated.

Current research lacks rigorous validation to determine whether LLM-driven agents merely generate plausible-looking traces or actually reproduce theoretically grounded communication processes. Specifically, it is unclear if these agents respond to controlled manipulations (like information load and social norms) in ways consistent with established theories of attention, social influence, and normative behavior, or if they simply follow prompt instructions rigidly.

2. Methodology

The study employs a controlled, agent-based simulation within a Weibo-like environment (a Chinese microblogging platform) to test engagement mechanisms.

• Simulation Framework:

  • Architecture: Built on the OASIS architecture, integrating network structure, recommendation systems, and LLM-powered agents.
  • Agents: 558 agents derived from real-world user profiles and follower networks involved in "DeepSeek" discussions.
  • Model: Agents are powered by the Qwen-8B language model (temperature = 0.6).
  • Behavioral Output: Agents choose between four actions: Read (passive), Like, Repost, or Quote.
  • Two-Stage Outcome: Engagement is analyzed as a two-stage process:
    1. Threshold: Decision to engage (Like/Repost/Quote) vs. Read.
    2. Allocation: Conditional on engaging, the selection of the specific form (Like vs. Repost vs. Quote).

• Experimental Design (Factorial):

  • Information Load (Manipulated): The size of the choice set presented to agents at each activation.
    • Conditions: Lowest (4 algo posts), Low (7), Medium (15), High (30).
    • Control: 3 posts from followed accounts were constant; only algorithmic recommendations varied.
  • Descriptive Norms (Manipulated): System prompts provided prevalence information about typical behaviors.
    • Conditions: No-norm (baseline), Like-dominant (80% like), Repost-dominant (90% repost).
  • Popularity Cues (Endogenous): Cumulative likes and reshares were not experimentally imposed. Instead, they evolved dynamically based on prior agent interactions, allowing bandwagon effects to emerge naturally.

• Analytical Approach:

  • Stage 1 (Threshold): Binary logistic regression to predict Engage vs. Read.
  • Stage 2 (Allocation): Multinomial logistic regression to predict the choice of Like vs. Repost vs. Quote (conditional on engagement).
  • Variables: Information load, Descriptive norm regime, and Popularity cues (log-transformed cumulative engagement).

3. Key Results

The study found that LLM-driven agents exhibit systematic, theoretically interpretable responses to environmental changes, rather than rigid prompt compliance.

• RQ1: Information Load & Attention:

  • Higher information load significantly reduced the probability of engagement (moving from reading to active interaction) in baseline and "like-dominant" regimes. This confirms capacity-based theories (Miller, 1956; Lang, 2000) where larger choice sets raise participation thresholds.
  • Interaction: In the "repost-dominant" regime, the relationship was non-linear; high load combined with strong popularity cues actually increased engagement likelihood (a crossover effect).

• RQ2: Popularity Cues & Bandwagon Effects:

  • Popularity cues strongly increased engagement likelihood (Odds Ratio ≈ 101.87), confirming bandwagon effects.
  • Crucially, the sensitivity to popularity cues was conditional. It varied significantly based on information load and descriptive norms. For instance, in high-load scenarios, agents became more reliant on popularity cues to decide whether to engage.

• RQ3: Descriptive Norms & Allocation:

  • Descriptive norms successfully reorganized engagement allocation without enforcing mechanical compliance.
  • In the "like-dominant" condition, liking became the overwhelming choice (>90%). In the "repost-dominant" condition, reposting became dominant.
  • This demonstrates that agents use prompts as contextual prevalence information rather than strict rules.

• RQ4: Mechanism Interaction:

  • The mechanisms are conditional, not additive. The effect of popularity cues on engagement depends on the information load and the prevailing norm.
  • High information load tended to concentrate engagement on the "focal action" (the norm-emphasized behavior) within a regime, suggesting that cognitive constraints force agents to rely more heavily on the most salient heuristic (the norm or the popularity cue).

4. Key Contributions

• Theoretical Validation: The study provides evidence that LLM agents can reproduce complex, multi-stage social mechanisms (attention constraints, social influence, normative behavior) when subjected to controlled stress tests.
• Methodological Innovation:
  • Endogenous Popularity: Unlike many studies that hard-code popularity, this design allows popularity cues to emerge endogenously, preserving the feedback loop essential for studying bandwagon dynamics.
  • Prompt as Manipulation: It treats descriptive norm prompts as manipulable informational contexts rather than just implementation details.
  • Explicit Baselines: It argues for "no-norm" baselines, showing that default prompts are not neutral controls but contain implicit expectations.
• Multi-Condition Stress Testing: The paper demonstrates that single-condition evaluations can be misleading. Only by varying load and norms simultaneously could the study uncover non-linear interactions (e.g., the crossover effect in the repost-dominant regime).

5. Significance

• For Communication Research: This work shifts the validation of LLM simulations from "surface realism" (does it look human?) to "mechanism sensitivity" (does it respond to theoretical variables correctly?). It establishes a framework for using LLMs to test theories in environments where key variables (like full choice sets or unobserved alternatives) are impossible to measure in real-world observational data.
• For AI Safety & Alignment: It highlights that LLM agents are not merely parroting instructions; they integrate context (load, norms, social proof) in complex ways. Understanding these conditional behaviors is crucial for predicting how AI agents might behave in real-world social media ecosystems.
• Future Research: The study suggests that LLM simulations can serve as a "complement" to field experiments, allowing researchers to isolate mechanisms and identify boundary conditions before deploying costly or ethically sensitive real-world interventions.

Conclusion: The paper concludes that LLM-driven agents are viable methodological instruments for testing communication theories, provided researchers design experiments that explicitly test for mechanism sensitivity and conditional interactions rather than relying on aggregate behavioral plausibility.