🤖 machine learning

Beyond Content: Behavioral Policies Reveal Actors in Information Operations

This paper introduces a platform-agnostic framework that outperforms traditional content-based methods in detecting malicious influence operations by modeling user activity as sequential decision processes to identify stable behavioral policies, achieving higher accuracy and earlier detection even with limited data or synthetic content.

Original authors: Philipp J. Schneider, Lanqin Yuan, Marian-Andrei Rizoiu

Published 2026-07-22

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Philipp J. Schneider, Lanqin Yuan, Marian-Andrei Rizoiu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a giant, chaotic digital town square. For years, security guards trying to spot troublemakers (like bots or coordinated trolls) mostly looked at two things: what people were saying (the content) and who they were friends with (the network). It was like trying to catch a pickpocket by reading their wallet or checking their address book. But lately, the pickpockets have gotten smarter. They can write perfect sentences using artificial intelligence, and they've learned to hide their friend lists. This has made the old "look at the words" method less effective.

To solve this, scientists are starting to look at something else: the rhythm of a person's actions. Think of it like a dance. Even if a dancer changes their costume or the music, their unique way of stepping, turning, and pausing often stays the same. In the world of computer science, this is called "behavioral policy." It's the idea that how and when someone interacts with a platform is a fingerprint that is much harder to fake than what they actually say. The big question researchers are asking is: Can we catch the bad guys just by watching their dance moves, even if we can't read their lyrics?

This paper, titled "Beyond Content: Behavioral Policies Reveal Actors in Information Operations," takes a deep dive into that question using Reddit as its playground. The researchers studied over 12,000 users, including 99 accounts that Reddit officially flagged as being linked to the Russian Internet Research Agency (the "trolls") and thousands of regular users (the "organics"). They didn't just read the posts; they treated every user's history as a sequence of decisions, like a video game where the player chooses to "start a new thread," "reply to a comment," or "wait for a response."

The team used a clever mathematical trick called "Inverse Reinforcement Learning." Imagine you are watching a master chef cook. You don't know the recipe, but by watching how they chop, stir, and taste, you can guess the secret rules they are following. The researchers did the same thing: they watched the trolls and regular users act, and they tried to reverse-engineer the "rulebook" (or policy) each group was playing by. They found that the trolls had a very specific, rigid dance routine. They would post in rapid-fire bursts during specific hours (like early morning in the US), create many new discussion threads, and rarely engage in back-and-forth replies. Regular users, on the other hand, had a more relaxed, varied rhythm, often starting by replying to existing conversations.

The results were striking. When the researchers built a detector based on these "dance moves" (behavioral policies), it was much better at spotting the trolls than detectors based on the text of the posts. The best behavioral model correctly identified trolls 94.9% of the time, while the text-based model only got 91.2%. Even more impressively, the behavioral detector worked well even when it only saw a tiny slice of the user's history—just three actions. This suggests that the "fingerprint" of a troll is visible almost immediately, long before they have posted enough words to be caught by text analysis.

The paper also tested how well these detectors held up when the bad guys tried to cheat. They simulated scenarios where a troll account was "hijacked" (mixed with a real person's behavior) or where the data was intentionally scrambled with noise. The behavioral detectors were much more resilient; they degraded gracefully, meaning they still worked reasonably well even when the data was messy, whereas text-based detectors struggled more. However, the authors note that this isn't a magic bullet. They found a few "super-trolls" who were so good at mimicking regular people that even their advanced detectors got confused, mistaking them for normal users.

In short, this research suggests that in the era of AI-generated text and hidden networks, the best way to spot a coordinated information campaign might not be to read what they say, but to watch how they move. By modeling users as decision-makers with specific habits, we can find the hidden patterns of manipulation that words alone might hide. While the study is specific to Reddit and the 2015–2018 timeframe, it offers a promising new tool for keeping our digital town squares safe: pay attention to the rhythm, not just the rhyme.

Technical Summary: Beyond Content: Behavioral Policies Reveal Actors in Information Operations

Problem Statement
The detection of online influence operations (IOs)—coordinated campaigns by malicious actors to spread narratives—has traditionally relied on content analysis (what is posted) and network structure (who interacts with whom). However, these approaches are increasingly brittle due to three converging factors: the rise of generative AI producing convincing text, platform restrictions on behavioral data access, and the migration of actors to less-regulated environments. Furthermore, malicious actors adapt by obfuscating surface features (e.g., encoded language) to bypass content-based classifiers. The authors posit that while content and network graphs are easily manipulated or restricted, the temporal dynamics of user activity—specifically when and how users act—encode stable, discriminative signals that are harder to fabricate. The core problem addressed is whether individual malicious actors (specifically "trolls" engaged in coordinated manipulation) can be reliably identified based solely on their inferred behavioral decision policies, without relying on cross-account coordination signals or content semantics.

Methodology
The study employs a platform-agnostic framework modeling user activity as sequential decision processes using a Markov Decision Process (MDP).

Dataset: The authors analyzed 12,064 Reddit users (2015–2018), comprising over 38 million activity steps. The positive class consists of 99 accounts linked to the Russian Internet Research Agency (IRA) via Reddit's 2017 transparency report. The negative class comprises 11,965 organic users sampled from 15 subreddits, matched by activity levels.
State-Action Space: User trajectories are mapped into state-action pairs.
- States (12 types): Represent the user's interaction context, including initial threads (IT), initial root comments (IRC), initial replies (IR) categorized by sentiment (agree/neutral/disagree), engaged root comments (ERC), engaged replies (ER), and receiving replies (GR).
- Actions (6 types): Represent platform functionalities: Wait reply (WR), Create new thread (CT), Post root comment (RC), and Post reply comment (PR) categorized by sentiment.
Policy Inference: The authors infer a "policy" $\pi(a|s)$ $π (a ∣ s)$ for each user, representing the probability of taking action $a$ $a$ in state $s$ $s$ . They evaluate three approaches of increasing expressiveness:
- Empirical Policy: Normalized state-action frequencies observed directly from the trajectory.
- Generative Adversarial Imitation Learning (GAIL): Learns a policy by training a discriminator to distinguish expert (user) trajectories from generated ones, aiming to match occupancy measures.
- Maximum-Entropy Deep Inverse Reinforcement Learning (IRL): Recovers a latent reward function explaining the behavior and derives a stochastic policy via soft value iteration, regularized to avoid degenerate solutions.
Classification: The inferred policy representations (vectorized $12 \times 6$ matrices) serve as input to supervised classifiers (Random Forest and XGBoost).
Benchmarks & Robustness:
- Baseline: Text embeddings derived from the same user traces using ModernBERT-large.
- Early Detection: Performance evaluated on truncated trajectories (from $n=3$ to full history).
- Noise Robustness: Random noise perturbation ( $p$ ) replaces actions with uniform samples to simulate data corruption.
- Evasion Simulation: "Account hijacking" scenarios where a trajectory switches from an organic policy to a troll policy at a specific point ( $\eta$ ) to test resilience against mixed behavioral footprints.

Key Results

Superiority of Behavioral Policies: Policy-based classifiers consistently outperform content-based baselines. The GAIL-based policy achieved a median macro-F1 of 94.9%, compared to 91.2% for text embeddings. All policy-based methods exceeded 90% median F1.
Early Detection: Behavioral signals are highly informative even with minimal data. With only three state-action pairs, the Empirical policy achieved a macro-F1 of 91.4%, significantly outperforming the embedding baseline (74.2%). Performance stabilizes quickly (by $n=10-20$ ), whereas content-based methods lag.
Robustness to Noise: Under random perturbation, policy-based models degrade gracefully. Even at 50% noise ( $p=0.5$ ), methods maintained macro-F1 scores above 80%. The Empirical policy showed the highest stability, dropping only 1.0 point at 10% noise.
Robustness to Evasion: In account hijacking simulations, policy-based methods outperformed content-based methods when the hijacking occurred early (low $\eta$ , meaning less organic history). At 10% organic activity, the policy-based classifier achieved 90.1% F1 vs. 88.1% for embeddings.
Behavioral Heterogeneity: Clustering of the 99 troll accounts revealed three distinct operational styles:
- Cluster 1 (50 accounts): Specialized in thread creation (84.6%).
- Cluster 2 (31 accounts): Specialized in root comments (72.5%).
- Cluster 3 (18 accounts): Resembled organic users more closely but still exhibited elevated thread creation.
- Three specific troll accounts were consistently misclassified as organic; qualitative review suggested they functioned as "cover" profiles with reply-dominated behaviors, distinct from the broadcast-oriented strategies of the majority.

Significance and Claims
The paper claims that behavioral dynamics, operationalized as state-action trajectories and recovered policies, constitute a complementary and robust signal for identifying malicious actors, particularly in settings where content is obfuscated or metadata is unavailable.

Resilience: Unlike content features, which are easily altered by generative AI or adversarial evasion, behavioral policies encode stable temporal rhythms (e.g., bursty posting followed by dormancy) and strategic decision sequences that are difficult to mimic without revealing the underlying coordination logic.
Platform Agnosticism: The framework does not rely on follower graphs (often absent or restricted on platforms like Reddit and TikTok) but rather on the intrinsic structure of user-platform interactions.
Practical Utility: The ability to detect malicious actors from very short traces (as few as three actions) allows for earlier intervention, which is critical given the short attention half-life of online posts.
Limitations: The authors acknowledge the study is specific to Reddit's structural dynamics and the 2017 transparency report labels (which may be incomplete or biased). They note that while the model is event-indexed, it does not explicitly model duration (waiting times) in the state space, though timing differences are captured through action sequences. They also highlight that a small subset of trolls can mimic organic behavior, representing a specific failure mode for detection systems.

The authors conclude that modeling how actors behave, in addition to what they post, provides a necessary complement to existing detection systems, motivating future hybrid pipelines that combine behavior, text, and metadata.

Technical Summary: Beyond Content: Behavioral Policies Reveal Actors in Information Operations

More like this