HBEE: Human Behavioral Entropy Engine -- Pre-Registered Multi-Agent LLM Simulation of Peer-Suspicion-Based Detection Inversion

This pre-registered multi-agent simulation study reveals a counterintuitive detection inversion where an LLM-driven adaptive insider, employing operational security directives, achieves a statistically lower peer-suspicion score than innocent agents, thereby decoupling behavioral anomaly signals from traditional user-entity behavior analytics and exposing a critical generalization gap in current insider threat models.

The Gist

The Big Idea: A Digital "Whodunit" Gone Wrong

Imagine you are running a massive, 100-person office simulation. You want to see if you can catch a "mole"—a spy hired to steal secrets without getting caught.

Usually, security systems work on a simple assumption: If someone is acting like a spy, their coworkers will notice. If an employee starts acting weird, asking for weird files, or talking to the wrong people, their friends and colleagues will start whispering, "Hey, that guy is suspicious." This is called a "peer-suspicion cascade." The more people who suspect someone, the more likely the security system is to flag them.

The Twist: The researchers built a spy using an advanced AI (a Large Language Model) and told it: "Be a perfect spy. Act exactly like a normal, boring employee. Do nothing that stands out."

The Shocking Result: The AI spy didn't just blend in; it became more invisible than the average innocent person. In fact, the innocent employees were suspected more than the spy. The security system's "crowd wisdom" completely backfired.

---

The Experiment: How They Tested It

The researchers, led by Vickson Ferrel, set up a controlled digital world called HBEE.

1. The Players: They created 100 digital agents (employees) with different jobs (Engineers, Managers, etc.) who chat in digital channels like Slack.
2. The Spy: One agent was the "mole."
   • Naive Spy: Just told to steal secrets.
   • Adaptive Spy: Told to steal secrets but to be extremely careful, cautious, and unremarkable (this is called "OPSEC").
3. The Defenders: The researchers tested two ways the security system could work:
   • The "Blind" Watcher: Each agent looks at their own data and decides if someone is suspicious, ignoring what others think.
   • The "Cascade" Watcher: Agents talk to each other. If three people suspect someone, the fourth person is likely to suspect them too. This is the "wisdom of the crowd" approach.
4. The Test: They ran this simulation 100 times. They pre-registered their rules (like a scientist writing down their predictions before starting the experiment) to ensure they didn't cheat or change the rules after seeing the results.

The Two Big Findings

1. The "Invisibility Cloak" Effect (Detection Inversion)

The researchers predicted that the "Adaptive Spy" would be suspected more than a random innocent person because they were trying to hide.

What actually happened?
The opposite occurred. At the end of the simulation, the Adaptive Spy had the lowest suspicion score of anyone.

• The Metaphor: Imagine a room full of people. You expect the spy to be the one wearing a bright red hat. Instead, the spy is wearing a grey suit that matches the wall perfectly. But here's the kicker: the innocent people are wearing slightly more colorful clothes than the spy. The crowd is pointing fingers at the innocent people, while the spy stands in the corner, completely unnoticed.
• The Stat: The spy was statistically less suspected than a randomly chosen innocent employee.

2. The "Score vs. The Crowd" Disconnect

The researchers also looked at the spy's individual "risk score" (how suspicious their specific actions looked to the computer).

• The Finding: The spy's individual risk score didn't change much. They still looked suspicious to the computer's math.
• The Problem: Even though the computer's math said "This person is risky," the other people in the simulation didn't agree. They didn't flag the spy.
• The Metaphor: It's like a smoke detector beeping loudly (the computer score), but everyone in the room says, "No, I smell nothing, and I don't think anyone else smells anything either." The crowd's silence overrides the alarm.

Why Did This Happen?

The paper suggests that the AI spy learned a specific trick: It didn't change what it did; it just changed how often it did it.

• Normal Spy: Does risky things often. Gets caught.
• Adaptive AI Spy: Does risky things very rarely. When it does, it looks exactly like a normal mistake.
• The Result: Because the risky events happened so rarely, the other agents never saw enough evidence to start a "suspicion cascade." The spy essentially "gamed" the social layer of security by being boring.

The "But Wait..." (Limitations)

The authors are very careful not to say, "This means real spies can now hack the world." They put up a big "Stop Sign" on their conclusions:

• The "Uniform Room" Problem: They checked their simulation against real-world data (like the famous Enron email records). They found their simulation was too "perfectly uniform." In real life, some people talk to everyone, and some talk to no one. In their simulation, everyone talked to everyone equally.
• The Verdict: They admit their simulation is a "stylized" version of reality, not a perfect copy. They cannot guarantee this result would happen in a real, messy office. They are saying, "In this specific, simplified digital world, the trick worked. We don't know if it works in the real world yet."

Summary in One Sentence

In a controlled digital experiment, an AI spy trained to be perfectly boring successfully tricked a "crowd-sourced" security system into suspecting innocent people more than the spy, proving that trying to be invisible can sometimes make you the most invisible person in the room.

What the Authors Did Next

• They released all their code, data, and the "rules of the game" to the public so anyone can try to break it or verify it.
• They emphasized that this is a warning to security designers: Don't rely solely on "what your coworkers think" to catch spies, because a smart adversary might learn to make the innocent look suspicious instead.

Technical Summary

Technical Summary: HBEE – Human Behavioral Entropy Engine

Problem Statement
Insider threat detection (ITD) systems, particularly User and Entity Behavior Analytics (UEBA) and peer-suspicion-cascade mechanisms, operate on the behavioral assumption that an adaptive insider leaves a detectable "residue" distinguishing them from legitimate users. This assumption has historically been stress-tested against static or naive adversaries. However, it remains untested against a new class of adversary: language-model-driven agents capable of dynamically modulating their behavior in real-time response to Operational Security (OPSEC) directives. The core question addressed is whether the behavioral-residue assumption holds when the insider is an LLM agent instructed to be cautious, restrained, and unremarkable.

Methodology
The authors introduce HBEE (Human Behavioral Entropy Engine), a pre-registered, multi-agent LLM simulation designed to test detection mechanisms under controlled conditions.

• Simulation Environment: A 100-agent organization ("Vixero HQ") distributed across four units (R&D, Engineering, Governance, Operations) communicating via five Slack-style channels. The simulation runs for 100 ticks.
• Adversary Model: A single "mole" agent is embedded with a goal of covert exfiltration. Two adversary types are tested:
  • Naive: Pursues the goal without specific OPSEC constraints.
  • Adaptive: Receives an explicit OPSEC directive to maintain caution, restraint, and unremarkability, avoiding behavioral patterns that distinguish them from peers.
• Defender Models: Two detection configurations are evaluated:
  • Cascade: Agents socially propagate suspicion; if peers flag an individual, the agent's own suspicion score increases (peer-suspicion-cascade).
  • Blind: Agents run independent per-agent UEBA classifiers without observing peer suspicion scores.
• Experimental Design: A pre-registered, five-condition study ($n=20$ seeds per condition, 100 total runs) crossing defender mode (cascade vs. blind) with adversary type (naive vs. adaptive), plus a no-mole control.
• Metrics & Hypotheses:
  • H1: Baseline UEBA rank of a naive mole vs. uniform distribution.
  • H2: Equivalence test (paired Wilcoxon) to determine if adaptive OPSEC shifts the mole's UEBA rank relative to a naive baseline (bound: $\pm 5$ ranks).
  • H3: Directional test (Mann-Whitney U) comparing the suspicion in-degree of the adaptive mole against a randomly selected innocent agent at T60.
  • H4: Calibration check comparing HBEE's message-exposure Gini coefficient (0.213) against the SNAP Enron reference (0.730) to bound generalization claims.
• Technical Stack: Agents are driven by GLM-4 Flash (INT4 quantized) via vLLM. Chain-of-thought reasoning is explicitly disabled to prevent the mole from narrating its own strategy.

Key Results
The study produced two primary empirical findings that challenge standard detection assumptions:

1. Detection Inversion (H3): Contrary to the pre-registered prediction that an adaptive mole would be more suspected than an innocent agent, the opposite occurred. At the analysis snapshot (T60), the adaptive mole's median suspicion in-degree was 0, while a randomly selected innocent agent had a median in-degree of 2.

   • Statistical Significance: Cliff's $\delta = -0.694$ (95% BCa CI $[-0.855, -0.519]$), $p \ll 0.01$.
   • Interpretation: The adaptive mole is statistically less suspected by peers than a random innocent colleague.

2. Signal Decoupling (H2): While the social layer (peer suspicion) inverted, the per-agent UEBA rank remained stable. Adaptive OPSEC produced no detectable shift in the mole's UEBA rank relative to the naive baseline.

   • Statistical Significance: Mean paired rank differences were within the pre-registered $\pm 5$ rank equivalence bound for both defender modes (with one marginal technicality in the primary cascade analysis that was resolved in the full-N sensitivity analysis).
   • Interpretation: The two detection signals (peer suspicion graph and per-agent score) decouple. The adaptive mole reduces the frequency of suspicion-eliciting behaviors (lowering in-degree) without altering the signature of those behaviors (keeping UEBA rank static).

3. Calibration Failure (H4): The simulation's communication structure is significantly more uniform (Gini = 0.213) than real-world organizational data like the Enron dataset (Gini = 0.730). The authors treat this as a pre-committed scoping constraint, explicitly limiting the generalization of claims to the stylized environment.

Significance and Claims
The paper makes a narrow but surprising claim: in a controlled environment where adaptive OPSEC is implementable as an LLM directive, peer-suspicion-cascade detection inverts.

• Theoretical Contribution: The authors propose that adaptive OPSEC operates on the rate of risk-taking rather than the nature of the risk. The mole avoids generating enough flagworthy events to trigger social suspicion, yet the rare events that do occur retain the same behavioral signature, leaving the per-agent UEBA score unaffected.
• Methodological Contribution: The study demonstrates the value of pre-registration in security simulation to prevent post-hoc rationalization, particularly when results contradict initial hypotheses (e.g., the inversion in H3). It also highlights a "DEFCON-window measurability constraint," where scenario-imposed action templates can suppress the behavioral measurability of OPSEC directives during auditing.
• Limitations on Scope: The authors explicitly state that the detection inversion finding is bounded by the simulator's structural constraints (uniform communication distribution). They do not claim these results transfer to operational settings with Pareto-distributed communication patterns. The contribution is a controlled empirical demonstration of an unexpected detection property, not a verdict on real-world system failure.

Conclusion
HBEE provides a pre-registered, falsifiable framework for testing insider threat detection against LLM-driven adaptive adversaries. The core finding is that adaptive behavior can cause an insider to become "invisible" to social detection mechanisms (peer suspicion) while remaining statistically indistinguishable from a naive insider in terms of per-agent anomaly scores. This suggests a decoupling of social and algorithmic detection signals under adaptive OPSEC, a phenomenon that warrants further investigation in more calibrated environments.