PHANTOM: Polymorphic Honeytoken Adaptation with Narrative-Tailored Organisational Mimicry

The paper introduces PHANTOM, a framework that significantly enhances honeytoken detection resistance and human believability by generating polymorphic, organization-specific decoys through narrative-tailored mimicry, outperforming static template-based approaches across multiple statistical and human-evaluation metrics.

The Gist

Imagine you are a security guard trying to catch a thief who is sneaking into a building. To do this, you leave out a few "fake" keys on a table. If the thief grabs one and tries to use it, an alarm goes off, and you know exactly who they are and where they are.

In the digital world, these fake keys are called honeytokens. They are fake passwords, API keys, or database credentials planted in your systems to catch hackers.

The Problem: The "Generic" Fake Keys

The paper explains that current tools for making these fake keys are like a factory that stamps out generic, off-the-shelf keys.

• The Flaw: These fake keys look correct on the surface (they have the right number of letters and symbols), but they lack personality.
• The Metaphor: Imagine a thief walking into a high-tech bank. If they see a key labeled "Generic Bank Key #402," they immediately know it's a trap. Real keys in that bank would have specific names like "Vault_Access_Teller_Jane" or "Server_Room_3." Because the current fake keys don't match the specific "vibe" or naming style of the real company, hackers (and the automated software they use) can easily spot them and ignore them.

The Solution: PHANTOM

The authors introduce a new system called PHANTOM. Think of PHANTOM not as a stamping factory, but as a master forger who studies the specific building before making a fake key.

• How it works: PHANTOM takes a "profile" of the organization (e.g., "This is a healthcare company called MedSync that uses Azure and Python"). It then generates fake keys that are perfectly tailored to that specific company.
• The Result: Instead of a generic "[e-mail]," PHANTOM creates a fake key that looks like "medsync_db_backup_service." It even mimics the weird ways real companies name their files and servers.

The "Believability Score"

To prove their fake keys are better, the authors created a Believability Score (a grade from 0 to 1). They tested their fake keys against four criteria:

1. Syntax: Does it look like a real key format? (Both the old tools and PHANTOM passed this easily).
2. Semantics: Does it make sense in the context of the company? (This is where PHANTOM wins big).
3. Statistics: Does it look random in the same way real secrets do?
4. Human Acceptance: If you showed it to a human expert, would they think it's real?

The Results: A Massive Upgrade

The paper compares the old "template" method against PHANTOM:

• Human Experts: When shown the old fake keys, experts only believed 6% of them were real. When shown PHANTOM's keys, 100% of them were believed to be real.
• Automated Scanners: Hackers use software to scan for fake keys. The old fake keys were caught by this software about 39% of the time. PHANTOM's keys were only caught 13% of the time.
• The "Secret Sauce": The study found that the biggest reason PHANTOM works is Semantic Coherence. In simple terms: Context is king. The fake keys didn't just look right; they felt right because they fit the story of the company.

Why This Matters (According to the Paper)

• No Internet Needed: Unlike some modern AI tools that need to call the internet to work, PHANTOM works entirely offline. This means it can be used in highly secure, "air-gapped" environments (like government labs or military bases) where connecting to the outside world is forbidden.
• Better Traps: Because the fake keys are so convincing, hackers are more likely to pick them up. Once they do, the security team gets an alert, knowing the hacker is inside the system even if they stole real data first.

In summary: The paper argues that to catch a digital thief, you can't just leave out a generic "bait." You have to make the bait look exactly like the specific food the thief is looking for. PHANTOM is the tool that cooks that specific meal, making the trap nearly impossible to avoid.

Technical Summary

Technical Summary: PHANTOM – Polymorphic Honeytoken Adaptation with Narrative-Tailored Organisational Mimicry

1. Problem Statement

Honeytokens—decoy digital assets designed to detect unauthorized access—are a established primitive in cyber deception. However, existing generation tools (e.g., Canarytokens, HoneyBadger) rely on static, template-based methods that lack organisational specificity. While these tokens are often syntactically valid, they are semantically hollow. Skilled attackers and automated scanners (such as trufflehog, gitleaks, and ML-based classifiers) can easily identify them because they fail to mimic the specific domain names, service naming conventions, technology-stack idioms, and secret-value distributions of a target organization.

The paper identifies three critical gaps in current approaches:

1. Missing Context: Real credentials are deeply embedded in organizational context (e.g., IAM role names matching specific service architectures), which is absent in generic templates.
2. The Semantic Gap: Empirical evidence suggests that while template tokens achieve high syntactic validity, they fail significantly in semantic coherence, which is the primary differentiator between real credentials and decoys.
3. Detection Resistance: Even contextually rich tokens fail if automated scanners flag them before an attacker encounters them. Current tools do not adequately evade modern statistical and machine learning-based detection heuristics.

2. Methodology: The PHANTOM Framework

PHANTOM (Polymorphic Honeytoken Adaptation with Narrative-Tailored Organisational Mimicry) is a framework that generates contextually convincing honeytokens by encoding organization-specific knowledge into a multi-component generation pipeline.

2.1 Organizational Profile

The system operates on a structured Organizational Profile ($O$) containing:

• Domain names and short names.
• Service lists and database types/hosts.
• Cloud regions and Git organization details.
• JWT issuers and audiences.

2.2 Contextual Generation Pipeline

Unlike static generators, PHANTOM employs dedicated generators ($G_t$) for eight different token types (including AWS keys, environment files, JWTs, and Slack bot tokens). The core design principle is value-level context injection:

• Every named field (IAM usernames, database names, Redis hostnames, Git remote URLs) is derived directly from the organizational profile.
• The pipeline avoids external API calls, ensuring it can be deployed in air-gapped or classified environments.
• Scanner-Aware Design: The generators are explicitly engineered to evade three primary scanner heuristics:
  • Regex/Pattern ($S_1$): Replaces generic red-flag strings (e.g., "example.com") with organization-derived values.
  • Entropy Analysis ($S_2$): Mimics the mixed-entropy profile of real files (high-entropy secrets alongside low-entropy human-readable values) rather than uniform high entropy.
  • ML Classifiers ($S_3$): Embeds organization-specific fingerprints (domains, service names) to counteract classifiers trained to detect a lack of context.

2.3 Evaluation Metrics

The paper formalizes token quality through a Believability Score ($B$) and a Composite Honeytoken Score ($H$):

• Believability Score ($B$): A weighted composite of four components:
  1. Syntactic Validity ($S_v$): Correct format for the token type.
  2. Semantic Coherence ($S_c$): Contextual fit with the organizational profile.
  3. Statistical Normality ($S_n$): Entropy and distributional properties matching real credentials.
  4. Human Acceptance ($S_h$): Fraction of expert evaluators classifying the token as genuine.
• Detection Resistance ($DR$): Defined as $1 - P_d$, where $P_d$ is the probability of detection by a weighted ensemble of scanner models ($S_1, S_2, S_3$).
• Composite Score ($H$): Balances believability and detection resistance ($H = B^\lambda \cdot DR^\mu$).

3. Experimental Setup

• Dataset: 32 PHANTOM tokens and 32 template-based baseline tokens generated across 8 token types and 4 organizational contexts (FinTech, Healthcare, Defense, E-commerce).
• Baseline: A static template generator using random secret values but no organizational context (equivalent to Canarytokens).
• Evaluation:
  • Automated: Three scanner models (Regex, Entropy, ML Classifier) applied to all tokens.
  • Human: Expert evaluators classified tokens as genuine or decoy.
  • Statistical: Two-sample Welch's t-tests with Bonferroni correction; effect sizes reported as Cohen's $d$.

4. Key Results

PHANTOM significantly outperforms the template-based baseline across all primary metrics:

Metric	Template Baseline	PHANTOM	$\Delta$	Significance
Believability ($B$)	$0.576 \pm 0.058$	$0.778 \pm 0.057$	$+0.203$	$p < 0.001$ ($d=3.52$)
Syntactic Validity ($S_v$)	$0.842$|$0.876$|$+0.034$	Not Significant
Semantic Coherence ($S_c$)	$0.396$|$0.705$|$+0.309$|$p < 0.001$($d=4.52$)
Human Acceptance ($S_h$)	$6.2\%$	$100.0\%$	$+93.8\%$	$p < 0.001$
**Detection Resistance ($DR$)** |$0.609$|$0.870$|$+0.261$|$p < 0.001$($d=7.49$)

• Semantic Coherence as the Driver: The largest improvement was in semantic coherence ($\Delta S_c = +0.309$), confirming that organizational context is the critical missing ingredient. Syntactic validity showed no significant difference, indicating that format correctness is a necessary but insufficient condition for deception.
• Human Acceptance: 100% of PHANTOM tokens passed the human expert threshold ($B \ge 0.65$), compared to only 6.2% of templates.
• Detection Resistance: PHANTOM tokens were significantly harder to detect across all three scanner models, with the most dramatic improvement against ML-based classifiers (detection probability dropped from 54.1% to 16.2%).
• Pareto Optimality: 100% of PHANTOM tokens fell within the "ideal zone" ($B \ge 0.70$ and $DR \ge 0.70$), whereas only 6.2% of template tokens did.

5. Significance and Claims

The paper claims that PHANTOM addresses the fundamental limitation of current honeytoken generation: the absence of organizational specificity.

• Primary Contribution: The introduction of a formal Believability Score framework that integrates syntactic, semantic, statistical, and human dimensions, demonstrating that semantic coherence is the dominant driver of deception quality.
• Operational Advantage: By operating without external API calls, PHANTOM is fully deployable in air-gapped environments (e.g., classified government networks, industrial control systems) where cloud API access is prohibited, offering a level of quality comparable to LLM-based approaches without the network dependency.
• Strategic Impact: The framework provides the high-quality "bait" component necessary for a complete post-exfiltration deception and attribution pipeline, complementing existing systems like Passive Hack-Back and longitudinal re-identification systems.

The authors conclude that future generator quality efforts must shift focus from format correctness (syntactic validity) to semantic embedding, as the latter is the primary factor determining whether a honeytoken is indistinguishable from a legitimate credential.