💻 computer science

Context-Aware Spear Phishing: Generative AI-Enabled Attacks Against Individuals via Public Social Media Data

This paper demonstrates how generative AI can leverage minimal public social media data to automate highly personalized, context-aware spear-phishing campaigns that significantly outperform traditional attacks in persuasiveness and evasion of both human suspicion and existing AI safeguards, highlighting the urgent need for enhanced platform-level defenses.

Original authors: Elham Pourabbas Vafa, Sayak Saha Roy, Shirin Nilizadeh

Published 2026-05-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Elham Pourabbas Vafa, Sayak Saha Roy, Shirin Nilizadeh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a security guard at a high-end club. Your job is to spot fake IDs and turn away people trying to sneak in. For years, the bad guys have been trying to get in by using cheap, obviously fake IDs. You could spot them a mile away because the photos were blurry, the names were misspelled, and the logic didn't make sense.

This paper is about how the bad guys just upgraded to Generative AI, and suddenly, their fake IDs look so real that even the best security guards are getting fooled.

Here is the breakdown of what the researchers discovered, using simple analogies:

1. The New "Scam Machine"

Previously, if a hacker wanted to trick you (a "spear phishing" attack), they had to do a lot of manual homework. They had to read your social media, figure out your hobbies, and then painstakingly write a fake email that sounded like it came from a friend or a boss. It was slow, expensive, and often sounded robotic.

The Change: The researchers showed that you can now feed a tiny amount of your public social media posts (as few as 10) into an AI. The AI acts like a super-creative ghostwriter. It instantly learns your style, your favorite coffee shop, your recent vacation, and your mood. It then writes a perfect, personalized email that sounds exactly like you or someone you trust.

2. The "Seven Disguises"

The paper identifies seven specific "costumes" or strategies the AI uses to trick people. Think of these as different ways a con artist might approach you:

The Bait: "Hey, I saw you love jazz; here are free VIP tickets!" (Lures you with a gift).
The Scare: "Your account is hacked! Click here to fix it!" (Uses fear).
The Honey Trap: "I saw your photos and thought we'd get along; let's chat." (Uses romance or friendship).
The Trade: "I'll give you this cool art tool if you give me your email." (A "quid pro quo").
The Tailgater: "I saw you applied for that job; here's a link from a recruiter." (Pretends to follow up on something you just did).
The Imposter: "Hi, it's your boss, need you to buy these gift cards." (Pretends to be someone you know).
The Emotional Manipulator: "I saw you were sad about your breakup; this site helps." (Pretends to care about your feelings).

The AI can mix and match these costumes with your personal data to make the trap feel incredibly real.

3. The "Magic Trick" of Bypassing Filters

You might think, "But don't these AI models have safety filters that stop them from writing bad things?"

The researchers found that the bad guys can use a magic trick called "prompt engineering." Instead of asking the AI, "Write a scam email," they ask, "Write a story about a character who needs to send a message to a friend about a concert." The AI, being a helpful assistant, doesn't realize it's being tricked into writing a scam. It just follows the instructions and produces the perfect phishing email, bypassing the safety guards.

4. The "Human Test" (The Most Scary Part)

The researchers didn't just look at the emails on a computer; they tested them on real humans.

The Setup: They showed people two types of emails: old-school scams (which look obviously fake) and the new AI-generated scams.
The Result: The old scams were easy to spot. But the AI-generated ones? People thought they were real.
The Shock: In fact, people rated the AI emails as less suspicious than the real-world scams they were used to seeing. The AI emails were so well-written, so grammatically perfect, and so personalized that they slipped right past human suspicion.

5. The "Security Guard" Struggles

Finally, the researchers tested the current "security guards" (the safety filters built into the AI models and other detection tools).

The Problem: The standard safety filters are like guards who only look for specific keywords like "scam" or "hack." Since the AI scammers are using clever language to hide those words, the guards often miss them.
The Solution Attempt: The researchers built a new, smarter guard (a specialized detector) that looks at the intent of the request before the email is even written. This new guard was very good (98% accurate), but it highlights that the old guards are no longer enough.

The Bottom Line

The paper concludes that Generative AI has lowered the barrier to entry for cybercrime. You no longer need to be a skilled hacker or spend weeks crafting a scam. With just a few public posts and an AI, anyone can create thousands of highly convincing, personalized scams that look and feel real to human victims.

The researchers warn that while we can build better detectors, the fundamental problem is that the "bad guys" now have a tool that makes their lies indistinguishable from the truth, at least for the average person looking at their inbox.

Technical Summary: Context-Aware Spear Phishing via Generative AI

Problem Statement

Spear phishing remains a pervasive and damaging social engineering threat, traditionally requiring significant manual effort to collect personal data, tailor persuasive narratives, and scale attacks. While breached datasets offer insights, they are costly and difficult to access. Conversely, public social media platforms provide rich, free contextual signals (interests, relationships, sentiments, life events), yet crafting credible, personalized phishing content from this data still demands non-trivial human labor.

This paper investigates how Generative AI (GenAI) models can be misused to automate and scale highly personalized, context-aware spear-phishing campaigns. The core problem is that adversaries can now leverage minimal public activity (e.g., a handful of social media posts) to extract interests and contextual cues, using GenAI to generate persuasive messages that mirror a target's communication style while bypassing generic content-moderation safeguards.

Methodology

1. Attack Generation Framework

The authors propose a modular, four-stage framework to automate context-aware spear phishing:

Contextual Extraction: Extracts personal signals (names, interests, relationships, events) from public social media content (specifically Instagram) using multimodal LLMs to minimize hallucinations.
Attack Type × Context Integration: Aligns one of seven identified phishing strategies with one of five contextual dimensions to ensure situational relevance.
Style Mimicry: Adapts tone, vocabulary, and structure to reflect the target's communication patterns.
Output Formatting: Produces structured, professionally worded emails with plausible sender identities and non-clickable plain-text URLs to evade link-based filters.

Prompt Engineering & Evasion: To bypass commercial safety filters (e.g., in GPT-4), the framework employs three lightweight strategies:

Chain-of-Thought Alignment: Decomposing the task into reasoning steps (extraction, analysis, justification) rather than direct malicious requests.
Keyword Substitution: Replacing sensitive terms (e.g., "scam") with benign alternatives.
Benign Framing: Embedding objectives within innocuous prompts (e.g., "write a note for a friend").

2. Taxonomy Development

The study introduces a unified taxonomy mapping seven attack strategies (Baiting, Scareware, Honey Trap, Quid Pro Quo, Tailgating, Impersonation, Personalized Emotional Exploitation) against five contextual dimensions (Location, Relationships, Interests, Sentiment, Events). This framework is grounded in a review of 21 prior papers (2015–2024).

3. Data Collection and Generation

Data Source: 200 public Instagram profiles were sampled, yielding 3,268 posts.
Generation: The pipeline generated 17,916 emails across five GenAI models (GPT-4, Claude 3-Haiku, Gemini 1.5-Flash, Gemma 7B, LLaMA 3.3).
Benchmarking: Results were compared against 3,937 real-world spear-phishing emails from the APWG eCrimeX corpus.

4. Evaluation Metrics

Eight dimensions were used to evaluate email quality and susceptibility:

Contextual Relevance
Persuasiveness
Emotional Manipulation
Personalization
Linguistic Naturalness
Specificity of Call to Action
Credibility of Sender
Technical Sophistication

Evaluation was conducted via a tri-LLM majority voting system (validated against human annotators) and a human user study.

5. Defense Evaluation

The paper evaluates proactive defenses, including:

Supervised Detection: A RoBERTa classifier trained on malicious prompts.
Sentence/Sub-prompt Screening: Using DeBERTa for early-stage interception of incremental prompt construction.
SOTA Safeguards: Testing instruction-tuned safety filters (ShieldGemma, LlamaGuard, WildGuard) and prompt-hardening techniques (Policy Injection, System Instructions + Chain-of-Thought).

Key Results

Attack Effectiveness

Superiority over Human-Crafted Attacks: GenAI-generated emails consistently outperformed real-world APWG phishing emails across all eight evaluation dimensions. Notably, GenAI emails achieved 85–90% personalization compared to 8.6% for APWG emails, and ~100% linguistic naturalness versus 51%.
Human Susceptibility: In a user study (N=70), participants rated GenAI-generated phishing emails as significantly less suspicious (mean 2.75) than real-world phishing emails (mean 4.44).
Detection Gap: Participants in the LLM condition exhibited a negative detection gap (Δsusp = -0.36), meaning they rated malicious emails as less suspicious than benign ones, whereas the APWG group showed a clear discrimination gap (Δsusp = 1.61).
Model Variance: Commercial models (GPT-4, Claude) generally produced more persuasive and credible content than open-source models (Gemma, LLaMA), though all demonstrated significant capability.

Defense Performance

Prompt-Level Detection: A RoBERTa-based classifier achieved 98.1% accuracy in detecting malicious prompts, with 100% precision and 96% recall. It generalized well across different models and attack types.
Safeguard Limitations: Default safety filters (e.g., LlamaGuard, WildGuard) failed to block the majority of malicious prompts (pass-through rates of 80–100%). However, Policy Injection and System Instructions + Chain-of-Thought (SI+CoT) significantly improved blocking rates (up to 98.7% for WildGuard with SI+CoT).
Adaptive Evasion: Sentence-level screening using DeBERTa achieved a macro-F1 of 0.945, demonstrating the feasibility of intercepting evolving malicious prompts before full generation.

Cost and Scalability

The cost to generate a single spear-phishing email is negligible ( $0.002–$ 0.004).
Attackers can target thousands of users per hour, shifting the primary bottleneck from content creation to delivery (spam filter evasion).

Significance and Contributions

The paper claims the following contributions:

Unified Framework: The first study to unify attack taxonomies, multi-dimensional evaluation metrics, and real-world baselines into a scalable methodology for measuring GenAI-enabled spear phishing.
Scalable Attack Pipeline: A modular prompt-engineering framework demonstrating that as few as 10–15 public posts are sufficient to generate highly contextualized, persuasive attacks.
Empirical Evidence of Risk: Large-scale evaluation (~18K emails) and a user study confirming that GenAI attacks are not only higher quality but also less detectable by humans than current real-world campaigns.
Defense Analysis: Systematic measurement of existing safeguards, revealing that while default filters are insufficient, prompt-aware instruction and structured reasoning can significantly improve resilience.

The authors conclude that GenAI drastically lowers the barrier for executing effective, personalized spear-phishing attacks, necessitating platform-level safeguards that explicitly account for contextualized abuse at scale. They emphasize that the primary challenge for defenders has shifted from content creation to detecting context-aware, adaptive intent.