User-Centric Phishing Detection: A RAG and LLM-Based Approach

Technical Summary: User-Centric Phishing Detection via RAG and LLMs

Problem Statement
The paper addresses the escalating sophistication of phishing emails, which increasingly bypass traditional rule-based filters and conventional machine learning detectors. While Large Language Models (LLMs) offer superior natural language understanding and semantic analysis capabilities, their application as standalone classifiers for email security presents a critical flaw: elevated false-positive (FP) rates. Generic LLM-based systems often mislabel legitimate emails as phishing when those emails deviate from general patterns, even if they align with a specific user's communication history. This lack of user-specific context creates a significant operational burden by generating excessive false alarms and fails to distinguish between universally malicious content and unusual-yet-legitimate communications for individual users.

Methodology
The authors propose a personalized phishing detection framework that integrates Retrieval-Augmented Generation (RAG) with open-source LLMs. The system operates through a six-stage pipeline designed to ground model decisions in user-specific context and real-time threat intelligence:

1. Data Preprocessing: Raw email corpora are transformed into a structured knowledge base through validation, normalization (handling various encodings like UTF-8, Latin-1), feature extraction (subject, sender, body), and header stripping.
2. Semantic Embedding & Vector Construction: Textual content is projected into a continuous vector space using the all-MiniLM-L6-v2 transformer. These embeddings are L2-normalized and indexed in a FAISS vector database to facilitate efficient approximate nearest neighbor search using cosine similarity.
3. Dual-Context Retrieval:
   • User-Specific History: For an incoming query email, the system retrieves the top-$k$ (set to 5) semantically similar legitimate emails from the user's historical corpus. This provides the LLM with concrete examples of the user's normal communication patterns.
   • Threat Intelligence: The system extracts domains and URLs from the query email and queries a multi-engine analysis platform (VirusTotal API) to gather real-time reputation data (harm, suspicion, malware status).
4. Structured Prompt Engineering: The LLM prompt is constructed by concatenating the role assignment (cybersecurity expert), the query email content, the retrieved historical context ($C_{context}$), and the threat intelligence ($T(e_{query})$). The output is constrained to a specific JSON schema requiring a classification decision, a phishing score, risk level, identified social engineering tactics, recommended actions, and a brief reasoning.
5. Model Evaluation: The framework was tested on four diverse open-source LLMs: Llama4-Scout (17B), DeepSeek-R1 (70B), Mistral-Saba (24B), and Gemma2-9B (9B). All models were accessed via the Groq API with a temperature of 0.2 to ensure deterministic responses.

Key Contributions

• Personalized RAG Framework: The introduction of a dual-context retrieval mechanism that combines user-specific email history with real-time threat intelligence. This architecture aims to reduce false positives by enabling the system to distinguish between generic malicious patterns and user-normal communications.
• Comprehensive Evaluation Methodology: A systematic assessment of four distinct open-source LLMs across both standalone and RAG-enhanced configurations, establishing a standardized benchmark for AI-powered security tools.
• Empirical Validation of Personalization: Demonstration that personalization benefits scale effectively across different model sizes (from 17B to 70B parameters), with specific evidence showing substantial reductions in false positive rates without compromising detection accuracy.

Results
The system was evaluated on a balanced dataset of 500 emails (250 legitimate, 250 phishing). The integration of RAG yielded consistent performance improvements across all tested models:

• False Positive Reduction: The most significant improvement was observed in the reduction of False Positive Rates (FPR). For instance, Llama4-Scout reduced its FPR from 12% to 4% when RAG was enabled. Similarly, DeepSeek-R1 dropped from 22% to 6%, and Mistral-Saba from 35.6% to 10%.
• F1-Score Improvement: The F1-scores increased across the board. Llama4-Scout achieved the highest performance with an F1-score of 0.9703 with RAG (compared to 0.9333 without).
• Model Comparison: While all models benefited, Llama4-Scout with RAG demonstrated the best balance of precision and recall with the lowest propensity for false alarms. DeepSeek-R1 followed closely, while Gemma2-9b, though improved, remained more prone to false positives compared to the larger models.

Significance and Claims
The paper claims that a RAG-based, user-profiling approach is both feasible and effective for constructing high-precision, low-friction email security systems. The primary significance lies in addressing the "aggressive classification" tendency of standalone LLMs by grounding their reasoning in individual user behavior. The authors assert that this approach validates the potential of adaptive, user-centric detection to significantly lower operational burdens caused by false positives while maintaining high detection accuracy. The work suggests that future systems should move beyond generic classifiers toward architectures that leverage retrieval to contextualize threats within the specific communication patterns of the end-user.