💻 computer science

An Explainable Agentic RAG Framework for Zero-Shot Phishing and Web Attack Detection

The paper introduces XAR-Detect, an explainable agentic RAG framework that leverages a DeBERTa encoder, dynamic threat signature retrieval, and autonomous reasoning to achieve high-accuracy, zero-shot detection of phishing and web attacks with transparent justifications, significantly outperforming existing state-of-the-art methods without requiring retraining.

Original authors: Mohammad Zahangir Alam, Mahdi H Miraz, Elhan Ayath, Sharmin Sultana, Nowshad Amin

Published 2026-09-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mohammad Zahangir Alam, Mahdi H Miraz, Elhan Ayath, Sharmin Sultana, Nowshad Amin

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the digital world, the most persistent threats often arrive disguised as ordinary messages or routine website requests. Phishing emails, designed to steal passwords, and web attacks, which try to break into servers, have become so sophisticated that they can mimic legitimate communication with frightening accuracy. For decades, security systems have relied on a strategy of recognition: they are trained on massive libraries of known bad examples, learning to spot the specific fingerprints of past crimes. However, this approach has a fundamental weakness. When attackers invent a new trick, one that has never been seen before, these static systems often fail because they have no reference point for the unknown. To solve this, researchers are now exploring a different path, one that combines the ability to learn from context with the power to look up fresh information in real time, much like a security guard who not only knows the faces of known criminals but also carries a live feed of current alerts to identify a stranger in the crowd.

A team of researchers has developed a new system called XAR-Detect that attempts to bridge this gap between static training and dynamic reality. Instead of relying solely on a pre-trained model that freezes its knowledge the moment it finishes learning, this framework acts more like an intelligent investigator. It takes a suspicious email or a web request and first analyzes its content to understand its structure and meaning. Then, rather than making a guess based only on what it has memorized, it immediately reaches out to a live database of global threat intelligence. This database is constantly updated with information from major security repositories, containing millions of records about known phishing links, malicious code, and recent vulnerabilities. The system retrieves the most relevant warnings from this live feed and uses them to inform its decision. If the incoming message shares characteristics with a newly reported threat, the system can flag it as dangerous, even if the system itself has never been explicitly trained on that specific type of attack.

The core innovation lies in how the system processes this information. It does not simply add the new data to the old; it employs an autonomous reasoning agent that acts as a critical thinker. This agent weighs the evidence from the message itself against the fresh intelligence it has just retrieved. It asks itself whether the patterns match, considers the context, and then forms a hypothesis about whether the message is safe or malicious. This multi-step process allows the system to adapt to new attack strategies without needing to be retrained from scratch. In their experiments, the researchers tested this approach on a variety of real-world datasets, including thousands of phishing emails and millions of web traffic records. They found that the system could correctly identify known attacks with an accuracy of nearly 98 percent, a performance level that surpassed existing methods by a significant margin. More importantly, when tested on completely new types of attacks that were hidden from its training data, the system still managed to detect them with an accuracy of 88.3 percent. This suggests that the ability to look up current threats in real time provides a powerful safety net against the unknown.

Beyond just catching bad actors, the researchers were concerned with the "black box" problem, where complex computer systems make decisions that humans cannot understand. In high-stakes security, knowing why a system flagged a message is just as important as the flag itself. XAR-Detect addresses this by providing a clear, two-layer explanation for every decision it makes. First, it highlights which specific features of the message—such as a strange link or an unusual word choice—contributed most to the suspicion. Second, it shows which pieces of external threat intelligence were used to reach the conclusion. This transparency allows human security analysts to verify the system's logic, ensuring that the detection is based on solid evidence rather than a random guess. The researchers measured how reliable these explanations were and found them to be highly consistent, meaning the system gives a stable reason for its choices even when the input is slightly rephrased.

The study also revealed the limits of this approach. The system's ability to spot new threats depends entirely on the quality and coverage of the live database it consults. If a completely novel attack appears that has not yet been recorded in any global threat repository, the system cannot look it up, and its performance may drop. The researchers acknowledge that while the system is a major step forward, it is not a perfect shield against every possible future threat. Furthermore, because the reasoning part of the system relies on powerful external computing services, there are practical considerations regarding speed and cost that must be managed in real-world deployments. Despite these constraints, the results demonstrate a viable path forward for cyber defense. By combining deep understanding of language with the ability to access live intelligence, XAR-Detect offers a way to detect threats that have never been seen before, providing a more flexible and transparent defense for the digital world.

Technical Summary: An Explainable Agentic RAG Framework for Zero-Shot Phishing and Web Attack Detection

1. Problem Statement

Phishing and web-based attacks (e.g., SQL injection, XSS) remain critical threats to industrial and enterprise systems, with attack volumes increasing significantly and evolving rapidly. Existing detection frameworks face three primary limitations:

Static Generalization: Most supervised and self-supervised models rely on fixed training distributions. Once trained, they cannot generalize to unseen attack patterns without retraining or fine-tuning.
Data Intensity: High-performance models often require large volumes of manually labeled data, which is scarce for emerging threats.
Lack of Transparency: Many adaptive methods, such as Reinforcement Learning (RL) or standalone Large Language Model (LLM) classifiers, operate as "black boxes," lacking the explainability required for regulatory compliance (e.g., GDPR, NIS2) and analyst trust.

Current approaches typically address only one aspect of this challenge (e.g., adaptability without explainability, or explainability without zero-shot capability). There is a gap for a unified framework that combines zero-shot generalization, agentic adaptive reasoning, real-time threat knowledge grounding, and transparent explainability.

2. Methodology: XAR-Detect

The authors propose XAR-Detect (eXplainable Agentic Retrieval-Augmented Detection), a framework designed to operate in an "open-world" setting where it must detect attack classes not present in its training data. The system integrates four core components:

A. Contextual Encoding

Raw email and HTTP request data are processed using DeBERTa-v3, selected for its disentangled attention mechanism and relative position encoding. This encoder generates rich contextual representations ( $e_i$ ) from the input text, capturing both semantic content and structural attributes.

B. Retrieval-Augmented Generation (RAG)

To overcome the limitations of static training distributions, XAR-Detect employs a RAG module. At inference time, the contextual representation serves as a query to retrieve the top- $K$ semantically matched threat signatures from a live knowledge base ( $\mathcal{E}$ ). This knowledge base aggregates data from:

PhishTank (phishing URLs)
VirusTotal (malware signatures)
CVE (Common Vulnerabilities and Exposures)

The retrieved signatures ( $\mathcal{R}_i$ ) are aggregated via attention-weighted mechanisms and fused with the local encoder representation to create an augmented input ( $\tilde{x}_i$ ). This allows the system to leverage external threat intelligence for attacks excluded from the training set.

C. Agentic Reasoning

Unlike standard RAG pipelines that passively concatenate context, XAR-Detect utilizes an autonomous LLM-based reasoning agent (specifically GPT-4o) to process the augmented evidence. The agent performs a structured, multi-step inference process:

Evidence Assessment: Evaluates the consistency between local features and retrieved threat signatures.
Hypothesis Formation: Generates candidate labels (Benign, Phishing, Web Attack) with confidence estimates.
Decision Consolidation: Resolves competing hypotheses to produce a final prediction.

This iterative process enables the model to refine its decision based on retrieved evidence without requiring the policy training cycles associated with Reinforcement Learning.

D. Dual-Layer Explainability

To ensure transparency, the framework provides two layers of explanation:

Global Attribution: Uses SHAP (SHapley Additive exPlanations) to determine the marginal contribution of specific features and retrieved threat dimensions to the prediction across the population.
Instance-Level Attribution: Aggregates token-level attention maps across encoder layers and heads to provide case-specific justifications, highlighting exactly which parts of the input and retrieved evidence influenced the decision.

3. Key Contributions

The paper outlines five primary contributions:

Unified Framework: The proposal of XAR-Detect, which integrates LLM-based reasoning, dynamic threat intelligence retrieval, and dual-layer explainability for both phishing and web attack detection.
Zero-Shot Mechanism: A RAG-enhanced architecture that enables the detection of previously unseen attack variants without retraining or fine-tuning.
Autonomous Reasoning Agent: An agent that synthesizes local feature representations with retrieved evidence through multi-step inference, extending the agentic RAG paradigm to unified threat detection.
Dual-Layer Transparency: The implementation of global SHAP analysis combined with instance-level attention attribution to provide both model-wide and case-specific interpretability.
Empirical Validation: Comprehensive evaluation demonstrating superior performance over state-of-the-art baselines in balanced, imbalanced, and zero-shot scenarios.

4. Experimental Results

The framework was evaluated on three phishing corpora (Nazario, Enron, Subhadeep) and the SR-BH web attack benchmark.

Overall Performance (Balanced): XAR-Detect achieved an F1-score of 97.8% and a False Positive Rate (FPR) of 2.1%. This outperformed the best self-supervised baseline (SSL Ensemble) by 2.4 percentage points in F1 and reduced FPR by 2.7 points.
Robustness (Imbalanced): Under a realistic distribution where attacks comprised ~15% of data, XAR-Detect maintained an F1 of 96.2% with an FPR of 3.8%, showing significantly less performance degradation than baselines.
Zero-Shot Generalization: When tested on attack classes entirely excluded from training, XAR-Detect achieved an F1 of 88.3%. This exceeded the SSL Ensemble by 34.2 percentage points and CyberRAG by 14.1 points, demonstrating the efficacy of retrieval-augmented reasoning for unseen threats.
Ablation Studies: Removing the RAG module dropped F1 by 1.7 points; removing the agent dropped it by 2.1 points. The combination of both yielded a 3.0-point gain over the encoder-only baseline, confirming their complementary nature.
Explainability Metrics: The framework demonstrated high explanation faithfulness (0.847) and stability (0.913) compared to post-hoc baselines.

5. Significance and Claims

The paper claims that XAR-Detect offers a practical, interpretable, and label-efficient path toward intelligent cyber threat detection. Its significance lies in addressing the "closed-world" limitation of traditional models by grounding decisions in live external threat intelligence rather than frozen training distributions.

The authors emphasize that the framework is federated-ready, as the reasoning module operates on encoded representations and public intelligence, supporting distributed deployment across privacy-sensitive organizations. However, the paper maintains a modest stance regarding limitations:

Coverage Dependency: Zero-shot performance is bounded by the coverage density ( $\rho$ ) of the external knowledge base; entirely novel attack families without similar signatures in the repository may still result in high error rates.
Operational Constraints: The reliance on GPT-4o introduces API dependency, inference latency, and potential costs, which may restrict high-throughput real-time deployment.
Analyst Validation: While the technical metrics for explanation faithfulness are strong, the paper notes that the practical utility of these explanations for security analysts (e.g., reducing triage time) has not yet been validated through human-subjects studies.

In conclusion, XAR-Detect represents a shift from static, data-intensive detection toward adaptive, knowledge-grounded, and transparent systems capable of handling the evolving landscape of cyber threats.