📄 other

X-WAF: A Hybrid Machine Learning Framework with SHAP-Driven Explainability and Automated Rule Synthesis for Real-Time Web Application Protection

This paper presents X-WAF, a hybrid machine learning framework that combines an ensemble of classifiers with SHAP-driven explainability to achieve high-accuracy, low-latency detection of zero-day web attacks while automatically translating insights into actionable ModSecurity rules for seamless integration into existing security infrastructures.

Original authors: Danish Quarni

Published 2026-07-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Danish Quarni

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a bustling, chaotic city where millions of people visit digital shops, banks, and social clubs every second. To keep this city safe, we have security guards at the gates called Web Application Firewalls (WAFs). For a long time, these guards have relied on a giant "Wanted Poster" book. If a visitor looks exactly like a criminal on the poster—wearing the same hat, carrying the same bag, or saying the same suspicious phrase—the guard stops them. This works great for known criminals, but it fails miserably against new thieves who wear disguises, change their voices, or invent entirely new ways to sneak in. These new tricks are called "zero-day" attacks, and they leave the city vulnerable.

To solve this, scientists are trying to teach security guards to be detectives instead of just matchers. Instead of just checking a list, a detective looks at behavior: "This person is walking too fast," or "They are asking questions that don't make sense." This is where machine learning comes in—computer programs that learn patterns from traffic to spot the weird stuff. But there's a catch: these computer detectives are often "black boxes." They might stop someone, but they can't explain why they did it. Security guards need to know the reason to trust the computer, and they need to be able to write down new rules based on what the computer learned. This paper, written by Danish Quarni, introduces a new system called X-WAF that tries to be the perfect detective: one that is super smart, explains its reasoning clearly, and can even write its own rulebook for the future.

The paper presents X-WAF, a hybrid machine learning framework designed to protect websites in real-time. Think of X-WAF not as a single guard, but as a high-tech security team made up of four different specialists working together. Two of them are "supervised" learners, meaning they have studied thousands of examples of bad guys and know exactly what to look for. The other two are "unsupervised" learners, meaning they are experts at spotting anything that just feels off or unusual, even if they've never seen that specific type of bad guy before. By combining these four perspectives, the system creates a safety net that catches both known threats and brand-new, sneaky attacks that traditional guards miss.

However, the team knew that a smart system is useless if no one understands how it works. To fix this, X-WAF uses a special tool called SHAP. Imagine SHAP as a magnifying glass that breaks down every single decision the security team makes. If the system blocks a visitor, SHAP points to the exact reasons: "We stopped you because your password field was too long, your message had strange symbols, and your computer name looked suspicious." This transparency builds trust, allowing human security analysts to verify the decision instantly.

The most creative part of X-WAF is its ability to turn these complex, computer-generated explanations into simple, written rules. Usually, if you want a computer to stop a bad guy, you have to keep the computer running and thinking about every single person who walks by, which is slow and expensive. X-WAF does something clever: it takes the lessons the computer learned (like "bad guys often have long passwords") and automatically writes them into a standard rulebook that any security guard can read. This means the system can learn from new attacks, write a new rule, and then let the standard, fast guards enforce that rule without needing the heavy computer brain for every single request.

The researchers tested this system on a mix of fake attacks and real-world web traffic. They found that X-WAF is incredibly effective, catching 96.5% of attacks while only making mistakes (blocking good people) 1.8% of the time. It's also fast, taking only 8 milliseconds to make a decision—fast enough to keep up with busy websites handling over 12,000 requests every second. The paper suggests that this approach bridges the gap between the flexibility of modern AI and the reliability of traditional security, offering a way to protect websites from evolving threats without slowing them down or leaving security teams in the dark about why a decision was made.

Technical Summary of X-WAF: A Hybrid Machine Learning Framework with SHAP-Driven Explainability and Automated Rule Synthesis

Problem Statement
Conventional Web Application Firewalls (WAFs), such as those utilizing the OWASP Core Rule Set (CRS), rely on signature-based detection. While effective against known attack vectors, these systems struggle to generalize against zero-day exploits, adversarial payloads, and evolving API-based threats. Attackers frequently employ encoding, fragmentation, and polymorphism to bypass static rules. Furthermore, the manual configuration and updating of rule sets are reactive, resource-intensive, and often result in a suboptimal trade-off between detection rates and false alarms. While Machine Learning (ML) offers a path to generalize beyond explicit signatures, its deployment in real-time security infrastructure faces two critical hurdles: the "black-box" nature of ensemble models which erodes analyst trust and hinders forensic analysis, and the computational latency introduced by running inference on every request.

Methodology
The paper introduces X-WAF, a hybrid framework designed to bridge the gap between statistical anomaly detection and operational security infrastructure. The system operates as a modular pipeline consisting of four primary components:

Traffic Preprocessing and Feature Extraction: X-WAF intercepts HTTP requests and extracts a 78-dimensional feature vector. This includes request metadata (method, path length), header statistics (entropy, size), payload characteristics (n-gram frequencies, presence of SQL keywords or script tags), structural features (JSON nesting depth), and temporal features (request rates). Features are normalized using robust scalers.
Hybrid Ensemble Model: The core detection engine utilizes a heterogeneous ensemble of four classifiers combined via a weighted voting scheme. The weights are learned based on the precision of each model on a validation set.
- Random Forest (200 trees): A supervised classifier for discriminative pattern matching.
- Support Vector Machine (SVM) with RBF kernel: A secondary supervised model for handling high-dimensional, non-linear relationships.
- Isolation Forest: An unsupervised model to isolate observations and detect anomalies.
- Local Outlier Factor (LOF): An unsupervised model to capture localized density variations.
- A request is classified as malicious if the weighted score exceeds a threshold of $\tau = 0.65$ .
SHAP Explainability Module: To address model opacity, X-WAF integrates SHAP (SHapley Additive exPlanations). For every request flagged as malicious, the system computes feature attributions to determine the contribution of each feature to the decision. The top-5 features with the largest absolute SHAP values are reported, providing granular insight (e.g., high query string entropy or presence of specific SQL keywords) to security analysts.
Automated Rule Synthesis: A key innovation is the translation of SHAP-derived decision boundaries into ModSecurity-compatible rules. This module operates offline or periodically (e.g., hourly). It analyzes features that frequently appear in top SHAP explanations, identifies value ranges associated with malicious classifications, and generates logical conjunctions in ModSecurity syntax. Only rules with a confidence score exceeding 90% are deployed. This allows ML-generated intelligence to persist within existing rule-based infrastructures without requiring per-request model inference.

Key Contributions
The paper outlines three primary contributions:

Heterogeneous Ensemble Architecture: A novel combination of supervised (Random Forest, SVM) and unsupervised (Isolation Forest, LOF) models to capture both discriminative patterns and unsupervised anomaly signatures in HTTP traffic.
Real-Time SHAP Integration: The embedding of SHAP directly into the decision pipeline to provide per-request, human-understandable explanations for detection decisions, enhancing transparency and auditability.
Automated Rule Synthesis Engine: A mechanism to convert complex ML decision boundaries into standard ModSecurity rules. This enables the deployment of ML-derived protections in production environments without the latency overhead of continuous model inference.

Experimental Results
X-WAF was evaluated on diverse datasets, including the CSIC 2010 HTTP Dataset, the ECML/PKDD 2007 Dataset, and real-world proxy logs from a mid-sized e-commerce company. The system was compared against baselines including ModSecurity with CRS 3.3, individual models (Random Forest, Isolation Forest, SVM, LOF), and an ensemble without SHAP.

Detection Performance: X-WAF achieved a detection accuracy of 96.5% with a False Positive Rate (FPR) of 1.8%. This outperformed the best single supervised model (Random Forest at 93.1% accuracy, 2.1% FPR) and the signature-based baseline (ModSecurity CRS 3.3 at 82.3% accuracy, 5.2% FPR).
Performance Metrics: The system maintained an average latency of 8 ms per request and a throughput exceeding 12,000 requests per second (RPS), meeting the demands of production environments.
Rule Synthesis Efficacy: In a 24-hour batch, the system generated 47 ModSecurity rules. When deployed, these synthesized rules alone blocked 72% of malicious requests in the subsequent hour without invoking the full ML model.
Adversarial Resilience: Under adversarial conditions using genetic algorithm-generated SQLi payloads, X-WAF maintained an 89.2% detection rate, significantly outperforming ModSecurity CRS (54.3%) and Random Forest alone (78.5%).

Significance and Claims
The paper posits that X-WAF addresses the critical security gap where traditional WAFs fail against novel attacks while overcoming the operational barriers that have hindered ML adoption in security. By providing transparency through SHAP and seamless integration through automated rule synthesis, the framework offers a practical path for deploying machine learning in security-critical applications. The authors claim that X-WAF not only enhances detection coverage against zero-day attacks but also ensures the auditability and low-latency performance necessary for real-world deployment, effectively allowing organizations to leverage ML intelligence within their existing rule-based WAF infrastructures.

Technical Summary of X-WAF: A Hybrid Machine Learning Framework with SHAP-Driven Explainability and Automated Rule Synthesis

More like this