🤖 machine learning

Can LLMs Handle WebShell Detection? Overcoming Detection Challenges with Behavioral Function-Aware Framework

This paper evaluates the capabilities of various large language models in detecting WebShells and proposes the Behavioral Function-Aware Detection (BFAD) framework, which significantly improves detection performance by anchoring analysis on security-sensitive functions and optimizing in-context learning, thereby enabling LLMs to outperform existing state-of-the-art detectors.

Original authors: Feijiang Han, Jiaming Zhang, Chuyi Deng, Jianheng Tang, Yunhuai Liu

Published 2026-02-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Feijiang Han, Jiaming Zhang, Chuyi Deng, Jianheng Tang, Yunhuai Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a security guard at a massive, chaotic library (the internet). Your job is to find a few specific books that contain instructions for blowing up the building (WebShells), while ignoring millions of perfectly normal books (benign code).

The problem is that the bad books are often:

Huge: Some are thousands of pages long.
Hidden: The dangerous instructions are buried deep inside, surrounded by boring text.
Disguised: They are written in secret codes or scrambled languages (obfuscation).

For years, security guards used two main methods:

The Rulebook: "If you see the word 'explode,' arrest it." (Too easy to trick; bad guys just change the word to 'detonate'.)
The Training Manual: Showing the guard thousands of examples of bad books so they learn the pattern. (This takes forever to update and fails when a new type of bad book appears.)

Recently, a new type of guard arrived: The Super-Intelligent AI (Large Language Models or LLMs). These AIs can read and understand almost any language. But when the researchers tried to use them to find these bad books, they hit a wall.

The Problem: The "Too Much Information" Trap

The researchers found that simply handing a giant, messy book to the AI didn't work well.

The "Too Long" Issue: If a book is 1,000 pages, the AI can only read the first 100. If the bomb instructions are on page 900, the AI misses it completely.
The "Confused" Issue: The AI is smart, but it gets distracted. It might see a harmless word like "kill" (which is used in video games) and think, "Oh no, this is a bomb!" (False Alarm). Or, it might see a real bomb hidden behind a wall of text and think, "This looks safe," (Missed Threat).

The researchers noticed a funny pattern:

Big AIs were very careful. They rarely made false alarms, but they missed a lot of real bombs because they were too cautious.
Small AIs were very paranoid. They caught almost every bomb, but they also arrested innocent librarians who just happened to use the word "kill" in a game script.

The Solution: The "Behavioral Function-Aware" Framework (BFAD)

To fix this, the researchers built a special toolkit called BFAD. Think of it as giving the AI a high-tech magnifying glass and a cheat sheet specifically designed for finding bombs.

Here is how BFAD works, using simple analogies:

1. The "Critical Function Filter" (The Metal Detector)

Instead of making the AI read the whole book, BFAD acts like a metal detector at the library entrance. It scans the text for specific "metal" items—dangerous PHP functions like exec (run a command) or eval (run code).

Why it helps: It ignores the boring story about a cat and zooms in only on the pages where the "metal" (dangerous code) is. This solves the "too long" problem.

2. Context-Aware Extraction (The Highlighter)

Once the metal detector finds a suspicious spot, BFAD doesn't just show the AI the single word. It highlights a small "window" of text around that word (the context).

Why it helps: It tells the AI, "Hey, look at this exec command. Is it being used to run a virus, or is it just a harmless system update?" This helps the AI understand the intent, not just the word.

3. Weighted Behavioral Function Profiling (The "Look-Alike" Cheat Sheet)

This is the smartest part. When the AI needs to decide if a file is bad, it looks at a "cheat sheet" of past examples (demonstrations) to learn from.

Old Way: The AI picks random examples or examples that look similar on the surface (like two books with the same font).
BFAD Way: The AI picks examples that are behaviorally similar. If the current file is trying to steal data, the AI looks at past files that also tried to steal data, even if they looked different on the surface.
The Metaphor: Imagine you are looking for a thief who steals red apples.
- Old Way: You show the guard a picture of a thief stealing a red apple, and a thief stealing a red ball. The guard gets confused.
- BFAD Way: You show the guard a picture of a thief stealing a red apple, and another thief stealing a red apple from a different store. The guard learns the behavior (stealing red apples), not just the object.

The Results: A Happy Ending

When the researchers tested this new system:

The Big AIs became much better at catching the bombs they were missing, without making more false alarms.
The Small AIs became much smarter, stopping them from arresting innocent librarians.
Overall: The new system made the AI guards perform better than the old "Rulebook" guards and even better than the most advanced "Training Manual" guards.

The Takeaway

The paper concludes that while AI is powerful, it's not a magic wand. You can't just throw a messy problem at it and expect a perfect answer. You have to guide the AI by:

Focusing its attention on the dangerous parts.
Giving it the right context.
Showing it the right examples to learn from.

By doing this, we can turn these super-smart AIs into reliable security guards for the digital world.

1. Problem Statement

WebShell attacks involve adversaries implanting malicious scripts on web servers to enable remote command execution and data exfiltration. These threats are evolving rapidly through obfuscation and encryption, making them difficult to defend against.

Limitations of Prior Work: Traditional rule-based systems are brittle against new variants. Supervised Machine Learning (ML) and Deep Learning (DL) models require substantial labeled data, suffer from catastrophic forgetting during updates, and often generalize poorly to out-of-distribution (OOD) obfuscation strategies.
The LLM Gap: While Large Language Models (LLMs) show strong code understanding capabilities, their application to WebShell detection faces specific challenges:
- Context Limits: WebShells can be extremely long (up to ~1.3M tokens), exceeding LLM context windows. Naive truncation often removes the malicious core.
- Noise: Malicious payloads are often buried within benign code, diluting the LLM's attention.
- Fragile In-Context Learning (ICL): Selecting relevant demonstrations for ICL is difficult; random or purely semantic similarity-based selection often fails to capture the specific behavioral patterns of WebShells.
- Performance Imbalance: Off-the-shelf LLMs exhibit a precision-recall trade-off: large models have high precision but miss attacks (low recall), while small models have high recall but generate many false positives (low precision).

2. Methodology: Behavioral Function-Aware Detection (BFAD)

The authors propose BFAD, a behavior-centric framework designed to adapt LLM inference to WebShell-specific execution patterns without requiring task-specific fine-tuning. BFAD consists of three core components:

A. Critical Function Filter

This component acts as a pre-processor to identify high-risk PHP functions associated with malicious behaviors. It categorizes functions into six behavior groups:

Program Execution: (e.g., exec, system)
Code Execution: (e.g., eval, preg_replace with /e)
Callback Functions: (e.g., array_map, register_shutdown_function)
Network Communication: (e.g., fsockopen, curl_init)
Information Gathering: (e.g., phpinfo, getenv)
Obfuscation and Encryption: (e.g., base64_encode, openssl_encrypt)

Statistical analysis shows WebShells invoke these critical functions significantly more frequently than benign files (avg. 22.76 vs. 0.74 calls).

B. Context-Aware Code Extraction

To address context window limits and noise, BFAD constructs a compact, informative code view:

Extraction: It locates occurrences of critical functions and extracts surrounding context windows (defined by size $\tau$ ).
Merging: Overlapping windows are merged to preserve local behavioral evidence while avoiding token redundancy.
Hybrid Strategy: If the context budget allows, non-overlapping segments from the remaining code are appended to provide global context without exceeding limits. This balances local behavioral focus with global structural understanding.

C. Weighted Behavioral Function Profiling (WBFP)

This module optimizes In-Context Learning (ICL) by selecting the most relevant demonstrations. Instead of generic semantic similarity, WBFP ranks examples based on behavior-weighted similarity:

Discrimination Scoring: For each critical function type, it calculates a score based on three statistics derived from the training corpus:
- Coverage Difference ( $r_c$ ): Frequency of the function appearing at least once.
- Frequency Ratio ( $r_f$ ): Ratio of average per-file call counts.
- Usage Ratio ( $r_u$ ): Ratio of total call counts.
Weighted Similarity: These scores are normalized into weights ( $w_f$ ). The similarity between a target file and a candidate demonstration is computed as a weighted sum of cosine similarities of function-specific embeddings. This ensures demonstrations match the behavioral profile of the target, not just its lexical content.

3. Experimental Setup

Dataset: 26,594 PHP scripts (21,665 benign, 4,929 WebShells). WebShells were augmented with synthetic obfuscation.
Models Evaluated: Seven LLMs ranging from 0.5B to 70B parameters (GPT-4, LLaMA-3.1-70B, Qwen-2.5 variants) compared against baselines:
- Sequence-based: GloVe+SVM, CodeBERT+Random Forest.
- Graph-based: GCN, GAT (State-of-the-art supervised detectors).
Metrics: Accuracy, Precision, Recall, and F1 Score.

4. Key Results

The study yields several critical findings regarding LLM performance and the efficacy of BFAD:

A. Baseline LLM Performance (Without BFAD)

Precision-Recall Asymmetry: Larger models (e.g., GPT-4) achieve near-perfect precision (100%) but suffer from lower recall (~~86%), missing disguised attacks. Smaller models (e.g., Qwen-2.5-0.5B) achieve perfect recall but very low precision (~~18%), flagging benign code as malicious.
Gap to SOTA: Off-the-shelf LLMs underperform strong supervised baselines. For instance, the best vanilla LLM (Qwen-2.5-Coder-14B) achieved an F1 of 96.39%, trailing the GAT baseline (98.87%).

B. Impact of BFAD

BFAD substantially improved performance across all model scales:

Average Improvement: BFAD boosted the average F1 score by 13.82% across all evaluated LLMs.
Large Models: GPT-4 improved from 92.46% to 99.35% F1, surpassing the GAT baseline.
Small Models: The Qwen-2.5-0.5B model saw a massive jump of +51.23 F1, rising from 31.44% to 82.67%, making it competitive with traditional methods.
Mechanism: BFAD mitigated the failure modes of both large and small models by anchoring analysis on behavior-critical calls and providing behavior-aligned demonstrations.

C. Component Analysis

Code Extraction: Using critical regions improved recall for large models (recovering missed attacks) and precision for small models (reducing noise). The hybrid strategy (critical + global snippets) offered the best balance.
WBFP vs. Other Retrieval: WBFP significantly outperformed random selection and standard semantic similarity (SC-Sim). SC-Sim failed because whole-file similarity is dominated by benign regions, diluting the malicious signal. WBFP's function-level weighting was crucial for effective ICL.

5. Key Contributions

Systematic Evaluation: The first comprehensive study characterizing the performance of diverse LLM scales (0.5B to 70B) on WebShell detection, revealing a distinct scale-dependent precision-recall trade-off.
BFAD Framework: A novel, training-free framework that integrates critical function filtering, context-aware extraction, and behavior-weighted demonstration selection to make LLMs reliable for security tasks.
Performance Parity: Demonstrated that with BFAD, LLMs can match or exceed the performance of state-of-the-art supervised graph-based detectors (like GAT) without requiring task-specific retraining or massive labeled datasets.
Insight into Scaling: Identified that very small models benefit disproportionately from behavior-aligned demonstrations (demonstration-following effect), while larger models benefit from improved recall via better context handling.

6. Significance and Future Directions

This work bridges the gap between the theoretical potential of LLMs and their practical application in cybersecurity. It proves that behavior-centric prompting is essential for security tasks where malicious code is sparse and obfuscated.

Future Directions proposed by the authors:

Synthetic Benchmarks: Using LLMs to generate privacy-preserving, diverse synthetic WebShells to stress-test generalization.
Multimodal Integration: Combining graph-based behavioral representations (which capture control flow better than sequences) with LLMs via adapters.
Agentic Pipelines: Developing "fast-slow" detection systems where lightweight models perform high-recall triage, followed by deep analysis by larger models, with autonomous loops to update detection patterns against distribution shifts.