💻 computer science

A Neural Screener–Reasoner Framework for Evidence-Grounded Log-Based Anomaly Detection and Explanation

This paper proposes a Neural Screener–Reasoner framework that combines a parser-free, linear self-attention detector with a retrieval-augmented LLM to achieve high-accuracy log anomaly detection and generate verifiable, evidence-grounded explanations while managing inference costs.

Original authors: Chi-Ming Chou, Shang-Kuan Chen

Published 2026-09-02

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Chi-Ming Chou, Shang-Kuan Chen

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Modern computing systems are vast, interconnected networks that power everything from scientific research to global finance. To keep these systems running smoothly, engineers rely on a constant stream of digital records called logs. These logs act as a diary of the system's activity, noting every action, error, and status change with a timestamp and a message. When something goes wrong—a server crashes, a file fails to save, or a security breach occurs—these logs contain the clues needed to understand what happened. However, as systems grow larger and faster, the volume of these logs becomes overwhelming. A single day's worth of activity can generate millions of lines of text, making it impossible for human operators to read them all. For decades, the solution has been to build automated tools that scan these logs to spot anomalies, or strange patterns that suggest a problem. But a significant gap has remained: while these tools can tell an operator that something is wrong, they often cannot explain why. They offer a binary warning, a simple "yes" or "no," without pointing to the specific lines of text that triggered the alarm or connecting the current error to past failures. This leaves engineers with a red light but no map, forcing them to manually sift through thousands of lines of text to find the root cause.

A team of researchers has developed a new framework designed to bridge this gap, moving beyond simple detection to provide evidence-backed explanations. Their approach treats the problem in two distinct stages, much like a triage nurse followed by a specialist doctor. The first stage, which they call the "Screener," is a fast, efficient detector that scans raw log messages without needing to break them down into smaller, pre-defined parts. Traditional methods often force logs into rigid templates first, a process that can strip away important details or introduce errors. This new Screener reads the raw text directly, using a streamlined attention mechanism that allows it to process long sequences of data quickly and accurately. In tests on two major datasets representing supercomputer and distributed storage systems, this Screener identified anomalies with near-perfect accuracy, correctly flagging almost every problematic session while ignoring normal activity. It acts as a highly reliable filter, ensuring that no real problem slips through the cracks.

Once the Screener flags a session as suspicious, the second stage, the "Reasoner," takes over to generate a human-readable explanation. Instead of guessing or relying solely on its internal knowledge, the Reasoner acts like a researcher digging through an archive. It retrieves specific examples of past anomalies and normal behavior from a database of historical logs. Using these retrieved examples as a guide, it constructs a structured report that links every claim it makes to a specific line of text in the current log and a corresponding line in the historical examples. This ensures that the explanation is not just a fluent story, but a grounded argument supported by concrete evidence. To guarantee reliability, an automated verifier checks every generated explanation against strict rules, ensuring that every cited line actually exists and that the reasoning holds up. In their experiments, every explanation generated by this system passed these rigorous checks, and human experts rated the quality of the explanations as highly correct and complete.

The researchers also addressed the practical reality that generating these detailed explanations is computationally expensive. Running a complex reasoning process on every single flagged anomaly would be too slow and costly for real-world use. To solve this, they introduced a smart gating mechanism that decides which anomalies deserve a full explanation. The system prioritizes sessions where the initial Screener was less confident, focusing its expensive reasoning power on the cases that are most ambiguous or difficult to understand. By doing so, the framework can explain a vast majority of unique failure patterns while using significantly fewer computing resources. The study found that by explaining only the most uncertain cases, the system could still cover the majority of known error types, striking a balance between thoroughness and efficiency. This work demonstrates that it is possible to build automated systems that not only detect problems with high precision but also provide the traceable, evidence-based reasoning that human operators need to fix them quickly and confidently.

Technical Summary: A Neural Screener–Reasoner Framework for Evidence-Grounded Log-Based Anomaly Detection and Explanation

Problem Statement

Log-based anomaly detection is critical for maintaining the reliability of large-scale computing systems, yet current deep learning approaches face a significant interpretability gap. While recent models achieve high detection accuracy, they typically provide only binary anomaly labels without explaining why a session is anomalous or identifying the specific log evidence supporting the decision. Furthermore, existing methods often rely on log parsing to convert raw messages into templates, a process that can strip dynamic runtime details, introduce semantic errors, and require maintenance as logging systems evolve. Additionally, the use of Large Language Models (LLMs) for generating explanations introduces risks of hallucination (fabricated evidence) and high inference costs, particularly when applied to every detected anomaly in high-velocity log streams.

Methodology

The paper proposes a unified Screener–Reasoner Framework designed to decouple efficient anomaly screening from evidence-grounded explanation generation. The framework consists of three primary components:

1. The Screener (Parser-Free Anomaly Detector)

The Screener is a lightweight, parser-free detector designed to process raw log sessions directly without input truncation.

Architecture: It utilizes a Linear Self-Attention (LSA) encoder. Unlike standard self-attention which has $O(n^2)$ complexity, LSA employs low-rank projection matrices to compress key and value sequences, reducing complexity to $O(nk)$ where $k \ll n$ . This enables efficient modeling of long raw-log sessions (thousands of tokens) without the quadratic memory and time costs of standard Transformers.
Input Processing: Raw log messages are tokenized directly using GPT-4 Byte Pair Encoding (BPE), preserving lexical units, symbols, and dynamic values. Sessions are constructed via session identifiers (e.g., block IDs in HDFS) or fixed-size sliding windows (e.g., BGL).
Training: To address class imbalance, the Screener employs Random Oversampling (ROS) on the training data and uses a weighted cross-entropy loss function to penalize misclassified anomalous sessions more heavily.

2. The Reasoner (Evidence-Grounded Explanation Generator)

For sessions flagged as anomalous, the Reasoner generates structured, forensic explanations using a Retrieval-Augmented Generation (RAG) architecture.

Evidence Store: A precomputed corpus of normalized historical sessions (from the training set only) serves as the evidence store. Log lines are normalized by replacing dynamic parameters (IPs, UUIDs, etc.) with typed placeholders to ensure structural pattern matching.
Retrieval: A BM25 retriever selects relevant evidence for a query session: four anomalous examples (E1–E4) and one normal contrast session (E5). The query session itself is designated E0.
Prompt & Generation: The Reasoner assembles a prompt containing the query (E0) and retrieved evidence (E1–E5). It instructs the LLM to generate a structured JSON output containing exactly three claim types: observation (from E0), pattern_match (comparing E0 to E1–E4), and contrast (comparing E0 to E5).
Verification: An automated, rule-based verifier checks the generated output for schema validity, citation integrity (ensuring cited line spans exist within the provided evidence), and grounding constraints (ensuring cited lines contain severity cues or anomaly indicators). This prevents hallucinated evidence references.

3. Confidence-Based Gating

To manage LLM inference costs, a gating module sits between the Screener and the Reasoner. It selects a subset of anomalous sessions for full explanation based on a budget ratio. The framework prioritizes sessions with low Screener confidence (high uncertainty), as these are deemed to benefit most from detailed reasoning, thereby balancing explanation coverage against computational cost.

Key Contributions

Unified Screener–Reasoner Formulation: The work formulates log anomaly analysis as a two-stage problem, separating efficient detection from evidence-grounded explanation to address the operational need for auditable diagnoses.
Parser-Free Linear Self-Attention Screener: It introduces a parser-free detector capable of processing long raw-log sequences efficiently without input truncation, avoiding the semantic loss associated with log parsing.
Structured Trace Schema & Automated Verification: It proposes a machine-verifiable trace schema where every explanatory claim must cite explicit evidence identifiers and line-level spans, coupled with an automated verifier to ensure grounding and prevent hallucinations.
Confidence-Based Gating: It evaluates a practical mechanism to control LLM inference costs by prioritizing uncertain detections, demonstrating a trade-off between budget and signature coverage.

Experimental Results

The framework was evaluated on the Blue Gene/L (BGL) and Hadoop Distributed File System (HDFS) datasets.

Detection Performance: The Screener achieved F1-scores of 0.999 (BGL) and 0.996 (HDFS), outperforming or matching state-of-the-art baselines (e.g., DeepLog, LogLAA) while operating on raw logs.
Efficiency: The Linear Self-Attention encoder demonstrated significant efficiency gains over standard Transformers. At a batch size of 16, LSA reduced peak memory usage by ~76% and improved throughput by ~12.9x compared to the standard Transformer, with only a 0.079% drop in F1-score.
Explanation Quality:
- Automated Verification: 100% of generated explanations passed rule-based verification checks for schema validity and citation integrity.
- Human Evaluation: On a stratified sample, explanations scored above 4.6/5 for correctness, completeness, and evidence grounding.
- RAG Ablation: Enabling retrieval (RAG-on) significantly improved Faithfulness (from ~0.59 to ~0.71 on BGL) and Grounding Breadth (from 0.0 to 0.667). Without retrieval, the LLM generated "phantom citations" (referencing non-existent evidence slots).
Gating Analysis: Confidence-based gating outperformed random, novelty, and combined strategies in signature coverage. At a 20% budget, it covered 66.2% of BGL signatures and 73.5% of HDFS signatures while reducing LLM token usage by ~80%.

Significance and Claims

The paper claims to shift the focus of log anomaly analysis from incremental gains in classification accuracy to the broader problem of explainable, auditable, and practically deployable analysis. It argues that high accuracy alone is insufficient for operational diagnosis; operators require traceable evidence to validate decisions.

The authors assert that their framework provides a practical path toward auditable, cost-aware, and deployable log anomaly analysis. By combining a parser-free screener with a retrieval-augmented, verified reasoner, the system ensures that explanations are not only readable but also factually grounded in specific log lines. The study concludes that the key challenge in the field is no longer just screening, but generating explanations that are traceable and verifiable, a capability their framework demonstrates through high verification pass rates and human evaluation scores.

Limitations Acknowledged: The authors note that the framework was evaluated using a single LLM backbone (GPT-5.1), tested only on BGL and HDFS (excluding microservices or security logs), and relied on a single human expert for evaluation. Additionally, the evidence store was static, and the impact of temporal drift in evolving log environments was not assessed.