Detect, Localize, and Explain: Interactive Hierarchical Log Anomaly Analytics with LLM Augmentation

Technical Summary: Krone-viz – Interactive Hierarchical Log Anomaly Analytics with LLM Augmentation

Problem Statement
Modern data-intensive infrastructures rely heavily on logs for system monitoring and anomaly diagnosis. However, existing log anomaly detection methods face significant limitations. Most traditional approaches operate on flat log sequences, producing a single anomaly score for an entire sequence without the ability to interpret results or localize specific anomalous segments. While recent Large Language Model (LLM)-based approaches improve interpretability through natural language explanations, they still operate on flat sequences. Furthermore, these LLM-based methods suffer from context window limitations and prohibitively high computational costs when applied to massive log datasets. There is a lack of systems that can effectively decompose logs into semantic units to enable precise localization and explanation while managing LLM costs.

Methodology: The Krone Framework
The paper introduces Krone, a framework designed to transform flat log sequences into semantically coherent hierarchical units. Krone-viz is the interactive visualization system built upon this framework to make these capabilities actionable for engineers. The methodology rests on three core innovations:

1. Krone Log Abstraction Model (Krone-Tree):
   Krone introduces a hierarchical abstraction that reconstructs execution semantics directly from logs. It defines a three-level semantic hierarchy:

   • Entities: Correspond to system components.
   • Actions: Capture operations performed.
   • Statuses: Reflect the outcome of each action.
     Using LLM-based Named Entity Recognition (NER) on log templates, Krone constructs a Krone-Tree. This tree allows the decomposition of long, flat log sequences into Krone Seqs—contiguous, semantically coherent segments represented as node sequences across entity, action, and status levels.

2. Hierarchical and Modular Anomaly Detection:
   Anomaly detection is formulated as modular tasks over the decomposed Krone Seqs. The system employs a bottom-up orchestration strategy:

   • Lightweight Filtering: A pattern-matching (PM) detector first evaluates Krone Seqs to filter out known normal patterns efficiently.
   • Selective LLM Invocation: Only uncertain or unknown Krone Seqs trigger LLM-based reasoning.
   • Early Stopping: The system examines lower-level segments (Status) first. If an anomaly is detected at a lower level, the process stops immediately to avoid unnecessary computation on higher-level compositions, as anomalies propagate upward.
   • Contextual Verification: The hierarchy provides a well-defined semantic scope, allowing the LLM to ground its reasoning in normal behavioral patterns (e.g., typical action transitions within an entity) using in-context examples.

3. Knowledge Base for Amortized Detection:
   Recognizing that Krone Seqs are atomic execution units that frequently recur across different log sequences, Krone maintains a hierarchical Knowledge Base.

   • Reuse: Once a Krone Seq is evaluated (either by PM or LLM), the result and explanation are stored.
   • Cost Reduction: During detection, the system queries this knowledge base first, invoking the LLM only if no matching entry exists. This significantly reduces redundant LLM calls.
   • Continuous Refinement: The knowledge base supports human-in-the-loop corrections, allowing users to revise stored knowledge to improve future detection and explanation quality.

Krone-viz Demonstration
The paper presents Krone-viz, an interactive platform demonstrated on the widely used HDFS benchmark dataset. The system visualizes the four stages of the Krone workflow:

1. Hierarchy Extraction: Users load log templates, and the system constructs the Krone-Tree, visualizing the semantic relationships between entities, actions, and statuses.
2. Training Data Processing: Users observe how normal log sequences are decomposed into hierarchical Krone Seqs and stored as ground-truth execution units.
3. Anomaly Detection: Users trigger detection on test sequences. The interface visualizes the bottom-up modular process, highlighting segments flagged by the pattern matcher and providing LLM-generated explanations for confirmed anomalies.
4. Knowledge Management: Users can explore the accumulated knowledge base, view statistics for specific nodes, inspect detailed detection results, and override or revise stored knowledge to guide future analysis.

Key Results
The paper reports comprehensive improvements of the Krone framework over prior methods based on real-world public benchmarks:

• Accuracy: The F1-score improved from 82.76% (prior methods) to 92.83%.
• Data Efficiency: The approach reduced the required data space by a factor of 117.3×.
• LLM Resource Efficiency: The system reduced LLM usage to 1.1%–3.3% of the test data size by leveraging the knowledge base and selective invocation.

Significance
The paper claims that Krone-viz addresses the critical gap in log analysis by making hierarchical decomposition, anomaly localization, and explanation interpretable and actionable for software engineers and system operators. By integrating efficient LLM usage with a modular, hierarchical framework, Krone enables:

• Precise Localization: Identifying specific anomalous segments within execution contexts rather than labeling entire sequences.
• Cost-Effective Scalability: Drastically reducing computational costs and data requirements compared to flat-sequence LLM approaches.
• Human-in-the-Loop Analysis: Empowering users to review, correct, and refine AI-generated insights, thereby mitigating hallucination risks and building a reusable knowledge foundation for continuous system improvement.

The work is presented as a demonstration of how to achieve scalable, interpretable log analysis through efficient, cost-aware LLM integration.