🤖 machine learning

EvoCause: LLM-Guided Evolution of Causal Graphs for Root Cause Analysis

EvoCause is a novel framework that leverages large language models to iteratively refine causal graphs using expert diagnosis labels for improved root cause analysis in complex systems, validated by a new expert-annotated benchmark (TeleRCA) and demonstrating significant performance gains over traditional causal discovery methods.

Original authors: Lei Zan, Keli Zhang, Shifeng Xie, Jiale Zheng, Zehao Xiao, Zhiwei Dong, Ke Zhang, Ruichu Cai, Malik Tiomoko, Lujia Pan

Published 2026-07-31

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Lei Zan, Keli Zhang, Shifeng Xie, Jiale Zheng, Zehao Xiao, Zhiwei Dong, Ke Zhang, Ruichu Cai, Malik Tiomoko, Lujia Pan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the captain of a massive, high-tech spaceship. Suddenly, the ship's computer starts screaming with thousands of alarms: "Engine hot!", "Hull pressure dropping!", "Navigation glitch!" It's a chaotic storm of noise. Your job, known as Root Cause Analysis (RCA), is to figure out which tiny, specific alarm started the whole mess so you can fix it before the ship crashes. In the real world, this happens in telecommunication networks and cloud systems every day. When one part breaks, it triggers a domino effect of errors.

To solve this, scientists often try to draw a "map" of how alarms trigger each other, like a family tree of mistakes. They use math to learn this map from past data. But here's the problem: math isn't perfect. Sometimes the map has wrong lines or missing connections. Usually, if the map is wrong, the computer gets lost. But what if you could ask a human expert, "Hey, you said this was the root cause last time, but your map says it wasn't," and then use a super-smart AI to fix the map based on that feedback? That is exactly the question this paper tackles: how to take a rough, math-made map of errors and polish it using human wisdom and a powerful language AI, so it becomes a highly effective guide for fixing future disasters.

The Problem: The "Broken Map" of Chaos

Imagine a giant, tangled ball of yarn where every knot is an alarm. When one knot pulls, it yanks others. To fix the system, you need to find the very first knot that was pulled. Scientists have tried to untangle this by using algorithms to learn the pattern of the yarn from history. They call this a "causal graph"—a fancy way of saying a map showing which alarm causes which.

However, these math-only maps are often messy. They might draw a line between two alarms that never actually talk to each other, or they might miss a secret shortcut. Worse, these maps are usually "fixed." Once the computer draws the map, it sticks with it, even if a human expert looks at a past disaster and says, "No, that wasn't the cause; this was!" The old methods ignore the expert's voice.

The Solution: EvoCause, the "Map Refiner"

The authors of this paper introduce a new system called EvoCause (Evolve Causal Graph). Think of EvoCause as a brilliant editor who takes a rough draft of a map and rewrites it until it's as accurate as possible for the task, using a special tool: a Large Language Model (LLM).

Here is how the story unfolds in their method:

The Rough Draft (Stage I): First, the system uses standard math tools to draw a basic map of how alarms trigger each other. This map is a good start, but it's not perfect.
The Expert Review (Stage II): Now comes the magic. The system looks at a pile of past disasters where human experts already said, "Here is the true root cause." It compares the experts' answers with what the rough map predicted.
- The Mismatch: If the map says "Alarm A caused the crash," but the expert says "No, Alarm B was the culprit," the system knows the map is wrong.
- The LLM's Job: Instead of just deleting the map, the system asks a super-smart AI (the LLM) to suggest fixes. The AI looks at the mismatch and the names of the alarms (like "Server Overload" or "Network Lag") and says, "Ah! Maybe we need to draw a line from B to A, or remove the line from A to C."
- The Safety Check: The AI's suggestions are wild guesses, so a strict, rule-following computer program checks them. It makes sure the new map doesn't create impossible loops (like A causes B, B causes C, and C causes A) and that the names match up. If the suggestion is safe, the map gets updated.
The Final Map: The system repeats this process, trying different edits, until it finds the map that best matches the experts' past diagnoses. It is important to note that the experts' feedback often narrows down the possibilities to a set of good maps rather than pointing to one single "perfect" map. EvoCause finds the one that works best for the job.

The Results: A Smarter Map

The authors tested this idea in two ways: with fake data where they knew the "true" answer, and with real data from a massive telecommunications network in Indonesia.

On the Fake Data:
They created 10 different fake worlds with known rules. When they started with a basic math map and let EvoCause refine it, the results were impressive. The system became much better at finding the true root cause.

It improved the accuracy of finding the right root alarm by 11.59 percentage points.
It improved the accuracy of getting the entire correct set of root causes for a specific incident by 9.40 percentage points.
It also made the map itself more accurate, reducing the number of wrong lines by a measurable amount.

On the Real Data (TeleRCA):
The authors also released a new, huge dataset called TeleRCA, containing 485,681 alarm events from 194 different alarm types across 5,621 resources. This is a real-world treasure trove of network chaos.

When they applied EvoCause to this real data, the improvement was even bigger. Starting with a basic map, they boosted the accuracy of finding the right root alarm from 65.18% to 92.58%.
They also tested if the AI needed to know the names of the alarms (like "CPU Overheat") or if just numbers would work. They found that knowing the names helped! When they hid the names and used anonymous IDs, the accuracy dropped by 6.12 percentage points. This suggests that the AI uses the meaning of the alarm names to make smarter guesses about how to fix the map.

What This Means

The paper shows that we don't have to choose between "math-only" maps and "human-only" guesses. By combining them, we get something better. The LLM acts as a creative editor that suggests how to fix the map based on human feedback, while a strict computer ensures the map stays logical.

Crucially, once the map is refined, the system doesn't need the AI or the human experts anymore. It just uses the final, polished map to predict the root cause of new disasters instantly. It's like teaching a student the rules of the road by showing them past accidents; once they learn, they can drive safely on their own without needing a teacher in the passenger seat.

The authors are careful to say they haven't "solved" everything. They found that the experts' feedback usually narrows down the possibilities to a set of good maps, but it doesn't always point to one single "perfect" map. However, their method finds a map that works incredibly well for the job of fixing the network, making it a powerful new tool for keeping our connected world running smoothly.

Technical Summary: EvoCause

Problem Statement
Modern telecommunication, cloud, and microservice systems generate correlated alarm cascades ("alert storms") when components fail. Root Cause Analysis (RCA) aims to identify the minimal set of initiating alarms (root causes) responsible for these cascades. A common approach involves learning a causal graph (Directed Acyclic Graph or DAG) from observational logs and predicting zero-in-degree nodes in the subgraph induced by an incident. However, standard causal discovery algorithms often produce spurious or missing edges due to data noise and strict assumptions. Crucially, existing methods fail to leverage a complementary information source: historical incident logs curated by Site Reliability Engineering (SRE) teams, which contain expert-labeled root causes. These labels provide constraints on which alarms should be source nodes but do not directly specify the edge edits required to satisfy these constraints, making the feedback indirect and underdetermined.

Methodology: EvoCause
The authors propose EvoCause (Evolve Causal Graph), a framework that refines an initially learned causal graph using expert feedback via Large Language Models (LLMs). The process consists of two stages:

Learning Stage:
- Initial Graph Construction: An initial alarm propagation graph (APG), $G_0$ , is learned from historical, label-free alarm sequences using off-the-shelf causal discovery algorithms (e.g., PC, NOTEARS, THPs).
- Iterative Refinement: The framework uses a labeled alignment set (incidents with expert root-cause labels) to refine $G_0$ $G_{0}$ into an optimal graph $G^*$ $G^{*}$ . This is achieved through a batch-based iterative optimization loop:
  - Prediction: The current graph predicts root causes for a batch of labeled incidents.
  - Evidence Aggregation: Mismatches between predicted and expert-labeled root sets are aggregated into an evidence set.
  - LLM Proposal: An LLM receives the current graph, its topological order, the evidence set, and optimization history. It proposes a small set of semantically plausible graph edits (edge additions, removals, or reversals) to resolve the mismatches. The LLM acts as a proposal model, not a causal oracle.
  - Deterministic Validation: A deterministic code validates the proposed edits to ensure node identity correctness and acyclicity (DAG property).
  - Selection: The graph is updated only if valid. The system retains the graph that maximizes the "Case Exact Match" (Case EM) on the alignment set, using Node F1 as a tie-breaker.
Inference Stage:
- At test time, RCA is performed using only the refined graph $G^*$ . No LLM calls or expert labels are required.
- For a new incident, the system restricts $G^*$ to the observed alarm types and identifies all zero-in-degree nodes in the induced subgraph as the predicted root causes. This ensures transparent, traceable predictions.

Key Contributions

Methodological: The paper formulates incident-level expert feedback as source-node constraints on incident-induced subgraphs. It introduces EvoCause, a discovery-agnostic framework that integrates LLM-guided semantic graph editing with deterministic structural validation. This allows the refinement of graphs from any causal discovery backbone without modifying their internal learning procedures.
Data: The authors release TeleRCA, a large-scale, expert-annotated benchmark derived from a production telecommunication network. It contains 10,922 incidents, 485,681 alarm events, 194 alarm types, and 5,621 resources. Unlike previous datasets, it pairs production alarm sequences with incident-level expert root-cause annotations.
Experimental: The authors evaluate the framework on ten synthetic DAGs and the TeleRCA dataset. They demonstrate that EvoCause improves RCA performance (Node F1 and Case EM) and graph reconstruction metrics (Graph F1, nSHD) across different discovery backbones. They also conduct a controlled ablation study on TeleRCA by replacing human-readable alarm titles with anonymous identifiers to quantify the contribution of semantic information.

Results

Synthetic Data: Initialized with the PC algorithm, EvoCause outperforms the unrefined PC baseline, increasing Node F1 by 11.59 percentage points, Case EM by 9.40 percentage points, and Graph F1 by 4.59 percentage points, while reducing the normalized Structural Hamming Distance (nSHD) by 0.2379. The framework shows robustness across different LLM backbones (Qwen3-4B and Qwen3-32B).
TeleRCA (Real-World): On the production dataset, EvoCause significantly improves RCA performance. Using PC initialization, Node F1 increased from 65.18% to 92.58%, and Case EM from 53.98% to 89.61%.
Semantic Contribution: Replacing alarm titles with anonymous identifiers on TeleRCA resulted in a performance drop (Node F1 decreased by 6.12 points; Case EM by 8.04 points), indicating that human-readable alarm-title information provides valuable guidance for the LLM's graph-edit proposals.
Convergence: EvoCause achieves nearly identical high performance regardless of the initial discovery backbone (PC, NOTEARS, or THPs), suggesting that the label-feedback alignment effectively reduces dependence on the initial inductive bias of the discovery algorithm.

Significance and Claims
The paper claims that EvoCause successfully closes the loop between expert diagnoses and causal graph learning. By treating expert labels as constraints on source nodes rather than direct edge supervision, and using LLMs to navigate the combinatorial space of graph edits, the method produces refined graphs that are structurally valid and consistent with expert annotations. The authors emphasize that the refined graph alone is sufficient for inference, eliminating the need for LLM calls or expert labels during deployment. The work highlights that while expert labels do not uniquely identify a single data-generating DAG (due to the underdetermined nature of the problem), they are sufficient to identify an "RCA-effective" graph set that significantly improves diagnostic accuracy. The release of TeleRCA provides a new standard for evaluating event-sequence-based RCA methods.

The Problem: The "Broken Map" of Chaos

The Solution: EvoCause, the "Map Refiner"

The Results: A Smarter Map

What This Means

Technical Summary: EvoCause

More like this