Praxium: Diagnosing Cloud Anomalies with AI-based Telemetry and Dependency Analysis

This paper introduces Praxium, an AI-driven framework that combines telemetry monitoring with dependency analysis to automatically detect cloud anomalies and infer their root causes related to recent software rollouts, demonstrating high accuracy in both detection and causal inference across extensive trials.

The Gist

Imagine you run a massive, bustling restaurant kitchen. In the old days, you had one giant stove (a "monolith") where everything happened. If the soup got cold, you knew exactly where to look: the stove.

But today, modern cloud applications are like a giant kitchen with hundreds of tiny, specialized stations (microservices). One station chops onions, another fries the steak, a third plates the dessert, and they all talk to each other via walkie-talkies (RPCs). This setup is super flexible and efficient, but it's also a nightmare to debug.

If the steak comes out burnt, was it the grill station? Did the onion chopper accidentally spill oil on the grill? Or did the dessert station send a signal that confused the grill?

Praxium is the new, super-smart kitchen manager designed to solve this exact problem. Here is how it works, broken down into simple steps:

1. The Problem: The "Who Did It?" Mystery

In these modern kitchens, chefs (developers) are constantly swapping out tools and ingredients (software updates) while the kitchen is open for business. This is called CI/CD (Continuous Integration/Continuous Deployment).

Sometimes, a new tool is installed, and suddenly the whole kitchen starts slowing down or making mistakes. Traditional managers (Site Reliability Engineers) have to manually check hundreds of logs to find out which specific tool change caused the mess. It's like trying to find a specific grain of sand on a beach by looking at every single grain one by one. It takes too long, and by the time they find it, the customers are angry.

2. The Solution: Praxium's Three-Part Detective Team

Praxium is a framework that acts like a trio of detectives working together to solve the mystery instantly.

Detective A: The "Change Tracker" (Software Discovery)

• The Job: This detective keeps a super-precise diary of every single tool and ingredient that enters the kitchen.
• The Analogy: Imagine a security guard at the door who doesn't just count people, but scans every single item they bring in. If a new brand of knife or a different type of oil is brought in, the guard writes down exactly when it happened and which station it went to.
• Praxium's Tool: It uses a tool called PraxiPaaS to automatically scan the "container images" (the boxes the software comes in) and log every dependency change.

Detective B: The "Pulse Monitor" (Anomaly Detection)

• The Job: This detective watches the kitchen's vital signs (CPU usage, memory, speed) 24/7. They know what "normal" looks like.
• The Analogy: Think of a heart monitor in a hospital. It knows your resting heart rate. If your heart suddenly starts racing, the monitor beeps.
• Praxium's Tool: It uses a smart AI called a Variational Autoencoder (VAE). This AI learns the "normal heartbeat" of the kitchen. If the kitchen starts acting weird (e.g., the grill is running too hot or the sauce is moving too slow), the AI screams, "Something is wrong!"

Detective C: The "Time Traveler" (Root Cause Analysis)

• The Job: Once the alarm goes off, this detective figures out why.
• The Analogy: This is the most magical part. Imagine the detective can rewind time and run a simulation: "What if we hadn't installed that new knife at 2:00 PM? Would the steak still be burnt?"
• Praxium's Tool: It uses Causal Impact Analysis. It looks at the list of changes from Detective A and the weird behavior from Detective B. It simulates "what if" scenarios to see which specific software update caused the crash. It doesn't just guess; it calculates the probability that a specific change is the culprit.

3. The Secret Sauce: The "Critical Path" Map

Sometimes, the problem isn't where the error happened, but where it started. If the onion chopper messes up, the grill might get confused later.

Praxium draws a map of the kitchen (a Causal Graph). It knows that the Onion Station feeds the Grill Station. If the Grill Station is acting up, Praxium doesn't just look at the Grill; it looks at the Onion Station too. It filters out the noise and only investigates the "Critical Path"—the chain of stations that actually matters for that specific dish.

4. Does It Work?

The researchers tested Praxium in a simulated cloud environment (a "research cloud" called NERC).

• The Test: They intentionally broke things (spiking CPU, filling up disk space, causing memory leaks) by installing bad software updates.
• The Result: Praxium caught the problems 97%+ of the time.
• The Root Cause: Even when they installed five different updates in rapid succession (like a chef swapping five tools in one minute), Praxium could still pinpoint exactly which one caused the fire.

Why Should You Care?

For the average person, this means fewer app crashes, faster loading times, and less downtime.

For the tech world, Praxium is a game-changer because it stops relying on humans to manually dig through millions of lines of code. Instead, it uses AI to connect the dots between "we installed this update" and "the app broke," saving Site Reliability Engineers (the kitchen managers) hours of headache and keeping the digital kitchen running smoothly.

In short: Praxium is the smart, automated detective that watches your cloud kitchen, notices when something goes wrong, and instantly tells you exactly which new tool caused the mess, so you can fix it before the customers even notice.

Technical Summary

1. Problem Statement

Modern cloud-native applications increasingly rely on microservice architectures and CI/CD pipelines, which introduce high complexity and frequent changes.

• The Challenge: Traditional root cause analysis (RCA) relies heavily on manual expert intervention to trace anomalies (e.g., SLO violations) back to specific causes. This approach lacks scalability.
• The Gap: Existing automated tools often focus on high-level service rollouts or trace data but lack software-level granularity. They fail to distinguish between a general deployment change and a specific software package installation (e.g., a Python or JavaScript library update) that may introduce bugs, incompatibilities, or vulnerabilities.
• The Consequence: Administrators must manually sift through hundreds of installation logs to find the culprit, leading to slow resolution times and prolonged system downtime.

2. Methodology: The Praxium Framework

Praxium is a unified framework designed to correlate telemetry data with software dependency changes to automate anomaly detection and root cause inference. It consists of three core components:

A. Software Dependency Logging System (PraxiPaaS Integration)

• Goal: To capture granular software changes (Software Bill of Materials - SBOM) during microservice rollouts.
• Mechanism: The system integrates PraxiPaaS, an ML-based tool that inspects container image layers to identify package installations and version changes.
• Optimization: Instead of continuous scanning, Praxium triggers SBOM generation periodically (e.g., daily/weekly) or upon service redeployment. It compares current states against previous inspections to generate timestamped logs of specific package changes, storing them in a persistent database (MongoDB).

B. Anomaly Detection System (VAE-based)

• Goal: To detect deviations in system metrics (CPU, memory, disk, network) indicating an anomaly.
• Mechanism:
  • Utilizes a Variational Autoencoder (VAE) (adapted from the Prodigy framework) trained on historical "healthy" telemetry data.
  • Input: Sliding windows of metrics (e.g., container memory usage, CPU seconds) collected via Prometheus.
  • Process: The VAE reconstructs the input metrics. If the reconstruction error exceeds a learned threshold, the window is flagged as anomalous.
  • Triggering: To reduce false positives, the system requires $\tau$ (e.g., 2) consecutive anomalous windows before triggering an alert.
  • Architecture: A single VAE model is trained on concatenated metrics from all pods to maximize GPU utilization and parallelism, rather than training individual models per pod.

C. Root Cause Analysis (Causal Impact & Graphs)

• Goal: To pinpoint the specific software installation responsible for the anomaly.
• Mechanism:
  1. Causal Graph Construction: Upon an alert, Praxium builds a causal graph using Jaeger trace data to map upstream/downstream service dependencies. It identifies the Critical Path (all services influencing the anomalous pod).
  2. Candidate Filtering: It filters the installation logs to include only changes occurring within the Critical Path, discarding irrelevant changes.
  3. Causal Inference: It employs CausalImpact (a Bayesian counterfactual algorithm). The model generates counterfactual metric data for each candidate installation timestamp to simulate "what would have happened" without that change.
  4. Decision: By comparing observed anomalous data against counterfactuals, the system calculates the Average Treatment Effect (ATE) and p-values. The installation with the highest effect and statistical significance ($p < 0.01$) is identified as the root cause.

3. Key Contributions

1. Granular Root Cause Inference: Praxium is the first framework to explicitly link container image changes (specific package installations) to anomaly detection, moving beyond coarse-grained rollout data.
2. Integrated Framework: It combines unsupervised learning (VAE) for detection with causal inference (CausalImpact) and dependency graph analysis for diagnosis.
3. Practical Implementation: The system was implemented and evaluated on a real-world Kubernetes-based research cloud (NERC) using OpenShift AI and the DeathStarbench microservice benchmark.
4. Hyperparameter Analysis: The paper provides a comprehensive analysis of tuning window sizes, strides, and thresholds to balance detection speed, accuracy, and computational overhead.

4. Experimental Results

The system was evaluated using 75 trials involving four types of synthetic anomalies (CPU spike, Disk saturation, Memory leak, Network latency) injected via package updates.

• Anomaly Detection Performance:

  • Achieved a Macro-F1 score > 0.97 consistently across all anomaly types.
  • With a Threshold (T=2) (requiring 2 consecutive anomalous windows), the system achieved 1.0 Precision, Recall, and Accuracy for CPU and RAM anomalies, effectively eliminating false positives caused by noise.
  • Optimal Configuration: A window duration of 600 seconds (10 mins) and a stride of 300 seconds (5 mins) provided the best balance between detection granularity and system efficiency.

• Root Cause Inference (Causal Impact):

  • Accuracy: The system correctly identified the anomalous installation in 100% of trials (9/9), even when multiple installations occurred in rapid succession (intervals as short as 2 minutes).
  • Robustness: The causal analysis successfully distinguished the true cause from other recent installations, validating its utility in Agile environments with frequent deployments.

• Causal Graph Analysis:

  • In scenarios where an upstream service caused latency in a downstream service (without a direct installation in the downstream pod), the causal graph correctly identified the upstream installation as the root cause, demonstrating the system's ability to handle complex dependency chains.

5. Significance and Future Work

• Significance: Praxium addresses a critical gap in Site Reliability Engineering (SRE) by automating the correlation between software supply chain changes and system performance. It reduces the "Mean Time to Resolution" (MTTR) by eliminating manual log searching and providing actionable, statistically validated root causes.
• Scalability: The use of a single VAE model and periodic SBOM scanning ensures the system remains computationally feasible for large clusters.
• Future Directions:
  • Refining causal inference with deeper code-level dependency information.
  • Evaluating performance in massive-scale clusters (thousands of pods) to assess running costs and latency.
  • Adapting to finer-grained metric intervals (sub-30 seconds) for faster detection.

In conclusion, Praxium represents a significant step forward in AIOps (Artificial Intelligence for IT Operations), offering a robust, data-driven approach to maintaining reliability in complex, rapidly evolving cloud environments.