💻 computer science

Explainable Agentic Decision Support for Project Governance in Agile–DevOps: A Multi-Agent Governance Framework for Project Managers

This paper presents the AgileOps Agentic Framework (AAF), a multi-agent decision-support system that integrates specialized DevOps, SRE, FinOps, and DevSecOps reasoning with explainable, evidence-grounded analysis to help Project Managers interpret fragmented operational telemetry into actionable governance recommendations, validated through controlled scenarios and real-world microservice benchmarks.

Original authors: Suresh Kandasamy, Suresh Arumugam, Cynthia Jayapal

Published 2026-07-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Suresh Kandasamy, Suresh Arumugam, Cynthia Jayapal

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the Project Manager of a massive, high-speed train system. This train represents your software company, and it runs on a complex network of tracks, engines, and signals known as Agile–DevOps.

Every second, thousands of sensors on the train (the software) are shouting out data: "Engine temperature rising!", "Ticket sales are up!", "Security gate is open!", "Fuel costs are spiking!".

The Problem:
Right now, these shouts are coming from different departments. The Engineers (DevOps) are talking about code. The Mechanics (SRE) are talking about reliability. The Accountants (FinOps) are talking about fuel costs. The Security Guards (DevSecOps) are talking about locks and keys.

As the Project Manager, you are standing in the middle of this chaos. You have all the data, but it's scattered, confusing, and often contradictory. You don't know if you should stop the train, speed it up, or just keep an eye on things. You need a clear, trustworthy answer, but the raw data is too noisy to understand.

The Solution: The "AAF" (AgileOps Agentic Framework)
The authors of this paper built a digital "Chief of Staff" to help you. They call it the AgileOps Agentic Framework (AAF). Think of it not as a robot that drives the train for you, but as a smart, multi-expert advisory team that sits in your office, reads all the sensor data, and gives you a clear, written report on what to do.

Here is how this team works, using simple analogies:

1. The Four Expert Advisors (The Agents)

Instead of one AI trying to know everything, the framework uses four specialized "agents," each with a specific job:

The DevOps Agent: The "Delivery Expert." They check if the software is ready to ship and if the assembly line is running smoothly.
The SRE Agent: The "Reliability Expert." They check if the train is likely to break down, how fast it's going, and if passengers are safe.
The FinOps Agent: The "Budget Expert." They check if the train is burning too much fuel or if the ticket prices are too high.
The DevSecOps Agent: The "Security Expert." They check for hackers, broken locks, or safety violations.

2. The "Council Meeting" (Consensus & RAR)

Once these four experts look at the data, they don't just shout their opinions. They hold a meeting.

Consensus: They try to agree. If the Budget Expert says "Stop!" because of cost, but the Delivery Expert says "Go!" because of speed, the system calculates a "Consensus Score."
The "Re-Grounded" Check (RAR): If the experts are too confused or disagree too much (low consensus), the system doesn't guess. Instead, it says, "Wait, we need more proof." It goes back to the sensors to gather more specific evidence (like checking the fuel gauge again or re-reading the security logs) until they can agree. This prevents the system from making wild guesses.

3. The "Scorecard" (Utility-Based Scoring)

Even if the experts agree, they might still have different priorities. The system uses a Scorecard to decide the best move. It weighs three things:

Performance: Will the train run faster?
Cost: Will we save money?
Risk: Will we avoid a crash?

The system calculates a "Utility Score" for every possible action (like "Delay the release," "Fix the bug," or "Do nothing"). It picks the action with the highest score, balancing speed, money, and safety.

4. The "Translator" (Explainable Output)

This is the most important part for you, the Project Manager. The system doesn't just give you a number. It has a Translator that writes a plain-English report.

No Magic: The Translator is strictly forbidden from making things up. It can only write what the experts and the scorecard decided.
Traceability: If the report says, "We should delay the release," it must also say, "Because the Security Expert found a lock issue and the Budget Expert said it's too expensive to fix right now."
The Result: You get a clear, readable summary that tells you what happened, why it happened, and what you should do, with a direct link back to the raw data.

What Did They Test?

The authors didn't just build this; they tested it in three ways:

The "Mock Exam": They created 120 fake scenarios (like "The server crashed" or "Costs went up") to see if the system could identify the problem and suggest the right action. It got about 87% of the problem types right and 79% of the action suggestions right, beating older, simpler methods.
The "Manager's Questions": They asked the system 100 questions a Project Manager might ask (e.g., "Should we release this?"). Even when the information was vague, the system gave consistent, logical answers that matched what a human expert would likely decide.
The "Live Fire Drill": They ran the system on a real, small software simulation (called "Sock Shop") that was intentionally broken in various ways. The system successfully took the messy, live data from the broken software and turned it into a clear governance report.

The Bottom Line

This paper introduces a tool that acts as a bridge between the noisy, technical world of software engineers and the decision-making world of Project Managers.

It doesn't try to fix the software automatically. Instead, it acts as a super-organized, evidence-based advisor that:

Listens to all the different experts.
Checks its work if it's unsure.
Balances speed, cost, and safety.
Explains its reasoning in plain English, so you never have to guess why it made a suggestion.

The goal is to help Project Managers make better, faster, and more confident decisions in a chaotic digital environment, without needing to become a data scientist themselves.

Technical Summary: Explainable Agentic Decision Support for Project Governance in Agile–DevOps

Problem Statement

Modern Agile–DevOps environments generate vast amounts of operational telemetry through deployment pipelines, observability platforms, cloud infrastructure, and security tooling. While this data is critical for engineering teams, it remains fragmented across multiple technical domains (DevOps, SRE, FinOps, DevSecOps) and is often presented in formats that are difficult for Project Managers to interpret holistically.

The core challenge identified is the disconnect between low-level operational telemetry and high-level governance interpretation. Project Managers currently lack a unified, actionable view to assess release readiness, operational risk, infrastructure costs, and security posture simultaneously. Existing AIOps and automation solutions primarily focus on infrastructure orchestration, anomaly detection, and autonomous remediation for engineering teams, offering limited support for managerial decision-making under uncertainty. Furthermore, general-purpose Large Language Models (LLMs) used for explanation often suffer from hallucinations and lack traceability to structured evidence, posing risks for governance decisions that require accountability.

Methodology: The AgileOps Agentic Framework (AAF)

The paper proposes the AgileOps Agentic Framework (AAF), a governance-oriented multi-agent decision-support system designed to translate fragmented operational evidence into explainable governance recommendations. AAF is explicitly not an autonomous remediation system; its purpose is to support Project Managers by interpreting operational conditions and guiding governance responses.

Architecture and Workflow

The framework operates through five distinct stages:

Operational Evidence Ingestion: Telemetry from pipelines, observability, and cloud infrastructure is transformed into structured evidence packages ( $E$ ) containing signals such as rollback events, latency metrics, cost indicators, and vulnerability findings.
Domain-Specific Reasoning Agents: Four deterministic agents independently analyze the evidence:
- DevOps RA: Evaluates deployment stability, CI/CD failures, and release conditions.
- SRE RA: Assesses reliability, latency, availability, and restart behavior.
- FinOps RA: Analyzes operational costs and infrastructure utilization efficiency.
- DevSecOps RA: Evaluates vulnerabilities, policy violations, and security risks.
  Each agent outputs a claim ( $c_i$ ), a confidence score ( $s_i$ ), and supporting evidence ( $\epsilon_i$ ).
Consensus Evaluation & Re-Grounded Agentic Reasoning (RAR):
- A consensus score ( $C$ ) is calculated based on the mean confidence and domain alignment.
- If $C$ falls below a threshold ( $\tau = 0.65$ ), the Re-Grounded Agentic Reasoning (RAR) process is triggered. RAR enriches the evidence package with additional context and re-evaluates the agents to improve alignment. This is capped at two iterations to preserve deterministic behavior. If consensus remains low, the case is marked as unresolved uncertainty rather than forcing a high-confidence decision.
Utility-Based Governance Scoring:
- Candidate governance actions (e.g., rollback, scale adjustment, patch, observe) are evaluated using a utility function: $U(a_i) = w_p P_i + w_c C_i + w_r R_i$ .
- Weights are fixed for the study: Performance ( $w_p=0.4$ ), Cost Efficiency ( $w_c=0.3$ ), and Risk Reduction ( $w_r=0.3$ ).
- The action with the maximum utility is selected.
Evidence-Grounded Explanation Generation:
- A constrained language model generates governance summaries for Project Managers.
- Crucially, the LLM is restricted to the structured outputs of the framework (claims, evidence, scores, consensus status). It cannot introduce new evidence or alter the selected governance action, ensuring traceability and reducing hallucination risks.

Evaluation Design

The framework was validated through three complementary studies:

Controlled Cross-Domain Governance Scenarios: 120 curated scenarios testing domain identification and action selection across DevOps, SRE, FinOps, and DevSecOps.
Project Manager Prompt Evaluation: 100 governance-oriented prompts evaluated under "Blind Evidence" (incomplete context) and "Aligned Evidence" (curated labels) modes.
Runtime Telemetry Pilot: A pilot using the Sock Shop microservices benchmark, generating 20 runtime cases and 733 operational artifacts (logs, container status, metrics) under controlled perturbations.

Key Results

Controlled Governance Scenarios (Study 1)

Performance: The full AAF configuration achieved a Domain Match Rate of 0.875 and a Governance Action Match of 0.792, significantly outperforming traditional baselines (0.633 domain match) and single-agent baselines.
Ablation Insights:
- Removing the Utility Scoring layer resulted in a governance action match of 0.00, demonstrating that identifying the operational domain is insufficient without a structured mechanism to weigh trade-offs (performance vs. cost vs. risk).
- Removing RAR or Consensus mechanisms showed smaller performance drops, suggesting these components primarily manage uncertainty rather than drive primary classification accuracy.
Utility Profile: The framework successfully balanced competing objectives, with Cost Efficiency scoring highest (0.68) in the utility profile, followed by Risk Reduction (0.52) and Performance (0.49).

Project Manager Prompt Evaluation (Study 2)

Blind Evidence Mode: Achieved a 0.78 domain match and a 0.79 Acceptable Governance Action Match (allowing for equivalent operational actions), despite the ambiguity of the prompts. RAR was triggered in 66% of cases, highlighting the prevalence of uncertainty in cross-domain governance.
Aligned Evidence Mode: Achieved 1.00 domain match and 1.00 acceptable action match, confirming internal consistency when evidence is complete.
Grounded Explanation: The explanation layer maintained 100% domain and action consistency with the underlying framework logic. The Unsupported-Claim Risk was low (0.281), indicating that constraining the LLM to structured evidence effectively prevents hallucinations.

Runtime Telemetry Pilot (Study 3)

Using Sock Shop artifacts, the framework achieved a 0.65 domain match and 0.80 governance action match.
The lower domain match rate in runtime scenarios (compared to controlled scenarios) reflects the noisier nature of live telemetry where signals (e.g., restart loops) can be interpreted as either DevOps or SRE issues. However, the framework maintained strong governance action consistency, suggesting it can recommend useful actions even when the dominant domain is ambiguous.

Significance and Claims

The paper claims that the AgileOps Agentic Framework addresses a critical gap in Agile–DevOps governance by bridging the divide between operational telemetry and managerial decision-making. Its primary contributions are:

Governance-Oriented Multi-Agent Reasoning: It integrates DevOps, SRE, FinOps, and DevSecOps perspectives into a unified workflow, moving beyond single-domain optimization.
Structured Uncertainty Management: The Re-Grounded Agentic Reasoning (RAR) mechanism explicitly handles uncertainty by refusing to force high-confidence decisions when evidence is insufficient, preserving governance integrity.
Utility-Based Trade-off Evaluation: By employing a weighted utility model, the framework provides a reproducible method for balancing competing governance goals (reliability, cost, security, speed) rather than optimizing for a single metric.
Traceable Explainability: The separation of deterministic reasoning from language-model-based explanation generation ensures that governance recommendations are grounded in verifiable evidence, reducing the risk of unsupported claims and enhancing accountability for Project Managers.

The authors emphasize that AAF is designed as a decision-support tool, not an autonomous remediation system. Its value lies in providing Project Managers with traceable, operationally grounded interpretations to guide governance responses in uncertain delivery environments. The work establishes a reproducible experimental pipeline linking raw telemetry to governance interpretations, offering a foundation for future research into adaptive weighting and human-centered validation.