Technical Summary: Governing AI-Assisted Security Operations

1. Problem Statement

Engineering managers face a critical adoption dilemma: Generative AI, Retrieval-Augmented Generation (RAG), and coding agents can accelerate technical workflows in Security Operations Centers (SOCs), but their unmanaged use introduces significant risks in high-risk, privileged telemetry environments. While AI can generate code-like artifacts, summarize evidence, and propose actions, it often blurs accountability and creates vulnerabilities regarding privacy, cost, auditability, and decision quality.

The specific problem addressed is how to govern AI-assisted operational decision support in security functions without weakening control. The paper uses Kusto Query Language (KQL) within the Microsoft Azure security ecosystem (Defender XDR, Sentinel, Azure Data Explorer) as a bounded technical instantiation. Although KQL queries are typically read-only, unmanaged AI-generated queries can still cause harm through:

• Broad scans leading to cost overruns and performance degradation.
• Sensitive-field exposure (e.g., command lines, URLs, email subjects) violating privacy.
• Schema invalidity (referencing unavailable tables or columns).
• Misleading interpretations of evidence or stale threat intelligence.
• Uncontrolled execution that bypasses established governance policies.

The core challenge is not merely whether an AI model can generate a syntactically correct query, but whether an organization can govern the lifecycle of that artifact—from proposal and validation to execution, auditing, and improvement.

2. Methodology

The study employs Design Science Research (DSR) to develop and evaluate a socio-technical artifact intended to improve organizational capability. The methodology follows the DSRM process: problem identification, objective definition, design and development, demonstration, evaluation, and communication.

• Artifact: A Governed AI Query Broker. This is a socio-technical system that separates AI planning from operational execution.
• Context: The artifact is instantiated within Microsoft-centric threat hunting workflows but is designed with transferable management principles.
• Evaluation Strategy: The study adopts a staged evaluation protocol to avoid premature claims of production effectiveness. It moves from:
  1. Offline/Package Validation: Testing against synthetic data and schema-normalized traces.
  2. Historical Replay: Recovering evidence from closed cases.
  3. Purple-Team Simulation: Testing detection utility against seeded behaviors.
  4. Shadow-Mode Pilot: Limited live telemetry with analyst oversight and no automated response.
  5. Controlled Production: Limited deployment with strict budget and audit monitoring.
• Baselines: The proposed broker is compared against manual analyst-written KQL, static approved query libraries, and unbrokered AI-generated KQL.

3. Key Contributions and Artifact Design

3.1 The Governed AI Query Broker Architecture

The central contribution is a management framework instantiated as a brokered architecture. The core control flow is:
Analyst Request → AI Hunt Plan → RAG Grounding → Approved Template Selection → KQL Validation → Policy Decision → Read-Only API Execution → Result Normalization → Analyst Feedback.

Key components include:

• AI Planner: Converts natural language intent into a structured hunt plan (intent, entities, source selection).
• RAG Grounding Layer: Retrieves current schemas, approved templates, playbooks, and threat intelligence context to prevent hallucinations.
• Template Resolver: Maps the plan to parameterized, approved KQL patterns rather than generating free-form code.
• KQL Broker: The enforcement engine that validates queries against identity, policy, source allowlists, cost limits, time/row bounds, and approval rules.
• Source Adapters: Isolate execution to specific read-only APIs (Microsoft Graph, Log Analytics, Sentinel Data Lake, ADX).
• Feedback Layer: Records analyst disposition and evaluation labels to support continuous improvement.

3.2 Design Propositions

The study establishes six design propositions (P1–P6) that form the theoretical backbone of the framework:

• P1 (Brokered Execution): Separating AI planning from executable access increases control by converting AI output into a policy-evaluated request.
• P2 (Template-First Generation): Queries derived from approved, parameterized templates with owners and version history are more governable.
• P3 (Provenance-Aware Intelligence): Threat intelligence must preserve source, confidence, and validity before influencing query generation.
• P4 (Evidence-Bounded Evaluation): Claims about the artifact must be matched to appropriate evidence levels (from offline tests to naturalistic deployment).
• P5 (Sandboxed Agentic Contribution): Coding agents should be restricted to non-production repositories and pull requests, never direct production API access.
• P6 (Role-Centered Governance): Sustainable adoption requires distributed responsibilities across SOC leadership, detection engineering, platform engineering, and AI governance, rather than centralizing in the model provider.

3.3 Operating Model and Maturity

The paper defines a maturity model for adoption:

• Stage 0 (Offline): Synthetic data and mock adapters.
• Stage 1 (Controlled POC): Non-production environment with package validation.
• Stage 2 (Shadow-Mode): Limited live telemetry with analyst oversight.
• Stage 3 (Controlled Production): Trained analysts, audited execution, and budget controls.
• Stage 4 (Agent-Assisted Engineering): Coding agents propose changes via pull requests under human review.

It also specifies an Engineering Review Board (ERB) gate structure to transition between stages, requiring specific evidence (e.g., audit logs, cost reports, RACI matrices) before promotion.

4. Results and Evidence Boundaries

The paper does not claim to have demonstrated multi-enterprise production effectiveness or specific detection lift metrics. Instead, the "results" are the design plausibility and the evaluation protocol itself.

• Traceability: The artifact enables end-to-end traceability from user request to analyst disposition, allowing managers to distinguish between AI planning failures, governance control failures, and operational fit failures.
• Risk Mitigation: The design explicitly addresses failure modes such as schema hallucination, excessive data scanning, sensitive data exposure, and prompt injection through specific controls (e.g., RAG grounding, row caps, treating logs as evidence rather than instructions).
• Reproducibility: The study proposes a reproducibility package including synthetic tables, versioned templates, policy files, and manifests, allowing the artifact to be inspected without exposing live enterprise telemetry.

5. Significance and Claims

The paper's significance lies in reframing AI-assisted security operations as an engineering management problem rather than a purely technical or model-performance issue.

• Management Framework: It provides a concrete framework for converting AI from an informal productivity aid into an accountable, socio-technical operating capability.
• Governance over Fluency: The central message is that the value of AI in high-risk engineering work depends less on model fluency and more on accountable operating models, evidence gates, and managerial control.
• Transferability: While instantiated in KQL/Microsoft, the contribution is the governance architecture (roles, templates, policy gates, evidence maturity) applicable to other high-risk technical functions like infrastructure automation and safety-critical analytics.
• Modest Claims: The authors explicitly state that the artifact is not a replacement for SOC analysts or a universal detection algorithm. It is a governance architecture designed to help managers control AI-enabled work, with production effectiveness claims reserved for future naturalistic evaluation.

In summary, the study contributes a design science artifact and a management framework that specifies how to govern AI-assisted operational decision support in high-risk digital infrastructure, ensuring that technical validity, auditability, privacy, and cost discipline are preserved.