⚡ electrical engineering

Specification-Driven DevOps for Multi-Service Environments

This study demonstrates that while frontier Large Language Models can successfully generate functionally operational multi-service Docker environments from repository artifacts alone, they consistently fail to infer critical deployment-intent specifications such as security policies and build optimizations, thereby necessitating an explicit specification-driven approach to bridge the gap between functional correctness and production readiness.

Original authors: Oleg Grynets, Kyrylo Fursov, Vasyl Lyashkevych, Volodymyr Veres

Published 2026-07-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Oleg Grynets, Kyrylo Fursov, Vasyl Lyashkevych, Volodymyr Veres

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a master chef who has just been handed a pile of raw ingredients: fresh vegetables, spices, a recipe card, and a list of tools. Your job is to build a fully functional, high-end restaurant kitchen that can serve a perfect meal to a hungry crowd. In the world of software, this is what "DevOps" is all about: taking the code (the ingredients) and turning it into a running application (the meal) that lives in a safe, organized, and efficient environment. For a long time, humans did this by hand, carefully writing instructions on how to set up the kitchen. But now, we have a new helper: Artificial Intelligence, specifically Large Language Models (LLMs). Think of these AIs as super-smart, incredibly fast sous-chefs who have read millions of cookbooks. They can look at your pile of ingredients and instantly guess how to build a kitchen. The big question everyone is asking is: Can this AI sous-chef just look at the ingredients and build a kitchen that is not only able to cook the meal, but also safe, secure, and ready for a real restaurant, or will it just build a kitchen that works for a quick snack but falls apart when you try to run a real business?

This is exactly what a team of researchers from EPAM Systems set out to find out. They wanted to see if a top-tier AI could look at a software project's code (the "ingredients") and automatically build the entire "kitchen" (the deployment environment) without any human help or pre-written instructions. They tested this on three different software projects, each with a mix of different programming languages and tools, like a kitchen with a stove, a fridge, and a pantry all working together. They asked the AI to write the blueprints for Docker (a tool that packages software into neat, portable boxes) and Docker Compose (the instruction manual that tells all the boxes how to talk to each other).

The results were a fascinating mix of "wow" and "whoops." The AI was surprisingly good at the basics. It correctly figured out which parts of the software needed to talk to each other, what ports (like doors) they should use, and even found hidden clues in the code to set up a secret password system. In fact, the AI built three kitchens that could all successfully cook a meal and serve it to a customer. However, when the researchers looked closer, they realized the kitchens weren't quite ready for a real restaurant. The AI built them with all the doors wide open to the street (a security risk), it didn't build separate rooms for different tasks (network segmentation), and it skipped steps that would make the kitchen run faster and more efficiently. The AI knew how to cook, but it didn't quite understand the rules of running a professional kitchen.

The researchers concluded that while the AI is a fantastic tool for figuring out the "what" and "how" of the code, it still needs a human to give it a short, clear list of rules about the "who," "where," and "when" of the business. They call this "Specification-Driven DevOps." It's like giving the AI sous-chef a note that says, "Keep the back door locked," "Use a two-step cooking process to save energy," and "Only the front door should be open to customers." Without this extra note, the AI will build a kitchen that works, but it might not be safe or efficient enough for the real world. The study suggests that the future of software isn't just letting the AI do everything; it's about a partnership where the AI handles the heavy lifting of construction, and humans provide the critical safety and strategy guidelines.

Technical Summary: Specification-Driven DevOps for Multi-Service Environments

Problem Statement

The increasing adoption of Large Language Models (LLMs) for generating executable software environments from repository artifacts has revealed a critical gap: functional executability does not guarantee conformity with deployment intent. While LLMs can often synthesize Dockerfiles and Docker Compose configurations that successfully build and run, they frequently fail to reconstruct implicit architectural, security, workflow, and production constraints inherent to the developer's intent.

Existing research focuses heavily on single-container environments or assumes that infrastructure intent is explicitly provided (e.g., via natural language prompts). However, in multi-service environments, critical decisions regarding network segmentation, port exposure policies, build strategies (e.g., multi-stage), secret management mechanisms, and production vs. development modes are often not explicitly encoded in source code. This study investigates whether a frontier LLM can reliably infer these "intent-only" constraints solely from repository artifacts and proposes a Specification-Driven DevOps approach to bridge the gap between observable runtime facts and explicit deployment policy.

Methodology

The study employs a repository-driven generation experiment involving three heterogeneous multi-service applications:

example-voting-app: A heterogeneous stack (Python, Node.js, .NET) with Redis and PostgreSQL.
react-rust-postgres: A React/Rust/PostgreSQL stack with hidden proxy configurations and missing credentials in the input.
react-java-mysql: A React/Java/MariaDB stack featuring a file-based secret mechanism (db/password.txt).

Experimental Constraints:

Input: Only source code, dependency manifests, configuration files, and auxiliary artifacts. Existing Dockerfile and docker-compose.yml files were explicitly excluded.
Model: GPT Codex 5.3 via GitHub Copilot.
Platform: Apple M1 Pro (ARM64).
Validation:
- Functional Oracle: Deterministic end-to-end HTTP requests verifying the complete multi-service pipeline (e.g., vote submission propagation through worker to database).
- Structural Comparison: Manual analysis comparing generated artifacts against developer-authored ground truths to assess fidelity in security, workflow, and production readiness.

Formal Framework:
The study formalizes the transformation $T_M: R \to E$ , where $R$ is the repository and $E$ is the generated environment. It distinguishes between:

Functional Correctness ( $E_G \equiv_F E_D$ ): The environment runs and passes the oracle.
Deployment-Intent Fidelity: The degree to which the structure matches the developer's architectural and policy decisions.

Key Contributions

Empirical Boundary Definition: The study establishes a taxonomy of DevOps knowledge, categorizing it into:
- Observable/Derivable: Service topology, ports, dependencies, and background workers (successfully inferred).
- Underdetermined: Specific credential values or compatible image versions (inferred but potentially inconsistent with ground truth).
- Intent-Only: Network segmentation, port exposure policies, build strategies, and production serving modes (consistently omitted or defaulted).
Specification-Driven DevOps Model: The paper proposes a hybrid generation model $E = G(R, S_{min})$ , where $S_{min}$ is a minimal explicit deployment specification. This specification contains only the irreducible intent that cannot be reliably inferred from the repository, avoiding the duplication of information already present in source code.
Evaluation Framework: A distinction is drawn between "executable correctness" and "structural fidelity," introducing a multidimensional evaluation model that assesses security, workflow, portability, and production readiness separately from functional execution.
Minimal Specification Template: A conceptual YAML template is derived to address observed omissions, covering deployment context (dev/prod), network segmentation, build strategies (multi-stage), secret mechanisms, and platform constraints.

Results

Functional Success:

All three generated environments achieved functional operational status (100% success rate) after resolving one base-image version conflict (Rust 1.85 to 1.88).
The LLM successfully reconstructed complex service topologies, including background workers, hidden proxy configurations, and file-based secret mechanisms.

Structural and Intent Deficiencies:
Despite functional success, the generated environments exhibited systematic omissions in deployment intent:

Network Segmentation: All environments were generated with a flat network (0/3), failing to isolate backend services from public access.
Port Exposure: Backend ports were unnecessarily exposed to the host in 2/2 applicable cases, violating restrictive security policies.
Build Strategy: No multi-stage builds or optimized dependency-layer caching were generated (0/3).
Production Serving: Frontend services utilized development servers instead of production-ready Nginx configurations (0/2 applicable).
Workflow: Live-reload volume mounts for development were absent (0/3).

Traceability:
The study demonstrated that while runtime dependencies are observable, negative constraints (e.g., "do not expose this port") and policy decisions (e.g., "use Docker secrets") are not reliably inferable from source code alone.

Significance and Claims

The paper modestly claims to have identified a practical boundary between what can be inferred from repository artifacts and what must be explicitly specified.

Not a Universal Solution: The authors explicitly state that the results apply to the three evaluated repositories and do not claim universal reliability for arbitrary microservice ecosystems. The sample size is small, and the repositories were selected for clarity, potentially introducing a familiarity bias.
Analytical Derivation: The proposed minimal specification and the hybrid generation model ( $G(R, S_{min})$ ) are analytically derived from the observed gaps. The study does not experimentally validate the effectiveness of this specification by re-running the generation with the spec; it only proposes the framework.
Core Insight: The primary contribution is the formalization of the "signal gap." The study concludes that repository evidence is sufficient for reconstructing runtime requirements but insufficient for reconstructing deployment intent. Therefore, a Specification-Driven approach is necessary to ensure that generated environments are not just executable, but also secure, maintainable, and aligned with production policies.

The paper positions itself as an exploratory feasibility study that motivates the need for explicit, machine-processable specifications to complement LLM-driven code generation in DevOps contexts.