Can AI Agents Generate Microservices? How Far are We?

This paper evaluates the capability of AI agents to generate functional microservices, finding that while they can produce maintainable code with high API contract adherence—particularly in clean-state scenarios—current limitations in consistency and the need for human oversight prevent fully autonomous generation.

The Gist

Imagine you are the manager of a massive, bustling restaurant kitchen. This kitchen is built on a Microservices architecture. Instead of one giant chef trying to cook the entire meal, you have dozens of specialized stations: one for chopping vegetables, one for grilling steaks, one for making sauces, and one for plating. Each station is independent, but they all need to talk to each other perfectly to get the meal to the customer.

Now, imagine you hire a team of AI Chefs (AI Agents) to help you. You want to know: Can these AI chefs actually cook a new station from scratch, or add a new station to an existing kitchen without ruining the whole meal?

This paper is a "taste test" to see how good these AI Chefs are at the job. Here is the breakdown in simple terms:

1. The Two Ways to Ask for Help

The researchers tested the AI in two different scenarios, like giving them two different types of instructions:

• Scenario A: "The Renovation" (Incremental Generation)

  • The Setup: The kitchen already exists. The AI is told, "Go to the 'Sauce Station,' delete the current chef, and hire a new one. But keep the rest of the kitchen exactly as it is."
  • The Challenge: The new AI chef must fit perfectly into the existing workflow. If they use a different type of knife or a different recipe format, the whole kitchen breaks.
  • The Result: Surprisingly, the AI did better when given minimal instructions (just "Make a sauce station") rather than a giant, detailed manual. When the researchers gave them too much text to read, the AI got confused and started ignoring the existing kitchen rules. When left to "explore" the kitchen on its own, it figured out the rhythm better.
  • Success Rate: About 50% to 76% of the time, the new station worked perfectly with the old one.

• Scenario B: "The Greenfield Project" (Clean State Generation)

  • The Setup: You are building a brand new restaurant in an empty field. You give the AI a list of requirements ("We need a sauce station that handles 500 orders an hour") but no existing kitchen to look at.
  • The Challenge: The AI has to invent the whole structure from scratch.
  • The Result: This went much better. The AI created stations that worked 81% to 98% of the time. Why? Because there was no "old kitchen" to clash with. The AI could build whatever it wanted, as long as it followed the rules.
  • The Catch: The code was sometimes "weird" (different file names or structures), but since it was a new building, it didn't matter as long as the doors opened and the lights turned on.

2. The Three AI Chefs (The Agents)

The researchers tested three different AI "chefs":

• Codex (The Veteran): Very smart, but sometimes takes a long time to think (up to 1.7 hours for one task!) and can be expensive.
• Claude Code (The Precise One): Fast, very accurate, but the most expensive per meal. It writes very concise code (short recipes).
• Code Qwen (The Budget Option): The fastest and cheapest. It's great, but sometimes it gets stuck in a loop if the instructions aren't clear.

3. The "Secret Sauce" (What They Found)

• Less is More (Sometimes): In the "Renovation" scenario, giving the AI a huge, detailed summary of the existing code actually made it worse. It was like telling a chef, "Here is a 50-page manual on how we used to do things," and the chef got so focused on the manual they forgot to look at the actual kitchen. A simple prompt ("Just build the station") let the AI explore and adapt better.
• Simpler is Better: The code the AI wrote was actually simpler and less complex than code written by humans. It wasn't "dumber," just more efficient. It didn't over-complicate things.
• The "Memorization" Problem: The AI performed much better on famous, open-source projects (like a popular recipe book everyone knows) than on private, student projects. This suggests the AI might be "memorizing" answers it saw during its training rather than truly "thinking" through new problems.
• Cost vs. Speed:
  • Fastest: Code Qwen (7.6 mins) and Claude (7.8 mins).
  • Slowest: Codex (16.6 mins, sometimes taking over an hour!).
  • Cheapest: Code Qwen ($3 per service).
  • Most Expensive: Claude ($13 per service).

4. The Verdict: Are We There Yet?

Can AI generate microservices? Yes.
Can they do it fully on their own without a human watching? Not yet.

Think of the AI as a brilliant but inexperienced apprentice.

• They can cook a great dish if you give them a clean kitchen and a simple list of ingredients.
• They can also cook a decent dish in an existing kitchen if you let them look around first.
• However, they sometimes miss the tiny details (like a specific type of connector or a hidden rule) that a human chef would catch. If you let them run the whole kitchen alone, they might accidentally delete the database or break the connection between the grill and the oven.

The Big Takeaway

AI is ready to be a super-powered assistant for software architects. It can do the heavy lifting, write the code, and even make it simpler than humans do. But we still need a Human Architect in the loop to double-check the work, ensure the "stations" talk to each other correctly, and make sure the AI didn't get stuck in a loop or memorize the wrong recipe.

We are not at the point of "Press Button, Get Perfect Software," but we are definitely at the point of "Press Button, Get 80% of the work done, then fix the last 20%."

Technical Summary

Here is a detailed technical summary of the paper "Can AI Agents Generate Microservices? How Far are We?" by Bassam Adnan et al.

1. Problem Statement

While Large Language Models (LLMs) have advanced code generation, their application to generating microservices—complex architectural units requiring strict API contracts, inter-service dependencies, and integration with existing systems—remains understudied.

• The Gap: Existing research focuses on single-pass code completion or small snippets. There is a lack of empirical evidence regarding whether AI Agents (which possess memory, tool usage, and iterative refinement capabilities) can autonomously generate functional microservices.
• The Challenge: Microservice generation requires both local correctness (internal logic of the service) and global correctness (adherence to API contracts and interaction patterns across the system). Small errors can break cross-service communication, making this a rigorous benchmark for AI capabilities.

2. Methodology

The study employed a controlled empirical experiment to evaluate the functional correctness, code quality, and efficiency of AI agents in generating microservices.

Experimental Design

• Subjects: 3 AI Agents were selected from the LMArena Coding Leaderboard:
  1. Claude Code (Proprietary, Subscription/API)
  2. Codex (Proprietary, GPT-5 based, API)
  3. Code Qwen (Open Source, API/Local)
• Datasets: 4 Projects were used (2 Open Source: PiggyMetrics, Train-Ticket; 2 Private/Student: TeamSync, Project Management).
• Scope: 3 microservices were selected per project for removal and regeneration, resulting in 144 total implementations (4 projects × 3 services × 3 agents × 2 scenarios × 2 prompt strategies).

Generation Scenarios

1. Incremental Generation (RQ1.1): The target microservice code is removed, but the rest of the system (existing architecture, API contracts, documentation) remains. The agent must integrate the new service into the existing context.
2. Clean State Generation (RQ1.2): The target microservice and all references to it (configurations, API calls from other services) are removed. The agent must generate the service and infer the architecture solely from requirements specifications.

Prompt Strategies

• P1 (Minimal Context): Provided only the microservice name and path to requirements. The agent must explore the codebase to infer patterns.
• P2 (Detailed Context): Provided the name, requirements, and a generated implementation summary (created by an expert agent) detailing functional scope, API responsibilities, and integration points.

Evaluation Metrics

• Functional Correctness:
  • Incremental: Measured via Unit Test Pass Rates against the original baseline.
  • Clean State: Measured via Integration Test Pass Rates (tests from dependent services) to account for structural variations (e.g., different package names) that might fail unit tests but work functionally.
• Code Quality: Measured using SonarQube metrics: Source Lines of Code (SLOC), Cyclomatic Complexity, and Cognitive Complexity.
• Efficiency: Generation time, token consumption (input/output), and monetary cost.

3. Key Contributions

1. First Empirical Study: The first systematic evaluation of AI agents generating full microservices across varying contextual scenarios (incremental vs. clean state).
2. Contextual Analysis: Demonstrated that the "best" prompt strategy depends heavily on the generation scenario and the specific agent used.
3. Efficiency Benchmarking: Provided a comparative analysis of time, cost, and token consumption across proprietary and open-source agents for architectural tasks.
4. Open Dataset: Released 144 generated microservice implementations and prompts for reproducibility.

4. Key Results

Functional Correctness (RQ1)

• Incremental Generation:
  • Counter-intuitive Finding: Minimal prompts (P1) outperformed detailed summaries (P2).
  • Reasoning: Detailed summaries caused agents to "anchor" on high-level patterns mentioned in the summary, causing them to miss low-level implementation details (e.g., specific database converters) that they would have discovered by exploring the codebase in P1.
  • Performance: Codex (75.9% pass rate with P1) and Claude Code (73.7%) outperformed Code Qwen (50.5%).
• Clean State Generation:
  • High Success: Agents achieved significantly higher pass rates (81–98%) compared to incremental generation.
  • Reasoning: Integration tests are more flexible regarding structural differences (package names, class names) as long as the API contract is met.
  • Prompt Impact: In this scenario, P2 (Detailed Context) helped Code Qwen recover from failure (jumping from 33% to 97% pass rate), whereas Codex performed best with P1.
• Data Contamination: Agents performed significantly better on popular open-source projects (likely memorized from training data) than on private student projects, highlighting a bias in current benchmarks.

Code Quality

• Generated code consistently exhibited lower complexity (Cyclomatic and Cognitive) than human-written baselines (15–30% reduction).
• However, lower complexity sometimes indicated a lack of defensive programming (e.g., missing error handling), though the generated code was generally maintainable.

Efficiency (RQ2)

• Time: Claude Code and Code Qwen were fast (7.6–7.8 mins). Codex was significantly slower (16.6 mins) with extreme outliers reaching 104 minutes.
• Cost: Code Qwen was the most cost-effective ($2.98/service). Claude Code was the most expensive ($13.28/service), despite generating the most concise output (lowest token count).
• Token Usage: Verbose output (Codex) did not correlate with higher functional correctness. Claude Code achieved high correctness with minimal tokens.

5. Significance and Implications

For Practitioners

• No "One-Size-Fits-All": Prompt engineering must be empirical. In incremental scenarios, let agents explore the codebase (minimal prompts). In greenfield/clean state scenarios, explicit architectural summaries may be necessary for open-source models.
• Human Oversight is Critical: While agents can generate functional code, they lack architectural reasoning for system-wide trade-offs. Human review is required to ensure API contract compliance and prevent "anchoring" errors.
• Cost vs. Reliability: The cost difference between agents ($3 vs $13) is negligible compared to the cost of developer time debugging failures. Reliability and generation time should be prioritized over token cost.

For Researchers

• Need for New Benchmarks: Existing benchmarks (like SWE-bench) focus on repository-level tasks. New benchmarks must include explicit API contracts, integration tests, and diverse architectural styles to properly evaluate architectural generation.
• Data Contamination: The performance gap between public and private datasets suggests current LLMs rely heavily on memorization. Future research must develop methods to detect and mitigate training data contamination.
• Human-Agent Collaboration: Research is needed to define optimal intervention points where humans should step in to correct architectural reasoning or validate contracts.

Conclusion

AI agents are capable of generating functional, maintainable microservices, particularly in "clean state" scenarios where they can define their own structure. However, in "incremental" scenarios, they struggle with integration details unless guided by minimal prompts that encourage codebase exploration. Fully autonomous microservice generation is not yet achievable due to inconsistent correctness and the need for human oversight to validate architectural integrity.