Beyond Local Code Optimization: Multi-Agent Reasoning for Software System Optimization

This paper introduces a multi-agent framework that moves beyond local code optimization to enable whole-system performance reasoning for microservices, achieving significant throughput and response time improvements by coordinating specialized agents to analyze architectural dependencies and construct cross-component optimization strategies.

The Gist

Imagine you are the manager of a massive, bustling restaurant. Your goal is to serve more customers faster without making the food taste bad or breaking the kitchen.

The Old Way: The "Single Chef" Approach

For a long time, AI tools trying to speed up software were like single chefs looking at one specific recipe card at a time.

• If a chef saw a step that took too long (like chopping onions), they would suggest a faster knife.
• The Problem: This chef doesn't know that the next station is waiting for those onions, or that the oven is already full, or that the delivery driver is stuck in traffic. They only see the one task in front of them. They might speed up chopping, but if the oven is the bottleneck, the whole restaurant is still slow.

This is what most current AI optimization tools do: they fix small pieces of code (like a single function) but miss the big picture of how different parts of the software talk to each other.

The New Way: The "Conductor's Orchestra"

This paper introduces a new system called Multi-Agent Reasoning. Instead of one chef, imagine a team of specialized experts working together, led by a conductor, to optimize the entire restaurant at once.

Here is how their team works:

1. The Map Makers (Summarization Agents)

Before fixing anything, this team creates a giant, detailed map of the restaurant.

• The Architect: Draws the layout of the kitchen, the dining room, and the delivery routes (System Structure).
• The Flow Tracker: Watches how orders move from the host to the kitchen to the table, noting where people bump into each other (Control Flow).
• The Context Giver: Checks the weather, the number of staff on shift, and the type of stove being used (Environment).
• Why it matters: They don't just look at a recipe; they understand how the whole building operates.

2. The Detectives (Analysis Agents)

Using the maps, these agents act like detectives looking for the real traffic jams.

• They might say, "Hey, the problem isn't that the chefs are slow; it's that the delivery driver is waiting 5 minutes for a table to clear up because the host isn't calling them fast enough."
• They find bottlenecks that happen between different parts of the system, not just inside one part.

3. The Fixers (Optimization Agents)

Once the problem is found, the fixers propose changes.

• Instead of just telling the chef to chop faster, they might say, "Let's change the layout so the delivery driver can walk straight to the table," or "Let's have the host call the driver before the table is fully cleared."
• Crucially, they promise not to break anything. They won't change the menu (the public interface) or serve bad food (break the code). They only tweak the internal process.

4. The Inspectors (Verification Agents)

Before the changes go live, the inspectors run a test.

• They simulate a rush hour to see if the new layout actually works.
• They check: "Did we serve more people? Did the food still taste good? Did anyone get hurt?"
• If it works, the changes are applied. If not, they go back to the drawing board.

The Results: A Faster Restaurant

The authors tested this "team of experts" on a real software system called TeaStore (a fake online shop).

• Before: The system could handle about 1,200 requests per second.
• After: With the AI team's help, it handled 1,635 requests per second.
• Speed: The average time to get a response dropped by nearly 28%.

Why This Matters

Think of it like upgrading from a bicycle to a high-speed train.

• The old AI tools were like fixing a loose bolt on a bicycle wheel. It helps a little, but you're still limited by the bike.
• This new system looks at the tracks, the engine, the schedule, and the passengers to build a train. It understands that the speed of the whole system depends on how all the parts work together, not just how fast one gear turns.

In short: This paper shows that by using a team of AI agents that talk to each other and look at the "big picture" of software architecture, we can make complex computer systems significantly faster and more efficient than ever before.

Technical Summary

1. Problem Statement

Current Large Language Model (LLM) approaches to software performance optimization are predominantly local and syntax-driven. They focus on optimizing individual functions, classes, or files. However, modern software systems (particularly microservices) exhibit performance bottlenecks that emerge from cross-component interactions, architectural dependencies, and system-wide resource contention (e.g., database locking, connection pool exhaustion, or inter-service latency).

The core problem is that isolated code-level reasoning fails to capture:

• Whole-system behavior and control/data flow across service boundaries.
• Architectural constraints and design-level decisions that impact performance.
• Context-dependent issues where a local optimization might degrade global throughput or increase latency.

The authors ask: Can automated techniques identify and reason about whole-system performance issues arising from architectural structure and cross-component interactions, beyond isolated code scopes, while preserving correctness?

2. Methodology

The authors propose a Multi-Agent Framework that integrates static program analysis with LLM-based reasoning to perform whole-system optimization. The system is decomposed into four coordinated agent roles operating within a stateful pipeline (illustrated in Figure 1 of the paper):

A. System Architecture & Agents

1. Summarization Agents (The "Context" Layer):

   • Component Summary Agent: Extracts structural architecture (service boundaries, hierarchical composition, exported interfaces, and static dependencies like call graphs and resource coupling).
   • Behavior Summary Agent: Captures execution-related properties, including inter-procedural call graphs, control-flow complexity, interaction sites (DB access, external calls), and synchronization constructs (locks).
   • Environment Summary Agent: Extracts runtime context from build configurations (programming language, compiler settings, CPU cores, dependencies) to ensure optimizations respect infrastructure constraints.
   • Input Source: These agents utilize CodeQL to generate semantic abstractions (ASTs, CFGs, call graphs) which are serialized into a language-agnostic representation.

2. Analysis Agent (The "Diagnosis" Layer):

   • Consumes the structured summaries to identify performance bottlenecks.
   • Uses a multi-stage reasoning process: extracting performance signals, inspecting critical code regions, and ranking opportunities based on estimated impact and confidence.
   • Outputs a structured report with evidence-backed priorities and source locations.

3. Optimization Agent (The "Action" Layer):

   • Translates high-level optimization intents into concrete code modifications.
   • Constraint: All changes must be non-breaking (no public API or interface changes) to preserve system correctness.
   • Generates unified diffs with technical justifications explaining the "What, Why, and How" of the change.

4. Performance Evaluation Agent (The "Verification" Layer):

   • Validates functional correctness using automated test suites (e.g., JUnit).
   • Performs dynamic profiling under representative workloads (using Apache JMeter) to measure latency, throughput, and resource utilization against a baseline.
   • Feeds results back into the loop to guide iterative refinement.

B. Implementation Details

• Orchestration: Built using LangGraph to manage the directed graph of agents, shared state, and control flow.
• Static Analysis Engine: CodeQL is used to query abstract syntax trees and control flow graphs.
• LLM: The prototype uses gpt-5.2 (simulated as SOTA for Jan 2026) with a temperature of 0.7.
• Target System: TeaStore, a Java-based microservices application modeling an online retail workflow with six interacting services.

3. Key Contributions

1. Novel Multi-Agent Framework: A system that elevates performance reasoning from local code edits to the whole software stack by integrating static analysis (control/data flow) with architectural abstractions.
2. Cross-Component Reasoning: The ability to identify bottlenecks that span multiple services and layers (e.g., database, middleware, shared infrastructure) rather than just single functions.
3. Proof-of-Concept Implementation: A working prototype demonstrating the feasibility of this approach on a complex microservice architecture.
4. Comprehensive Evaluation Strategy: A methodology that combines static analysis signals, automated testing for correctness, and dynamic benchmarking for performance.

4. Results

The framework was evaluated on the TeaStore microservice application. The system successfully identified and applied three specific optimizations:

• O1 (HTTP Client Reuse): Replaced per-request client creation with a singleton pattern to avoid redundant connection pool initialization.
• O2 (Lock Contention): Replaced synchronized methods with volatile flags on request-critical paths to reduce thread blocking.
• O3 (Object Allocation): Replaced per-request ObjectMapper instantiation with a shared static instance to reduce GC pressure.

Performance Metrics:

• Throughput: Increased by 36.58% (from 1,197.79 to 1,635.89 req/sec).
• Average Response Time: Reduced by 27.81% (from 12.84ms to 9.27ms).
• Latency: Significant improvements across P50 (−30.77%), P90 (−21.74%), and P99 (−11.54%).
• Cost/Efficiency: The full optimization run consumed ~1.6M tokens, cost $1.28, and took approximately 24 minutes.
• Correctness: All optimizations passed the existing JUnit test suites, ensuring no functional regression.

5. Significance

This paper represents a paradigm shift in AI-driven software engineering. It moves beyond the "patch-and-pray" approach of local code generation to a systematic, architectural approach to performance engineering.

• Scalability: By using static analysis to abstract system complexity, the framework can reason about systems too large for an LLM to process in a single context window.
• Holistic Optimization: It addresses the "unknown unknowns" of distributed systems where the bottleneck is often the interaction between components, not the code within a single component.
• Future Direction: The authors position this as a foundation for future work in automated performance engineering, suggesting that multi-agent systems can eventually replace or augment human performance engineers in identifying and resolving complex, cross-cutting system inefficiencies.

The work is submitted to JAWS for early feedback, with plans to expand the evaluation to diverse benchmarks (e.g., DeathStarBench) and compare against state-of-the-art baselines like SysLLMatic and other agent-based systems.