CODESTRUCT: Code Agents over Structured Action Spaces

1. Problem Statement

Current Large Language Model (LLM) based code agents suffer from a fundamental abstraction mismatch. They treat software repositories as unstructured flat text rather than structured artifacts.

• Brittleness: Agents rely on string matching and line numbers to read and edit code. This approach is highly fragile; minor formatting changes (whitespace, indentation) or ambiguous patterns cause "no occurrence" or "multiple occurrence" errors.
• Inefficiency: To locate a specific function, agents often read entire files (introducing irrelevant context) or large line ranges. To modify code, they frequently regenerate large blocks of unchanged text verbatim, leading to high token consumption and redundant reasoning.
• Failure Modes: These issues force agents into costly trial-and-error cycles, often resulting in empty patches (failure to produce any code) or syntax errors that corrupt the repository.

2. Methodology: CODESTRUCT

The authors propose CODESTRUCT, a framework that reframes the codebase as a structured action space based on the Abstract Syntax Tree (AST). Instead of operating on text spans, agents interact with named semantic entities (e.g., file.py::ClassName::method).

Core Components

The framework introduces two structure-aware primitives that operate directly on AST nodes:

1. readCode (Structure-Aware Retrieval):

   • Function: Retrieves complete syntactic units (functions, classes, methods) based on a selector (e.g., User.load) rather than line numbers.
   • Modes:
     • Directory Browsing: Lists files in a directory.
     • File Summarization: Returns a structural summary (signatures of top-level entities) for large files to avoid loading full context.
     • Entity Retrieval: Returns the full implementation of a specific matched entity.
   • Benefit: Eliminates irrelevant context and ensures the agent receives complete, valid syntactic units.

2. editCode (Structure-Aware Modification):

   • Function: Applies transformations (insert, replace, remove) directly to AST nodes.
   • Mechanism: The agent specifies the target entity and the new content. The tool handles indentation, validates the resulting AST for syntax correctness, and writes the file.
   • Benefit: Prevents "string matching" failures. If an edit would result in a syntax error, the tool rejects it immediately, ensuring the repository remains in a valid state. It avoids regenerating unchanged code.

Integration

CODESTRUCT is implemented via the Model Context Protocol (MCP), allowing it to integrate with existing agent frameworks (like SWE-Agent or OpenHands) without modifying the agent's planning logic. The agent simply swaps text-based tools for these structured primitives.

3. Key Contributions

1. Structured Action Space Interface: A novel interface bridging AST-based program representations and LLM agents, exposing named semantic entities as the primary units of interaction.
2. Two Structure-Aware Primitives: The design of readCode and editCode which feature human-readable selectors, automatic scope resolution, and syntax-validated edits to support robust agent recovery.
3. Empirical Validation: Extensive evaluation across two benchmarks (SWE-Bench Verified, CodeAssistBench) and six different LLMs, demonstrating that structure-aware interfaces improve both effectiveness and efficiency, particularly when text-interface brittleness is the dominant failure mode.

4. Experimental Results

The framework was evaluated on SWE-Bench Verified (500 real-world Python bug fixes) and CodeAssistBench (149 multi-turn interactive tasks) using models ranging from GPT-5 to Qwen3-8B.

SWE-Bench Verified Results

• Accuracy: CODESTRUCT improved Pass@1 accuracy by 1.2–5.0% for frontier models.
  • Notable Gain: GPT-5-nano saw a massive 20.8% improvement (from 19.6% to 40.4%). This was driven by a reduction in "empty-patch" failures (dropping from 46.6% to 7.2%), as structured navigation allowed the smaller model to explore deeper without getting lost in text.
• Efficiency:
  • Token Reduction: Most models saw a 12–38% reduction in token consumption due to targeted retrieval and reduced regeneration of unchanged code.
  • Cost Reduction: Inference costs dropped by up to 33% (e.g., GPT-5-mini saw a 32.6% cost reduction).
  • Exception: GPT-5-nano used more tokens/compute, but this was a deliberate trade-off where structured actions enabled sustained exploration that led to successful solutions where text-based approaches failed early.

CodeAssistBench Results

• Consistent Gains: Accuracy improved by 0.8–4.4% across all models.
• Efficiency: Significant token and cost reductions (up to 33%) were observed, confirming that structured interactions generalize well to interactive, multi-turn workflows.

Error Analysis

• Reliability: For high-capacity models, structured operations reduced tool-level errors (failed edits) by 76–88% compared to text-based string matching.
• Ablation: Removing readCode caused the largest accuracy drops, proving that structured navigation is critical for preventing inefficient trial-and-error. Removing editCode increased costs significantly as agents fell back to brittle string edits.

5. Significance and Conclusion

• Paradigm Shift: CODESTRUCT moves code agents away from treating code as "text" to treating it as "structured data." This aligns agent actions with how human developers reason about code (by name and structure, not line numbers).
• Robustness: By decoupling semantic intent from textual realization, the framework eliminates a major class of failures (formatting drift, regex mismatches) that plague current agents.
• Scalability: The approach improves performance across models of varying sizes, offering the most significant gains for smaller models that struggle with the noise and ambiguity of unstructured text.
• Future Impact: This work suggests that future code agents should prioritize structure-aware interfaces to achieve reliable, cost-effective, and scalable software engineering automation.

Limitations: The current evaluation focuses heavily on Python (SWE-Bench) and requires syntactically valid source files for AST parsing. Future work aims to extend this to statically-typed languages with complex generics and tasks like code review.