Technical Summary: Reducing Token Usage of State-in-Context Agents using Minification

Problem Statement

Large Language Models (LLMs) are increasingly utilized as reasoning engines for software engineering (SE) agents, particularly for repository-level tasks like those found in the SWE-bench benchmark. In these workflows, agents must retrieve and reason over large codebases, leading to rapidly growing context sizes and high token consumption. Since token usage directly correlates with operational costs and can lead to performance degradation due to context limitations, effectively reducing context size while retaining relevant information is critical.

Recent work by Jiang et al. introduced "state-in-context" agents, which simplify architecture by feeding a compressed version of the entire environment state into a single prompt rather than incrementally reconstructing state via tool calls. However, analysis of these agents reveals that the vast majority of token consumption occurs during the repair step, where source code dominates the prompt. Specifically, over 90% of tokens in the repair prompt are consumed by source code, while natural language instructions and file ranking account for less than 10%. This paper addresses the challenge of reducing this token consumption without incurring prohibitive performance losses.

Methodology

Replication and Baseline

The authors first performed an independent replication and extension of the DirectSolve state-in-context agent. Since no public artifact was available, they implemented a clean-room version based on the original specifications. The workflow involves three steps:

1. Ranking: An LLM ranks repository files based on relevance to an issue description using only file paths.
2. Aggregation: Top-ranked files are aggregated into a context up to a predefined token budget (capped at 100,000 tokens in this study to ensure fair comparison across models).
3. Repair: An LLM generates a patch in a SEARCH/REPLACE format based on the aggregated context.

The authors evaluated this baseline on the SWE-bench Verified benchmark using GPT-5-mini, achieving a 50.0% pass@1 rate, consistent with the original report of 50.8% using Gemini-2.5-Pro.

Minification Strategy

To reduce token usage, the authors applied a series of code minification techniques to the source files before they were passed to the LLM for repair. These transformations are designed to remove or shorten non-essential lexical elements while preserving program semantics. The specific techniques include:

• Line/Operator Whitespace Removal: Eliminating blank lines and removing whitespace around operators (e.g., a + b becomes a+b).
• Comment/Docstring Removal: Stripping comments and docstrings, which are ignored at runtime.
• Dedentation: Reducing indentation levels (e.g., from 4 spaces to 2) while maintaining consistency within blocks. To prevent the LLM from generating invalid patches due to indentation mismatches, the system instructs the model to output SEARCH/REPLACE blocks with a fixed 4-space indentation style regardless of the input format.
• Identifier Shortening: Replacing variable, function, and class names with shorter aliases. This was tested in two modes:
  • Non-semantics preserving: Direct replacement without context.
  • Semantics preserving: Renaming identifiers and prepending a mapping table to the prompt so the LLM can reconstruct original names.
• Import Manipulation: Merging all imports into a single synthetic file or removing them entirely.

These transformations were integrated into the repair pipeline. Post-patch generation, identifier mappings were reversed, and a matching procedure tolerant of whitespace and comment differences was used to apply patches to the original repository.

Key Contributions

1. Independent Replication: A clean-room re-implementation of the DirectSolve state-in-context agent, demonstrating that a single-call, tool-free agent can achieve competitive performance (50% pass@1) on SWE-bench Verified when paired with a capable model (GPT-5-mini).
2. Token Allocation Analysis: An empirical analysis revealing that source code in the repair prompt is the dominant factor in token usage, overshadowing natural language instructions and ranking prompts.
3. Minification Framework: The proposal and integration of a set of lightweight, semantics-preserving code minification transformations tailored for Python source code within the state-in-context repair workflow.
4. Empirical Evaluation: A systematic evaluation on SWE-bench Verified showing that minification reduces average input token usage by approximately 42% with a 12 percentage-point drop in resolution rate (from 50.0% to 38.0%).
5. Ablation and Failure Analysis: Detailed studies isolating individual transformations and stacked combinations, quantifying their specific trade-offs. The authors identified that while most transformations yield modest performance losses, the "dedent" transformation causes a significant drop in resolution rate due to the generation of syntactically invalid patches (indentation errors).

Results

• Efficiency vs. Effectiveness: The primary finding is a clear trade-off. Minification reduces average input tokens from ~90,535 to ~52,776 (a 42% reduction). This results in a 12% absolute decrease in the resolution rate.
• Cost Implications: Since input tokens account for approximately 93% of the repair stage cost, the token reduction translates directly to significant cost savings.
• Transformation Specifics:
  • Removing docstrings and comments provided the largest token reductions with relatively small performance penalties (resolution rates remained above 40%).
  • The "dedent" transformation was the primary outlier, causing a resolution rate drop to 27-36% depending on the model and stacking configuration. Failure analysis indicated this was due to the LLM failing to restore the required 4-space indentation in the output, leading to IndentationError in the generated code.
  • Stacking multiple transformations (excluding dedent) consistently reduced costs while maintaining performance close to the baseline.
• Generalizability: The trends were observed to be stable across different repositories and issue difficulty levels, suggesting the approach generalizes well across different conditions.

Significance

The paper claims that lightweight source code transformations offer a promising path toward more cost-effective software engineering agents. By demonstrating that a substantial fraction of baseline performance can be retained while achieving nearly 50% token reduction, the work suggests that aggressive context compression is viable for repository-level tasks. The authors emphasize that their findings highlight an explicit trade-off between efficiency and effectiveness, providing a data-driven basis for selecting transformation strategies based on specific cost and performance requirements. The full implementation is made publicly available to support further research in this area.