To Diff or Not to Diff? Structure-Aware and Adaptive Output Formats for Efficient LLM-based Code Editing

1. Problem Statement

Large Language Models (LLMs) are increasingly used for code editing, but the prevailing paradigm relies on full-code generation. In this approach, even for minor edits, the model rewrites the entire source code. While accurate, this method suffers from severe efficiency bottlenecks:

• High Latency and Cost: Generating vast amounts of redundant tokens (unchanged code) increases inference time and API costs.
• Unintended Modifications: Re-generating the whole file increases the risk of introducing bugs in unrelated sections.

Conversely, existing attempts to use diff formats (outputting only the changes) have largely failed in practice. The paper identifies two main reasons for this failure:

1. Fragile Offsets: Traditional line-number-based diffs (e.g., Unified Diff) require precise numerical offsets, which probabilistic LLMs struggle to generate accurately.
2. Fragmented Hunks: Content-addressed diffs often break code into arbitrary line fragments, destroying syntactic integrity. This makes the generation process "unnatural" for LLMs pre-trained on well-formed code blocks, leading to low accuracy.

The core challenge is to design an edit format that reduces token counts (efficiency) without sacrificing the syntactic coherence required for high accuracy (naturalness).

2. Methodology

The authors propose a three-pronged approach: a systematic analysis of existing formats, the introduction of structure-aware formats, and an adaptive selection strategy.

A. Analysis of Conventional Formats

The paper systematically evaluates standard diff formats:

• Number-Indexed (UNIDIFF, MINUNIDIFF): Performance is poor because LLMs fail to generate precise line numbers. Even adding line numbers to the input does not fully solve the issue.
• Content-Addressed (CONTENTDIFF, MINCONTENTDIFF): While more robust than line numbers, these formats often result in "fragmented hunks" (disjointed line snippets). This fragmentation disrupts the LLM's ability to generate coherent code, resulting in accuracy significantly lower than full-code generation.

B. Structure-Aware Diff Formats

To address the "unnaturalness" of fragmented diffs, the authors introduce two new formats that align edits with the Abstract Syntax Tree (AST) of the code:

1. BLOCKDIFF: Represents changes as rewrites of fine-grained syntactic units (e.g., control structures like if blocks, loops, and functions).
2. FUNCDIFF: Represents changes as rewrites of entire functions, ignoring internal control structures to favor broader stability.

Key Mechanism:

• AST-Based Construction: Using tree-sitter, the code is parsed into a block tree.
• Mapping: Diff hunks are mapped to the smallest set of contiguous AST nodes that contain the changes.
• Anchor Expansion: To ensure unique patching, the "anchor" (the context used to locate the code) is progressively expanded to sibling or parent nodes until the content is unique within the file.
• Merging: Overlapping or adjacent hunks within the same parent node are merged into a single logical edit block, preventing fragmented generation.

C. ADAEDIT: Adaptive Edit Strategy

The authors recognize that no single format is optimal for all scenarios. If changes are pervasive across a file, the overhead of diff anchors may exceed the cost of full-code generation.

ADAEDIT is a training strategy where the LLM learns to dynamically choose the most token-efficient format for a given task:

• Training: For each training sample, the system calculates the token count of both the full target code and the diff representation. The ground truth is redefined as the shorter of the two.
• Inference: The model learns to predict the optimal format (Full Code vs. Diff) based on the input intent and code complexity, effectively internalizing a cost-benefit logic.

3. Key Contributions

1. Systematic Study of Diff Formats: The paper reveals that conventional diff formats fail due to fragile offsets and fragmented hunks, which mismatch the generative nature of LLMs.
2. Structure-Aware Formats (BLOCKDIFF & FUNCDIFF): Novel formats that represent edits as block-level rewrites of syntactically coherent units, restoring the "naturalness" of generation for LLMs.
3. ADAEDIT Strategy: A general adaptive framework that trains LLMs to switch between full-code and diff formats based on token efficiency, avoiding the "one-size-fits-all" limitation.
4. Comprehensive Evaluation: Extensive experiments across multiple datasets (EditEval, CanItEdit, HumanEvalFix, Aider) and base models (Qwen2.5-Coder, DeepSeek-Coder).

4. Experimental Results

The experiments demonstrate that the proposed approach achieves a superior trade-off between accuracy and efficiency:

• Accuracy:

  • Structure-aware formats (BLOCKDIFF/FUNCDIFF) consistently outperform conventional diff formats (like CONTENTDIFF) and often match or slightly exceed the accuracy of full-code generation, especially on larger models (e.g., Qwen2.5-Coder-14B).
  • Usability: Unlike line-number diffs which fail patching due to offset errors, structure-aware diffs maintain high patch-apply success rates and pass strict linter checks.

• Efficiency:

  • Latency & Cost: On long-code editing tasks (input > 300 tokens), ADAEDIT paired with structure-aware formats reduces both latency and cost by over 30% compared to full-code generation.
  • Adaptive Behavior: ADAEDIT correctly selects the most efficient format in >90% of cases. It switches to full-code for small, scattered changes and to diff formats for localized, large-scale edits.
  • Scaling: While full-code generation costs scale linearly with code length, ADAEDIT maintains optimal efficiency across all code scales.

• Generalizability: The approach works across different base models (7B, 14B, 6.7B) and programming languages (Python, JavaScript), showing robust cross-language performance.

5. Significance

This paper shifts the focus of LLM code editing from solely scaling model capabilities to optimizing the output format.

• Industrial Impact: By reducing latency and token costs by >30% without sacrificing accuracy, this work makes real-time, interactive coding assistants (like IDE plugins) significantly more viable and cost-effective.
• Theoretical Insight: It highlights that the "naturalness" of the generation target is crucial for LLM performance. Aligning output formats with the syntactic structure of code (AST) allows LLMs to leverage their pre-training more effectively than arbitrary text fragments.
• Future Direction: The authors suggest that future work could explore reinforcement learning to further optimize format selection and fluid granularity beyond fixed AST boundaries.

In summary, ADAEDIT with structure-aware diff formats provides a robust solution to the efficiency bottleneck in LLM-based code editing, proving that "diffing" is viable if done structurally and adaptively.