Cascaded Code Editing: Large-Small Model Collaboration for Effective and Efficient Code Editing

1. Problem Statement

The paper addresses the fundamental trade-off between effectiveness and efficiency in Large Language Model (LLM)-based code editing.

• The Efficiency Bottleneck: Current state-of-the-art large models (e.g., DeepSeek-R1, Claude-4) excel at understanding complex codebases and natural language instructions but suffer from high computational costs and latency. This is largely because they are forced to regenerate entire files, even when only 15% of the code changes, leading to massive token redundancy.
• The Effectiveness Gap: Smaller, more efficient models (e.g., <7B parameters) are fast and cheap but lack the reasoning capacity to handle long-context codebases, cross-file dependencies, and nuanced editing requirements, resulting in poor editing quality.
• The Limitation of Current Small Models: While a "cascaded" approach (where a large model plans edits and a small model applies them) seems logical, current small models fail at the Edit Sketch Application phase. They struggle to correctly merge concise edit sketches into long, complex original codebases without introducing errors, omissions, or hallucinations.

2. Methodology

The authors propose a Cascaded Code Editing Framework that decomposes the task into two distinct stages, leveraging the strengths of different model sizes, coupled with specialized data and training strategies to overcome the limitations of small models.

A. The Cascaded Framework

1. Stage 1: Edit Sketch Generation (Large Model):
   • A powerful, computationally expensive LLM (e.g., DeepSeek-R1) analyzes the original code and natural language instructions.
   • Instead of regenerating the full file, it produces a concise Edit Sketch: a targeted snippet containing only the necessary modifications, omitting unchanged code.
   • Benefit: Drastically reduces the token count generated by the expensive large model.
2. Stage 2: Edit Sketch Application (Small Model):
   • A smaller, efficient model takes the original code and the edit sketch.
   • Its task is to locate the specific segments in the original code and merge the sketch to produce the final edited file.
   • Benefit: Small models are fast and cheap for this mechanical integration task.

B. Addressing Small Model Limitations

To enable small models to perform Stage 2 effectively, the authors introduce three key innovations:

1. Large-Scale Sketch Application Dataset:

   • The first dataset of its kind, containing 118,280 training instances (73,762 from real-world Git commits and 44,518 synthesized) and 800M tokens.
   • Includes a human-evaluated benchmark with 1,981 high-fidelity test instances.
   • Data covers diverse languages, long contexts (>1k lines), and cross-file dependencies.

2. Curriculum-based Long-Context SFT (CLC SFT):

   • A two-stage training strategy to handle long contexts without overwhelming memory.
   • Stage 1: Fine-tuning on short-context data (<4k tokens) to establish baseline editing skills.
   • Stage 2: Training on long-context data (≥4k tokens) and multi-file augmented data (combining multiple short pairs to simulate complex scenarios) using a larger context window (16k tokens).

3. Generalized Curriculum Long-Context SFT (G-CLC SFT):

   • An extension of CLC SFT that mixes the task-specific sketch application data with general-domain coding tasks (from OpenCoder datasets) at a 1:1 ratio.
   • Goal: Prevents "catastrophic forgetting" of general coding abilities and improves out-of-domain generalization.

3. Key Contributions

• Framework Design: A novel cascaded architecture that separates high-level planning (Large Model) from low-level implementation (Small Model), optimizing the effectiveness-efficiency trade-off.
• Resource Creation: The creation of the first large-scale, task-specific dataset and benchmark for Edit Sketch Application, addressing a critical gap in existing resources.
• Training Innovations: Proposing CLC SFT and G-CLC SFT strategies that significantly enhance small models' ability to handle long contexts and multi-file dependencies.
• Empirical Validation: Comprehensive experiments demonstrating that the cascaded approach with fine-tuned small models can outperform direct editing by large models while reducing costs.

4. Experimental Results

The authors evaluated their approach on the Aider and CanItEdit benchmarks using DeepSeek-V3/R1 as generators and Qwen2.5-Coder (0.5B–14B) as applicators.

• Effectiveness (Performance):

  • Surpassing Direct Editing: On the Aider benchmark, using DeepSeek-R1 for sketch generation and a fine-tuned Qwen2.5-Coder 14B for application achieved a Pass@2 of 75.1%. This is an 11.1% improvement over direct editing with DeepSeek-R1 alone (67.6%).
  • Small Model Gains: Fine-tuning small models (e.g., 0.5B) with G-CLC SFT improved their Pass@2 in the cascaded pipeline by up to 158.1% compared to their untuned counterparts.
  • Generalization: The framework works effectively with various large models (Qwen3, GPT-OSS), consistently improving Pass@2 scores.

• Efficiency (Cost & Time):

  • Time Reduction: The cascaded approach reduced execution time by 13% (Aider) and 19% (CanItEdit) compared to direct editing with DeepSeek-R1.
  • Cost Reduction: Computational costs were reduced by 19% on Aider and 14% on CanItEdit.
  • Token Efficiency: The large model generated significantly fewer tokens (e.g., ~221k vs. 2.6M for DeepSeek-R1 on Aider) because it only produced sketches, not full files.

• Ablation Studies:

  • Training Strategy: G-CLC SFT consistently outperformed standard SFT and CLC SFT, proving the value of mixing general-domain data.
  • Context Handling: Models trained with CLC SFT showed significantly less performance degradation as code length increased (from 1k to 4k+ tokens) compared to baseline models.

5. Significance

This work establishes a new paradigm for AI-assisted software development by demonstrating that specialization can outperform monolithic scaling.

• Practical Impact: It provides a viable path to using powerful LLMs for code editing without the prohibitive costs and latency associated with regenerating entire files.
• Scalability: By offloading the bulk of token generation to efficient small models, the approach makes advanced code editing accessible and scalable for real-world IDEs and CI/CD pipelines.
• Research Direction: It highlights the critical importance of "application" capabilities in small models and provides the necessary data and training methodologies to unlock them, shifting the focus from merely making models bigger to making them more specialized and efficient.