Technical Summary: TICoder

Problem Statement

Repository-level code generation using Large Language Models (LLMs) faces significant challenges due to complex code dependencies and limited context windows. While existing approaches utilize Retrieval-Augmented Generation (RAG) and planning mechanisms to reuse functions from a repository, the authors identify two critical limitations in current state-of-the-art (SOTA) methods:

1. Lack of Test-Driven Behavioral Guidance in Planning: Existing planning-based methods derive implementation steps directly from natural language (NL) requirements. However, NL requirements are often vague or incomplete, leading to unreliable plans. Current test-driven generation methods primarily apply test cases during the code generation or post-generation stages, leaving their potential to guide the planning process unexplored.
2. Lack of Code-Level Implementation Awareness in Reuse: Existing retrieval strategies rely heavily on semantic similarity between implementation steps and function descriptions. They often overlook the actual implementation logic embedded in the code. Furthermore, retrieved functions are frequently inserted into prompts without understanding how they are actually used (usage patterns) within the repository, leading to ineffective reuse.

Methodology: TICoder Framework

The authors propose TICoder, a framework designed to improve both the planning and reuse phases of repository-level code generation. The framework consists of three main stages:

1. Test-Driven Iterative Planning

Instead of generating plans solely from NL requirements, TICoder leverages test cases as behavioral specifications to refine implementation steps. This stage employs a "judge-and-reflection" mechanism involving two LLM roles:

• LLM-as-a-Planner: Generates an initial sequence of implementation steps based on the requirement and test cases, guided by principles such as clarity, boundary condition awareness, and explicit dependencies.
• LLM-as-a-Judge: Evaluates the generated steps against three dimensions: functional coverage/consistency, independence, and granularity.
• Iterative Refinement: If the steps do not meet quality thresholds, the judge provides feedback, and the planner refines the steps. This loop continues until the plan is satisfactory or a maximum iteration limit is reached.

2. Implementation-Aware Code Reuse

This stage focuses on retrieving relevant callee functions and identifying how to use them effectively.

• Dual-View Callee Function Retrieval: To retrieve functions, TICoder computes a joint similarity score combining:
  • Functional Similarity ($Sim_F$): Semantic similarity between the implementation step and the function description.
  • Implementation Similarity ($Sim_I$): Similarity between a code representation of the implementation step (generated by an LLM) and the actual function implementation code.
  • The final score is a weighted sum ($Sim = \alpha \cdot Sim_F + \beta \cdot Sim_I$).
• Dual-Stage Usage Pattern Selection: To ensure functions are used correctly, TICoder selects representative usage patterns from the repository:
  • Graph-Based Identification: Constructs a Repository Call Graph (RCG) to identify all caller functions for a retrieved callee.
  • Structure-Based Clustering: Groups caller functions based on the set of functions they invoke (structural equivalence).
  • Perplexity-Based Filtering: Within each cluster, the most relevant caller (lowest perplexity) is selected as the center. A global filter then ranks these centers by perplexity to select the top $n_p$ patterns, balancing context richness with token efficiency.

3. Enhanced Code Generation

The final code generation phase integrates the refined implementation steps, retrieved callee functions, selected usage patterns, test cases, and the original requirement into a single prompt for the LLM to generate the final code.

Key Contributions

The paper claims three primary contributions:

1. TICoder Framework: A novel repository-level code generation framework that integrates test-driven iterative planning and implementation-aware code reuse.
2. Implementation-Aware Reuse Strategy: A dual-view retrieval mechanism (functional + implementation similarity) and a dual-stage selection strategy (clustering + perplexity filtering) to precisely identify and reuse repository functions.
3. Extensive Empirical Evaluation: Comprehensive experiments demonstrating that TICoder outperforms SOTA baselines across multiple LLMs and benchmarks.

Experimental Results

The authors evaluated TICoder on two widely used benchmarks, CoderEval (Python and Java) and DevEval (Python), using three backbone LLMs: GPT-4o-mini, DeepSeek-V3, and Qwen2.5-Coder-7B.

• Overall Performance: TICoder consistently outperformed all baselines (including SimpleRAG, RepoCoder, AllianceCoder, and RepoScope). It achieved an average improvement of 11.52% over the best-performing baseline.
• Statistical Significance: A two-tailed t-test confirmed that the performance gains were statistically significant ($p < 0.05$).
• Ablation Study: Removing any key component (planning, test cases, iterative refinement, dual-view retrieval, or usage patterns) resulted in performance degradation, validating the necessity of each module.
  • Notably, removing the planning module caused a 6.87% drop in Pass@1.
  • Removing test cases from the generation stage caused a substantial 10.42% drop.
• Hyperparameter Sensitivity:
  • Iterations: Performance generally improved with iterations but was non-monotonic; excessive iterations could introduce noise.
  • Similarity Weights: Functional similarity was generally more important, but implementation similarity provided significant gains, particularly for Java.
  • Usage Patterns: A moderate number of patterns (e.g., 2) yielded the best results; too few limited context, while too many introduced noise.

Significance and Claims

The paper asserts that TICoder addresses the specific limitations of prior planning-based RACG approaches by explicitly incorporating test cases into the planning phase and grounding retrieval in actual code implementation logic. The authors claim that by bridging the gap between natural language requirements and executable code through behavioral specifications (tests) and structural context (usage patterns), TICoder enables LLMs to generate more accurate and contextually appropriate code in complex repository settings. The work highlights that successful repository-level generation requires not just finding similar code, but understanding how that code is behaviorally specified and structurally integrated within the existing codebase.