Fine-Tuning Models for Automated Code Review Feedback

Technical Summary: Fine-Tuning Models for Automated Code Review Feedback

Problem Statement
While Large Language Models (LLMs) have transformed programming education by enabling personalized feedback and content creation, significant barriers remain. Proprietary models, though effective, are costly, lack transparency, and raise ethical concerns regarding student data privacy. Conversely, open LLMs offer accessibility and control but currently lack the performance capabilities of their proprietary counterparts. Furthermore, existing automated feedback tools often rely on proprietary black-box models or fail to generate high-quality "Knowledge about How to Proceed" (KH) feedback, which is crucial for guiding learners through problem-solving rather than simply providing answers. There is also a scarcity of publicly available, annotated datasets for training models to generate pedagogically structured feedback (specifically "Knowledge about Mistakes" or KM, and KH).

Methodology
The study proposes a framework to enhance the feedback quality of the open-source Code Llama-7B-Instruct model using Parameter-Efficient Fine-Tuning (PEFT) and synthetic data generation.

1. Dataset Creation: Due to the lack of public datasets, the authors generated a new dataset of 425 annotated triplets consisting of buggy Java code, KM feedback (identifying errors), and KH feedback (guiding fixes). This was achieved by:

   • Analyzing 85 distinct bug types identified in literature from 2005–2024.
   • Using DeepSeek-R1 (chosen for its permissive license) to generate the triplets via prompt engineering.
   • Manually validating the generated feedback for correctness.
   • The dataset covers both imperative and object-oriented error types.

2. Model Architecture and Training:

   • Base Model: Code Llama-7B-Instruct, selected for its balance of performance and resource efficiency.
   • Fine-Tuning Strategy: The authors employed Quantized Low-Rank Adaptation (QLoRA) to fine-tune the model. This approach freezes the base model weights and updates only a small subset of parameters (specifically $q\_proj$ and $v\_proj$ layers) using 4-bit quantization. This reduced trainable parameters from ~6.7 billion to ~5.2 million, minimizing computational overhead and the risk of catastrophic forgetting.
   • Training Protocol: A 5-fold cross-validation approach was used to ensure generalizability.

3. Comparative Baselines: The PEFT model was compared against:

   • Baseline: Standard Code Llama-7B-Instruct with zero-shot prompting.
   • Prompt Engineering (PE-FewShot): In-context learning using three examples from the training dataset.

4. Evaluation: A multi-faceted evaluation strategy was employed:

   • Instructor Evaluation: Manual annotation using a rubric measuring KM Accuracy, KH Helpfulness, Misleading Suggestions, and Prompt Adherence.
   • Automated Metrics: BLEU, ROUGE-L, and BERTScore to measure similarity against the ground truth.
   • Student Evaluation: A focus group with seven CS1 students who rated feedback from the PEFT model, ChatGPT (proprietary), and compiler error messages on usefulness, clarity, and structure.

Key Contributions

• A Novel PEFT Framework: A method to generate pedagogically structured KM and KH feedback for Java programming using open LLMs.
• New Public Dataset: A dataset of 425 annotated Java bug triplets (Code, KM, KH) covering diverse error types, addressing the scarcity of public resources in this domain.
• Empirical Evidence: Demonstration that PEFT significantly outperforms prompt engineering and baseline open models, achieving performance comparable to proprietary models in student evaluations.

Results

• Instructor Evaluation: The PEFT model (CL7B-LoRA-JavaFB) outperformed both the baseline and the few-shot prompt engineering approach across all rubric criteria.
  • KM Accuracy: Improved from 20% (baseline) to 61% (PEFT).
  • KH Helpfulness: Improved from 26% (baseline) to 60% (PEFT).
  • Misleading Suggestions: Reduced significantly from 83% (baseline) to 47% (PEFT).
  • Prompt Adherence: Increased from 54% to 95%.
• Automated Metrics: While BLEU and ROUGE scores were generally low (indicating lexical divergence from ground truth), BERTScore was high for the PEFT model, suggesting strong semantic alignment with the correct feedback. The authors note that high BERTScore alone does not guarantee pedagogical soundness.
• Student Evaluation: Students rated the PEFT model's feedback as equally effective as ChatGPT and significantly better than raw compiler errors.
  • Strengths: The PEFT feedback was praised for being concise, precise, and encouraging critical thinking without providing the direct solution.
  • Weaknesses: Some students noted that the proprietary model (ChatGPT) offered more detailed explanations, while the PEFT model sometimes lacked sufficient elaboration on technical terms.
  • Preference: Students indicated the PEFT model was better suited for formative lab settings where critical thinking is prioritized.

Significance and Claims
The paper claims that parameter-efficient fine-tuning is a viable and effective strategy for adapting open LLMs to educational tasks, offering a freely deployable alternative to costly proprietary models. The study demonstrates that distilling knowledge from a larger model (via synthetic data generation) into a smaller, fine-tuned open model can double the accuracy and helpfulness of feedback while reducing hallucinations.

The authors modestly conclude that while PEFT models show great promise for scalable pedagogical systems, they are not yet perfect. The presence of misleading suggestions, though reduced, necessitates human verification before full deployment. Additionally, the study highlights that while the PEFT model excels in formative settings by fostering critical thinking, it may require further optimization (e.g., adding technical term explanations) to match the depth of proprietary models in all contexts. The work underscores the importance of open models in ensuring transparency and privacy in programming education.