Helping LLMs Improve Code Generation Using Feedback from Testing and Static Analysis

This paper presents a framework that uses testing and static analysis feedback to guide the self-improvement of open-source LLMs, revealing that while these models struggle to generate safe code or detect their own errors, they demonstrate a significant ability to fix flaws when provided with specific diagnostic feedback.

The Gist

Imagine you have a team of incredibly talented, fast-talking apprentices. They can write computer code (the instructions that make software run) at lightning speed. These apprentices are Large Language Models (LLMs). They are great at learning patterns and mimicking what they've seen before, but they are also prone to making silly mistakes, just like any human learning a new trade.

This paper is about a group of researchers who decided to put these digital apprentices through a rigorous "boot camp" to see how good they really are at writing safe, working code, and if they can fix their own mistakes when told about them.

Here is the story of their experiment, broken down into simple steps:

1. The Setup: The "Code Factory"

The researchers gave four different AI models (think of them as four different apprentices: Llama 3, Gemma, and Mixtral) a list of 100 simple programming puzzles. They asked the AI to write C code (a strict, old-school programming language) to solve them.

• The Problem: The AI is like a chef who loves to talk while cooking. Sometimes, it writes the recipe but forgets to add salt, or it writes a recipe that sounds good but would actually burn the kitchen down.
• The Result: When the AI wrote the code, only about half of it actually worked correctly. The rest had bugs (errors) or wouldn't even run.

2. The Safety Inspector: The "Static Analysis Tool"

Next, the researchers didn't just run the code; they hired a super-strict safety inspector named Infer. This inspector doesn't run the code; it reads the blueprint to find hidden dangers like memory leaks (leaving the gas on) or buffer overflows (trying to pour a gallon of water into a pint-sized cup).

• The Finding: The inspector found that while the code wasn't always dangerous, it did have safety holes. About 90% of the code passed the safety check, but that 10% failure rate is huge in the real world where one mistake can crash a system.

3. The Self-Reflection: "Do You Know You're Wrong?"

This is the most interesting part. The researchers asked the AI: "Hey, look at the code you just wrote. Is it correct? Is it safe?"

• The Reality Check: The AI was terrible at this. It was like asking a student to grade their own homework without looking at the answer key.
  • Some AIs were overconfident, saying "Yes, this is perfect!" even when it was broken.
  • Others were overly pessimistic, saying "No, this is broken!" even when it worked.
  • The Lesson: The AIs have very little "self-awareness." They can't reliably tell if they made a mistake.

4. The Repair Shop: "Can You Fix It?"

Finally, the researchers gave the AI a second chance. They showed the AI the broken code and said, "Here is exactly what is wrong: The code crashed here, and this safety inspector found a hole here. Please fix it."

• The Result: This is where the magic happened. When the AI was given specific feedback (like a teacher pointing out the exact error), it became a repair expert.
  • For fixing broken logic, the best AI managed to fix about 62% of the errors.
  • For fixing safety holes, the best AI managed to fix a whopping 89% of the vulnerabilities.

The Big Takeaway (The Analogy)

Think of these AI models like brilliant but scatterbrained interns.

1. Generation: If you just tell them "Write me a bridge," they might build one that looks great but collapses under a car. They get the idea, but the details are often wrong.
2. Self-Evaluation: If you ask them, "Is this bridge safe?", they will confidently say "Yes!" even if it's about to fall. They don't know what they don't know.
3. Repair: However, if you hand them a blueprint with a red marker saying, "The support beam here is too thin," they can usually fix it perfectly.

Why This Matters

The paper concludes that we shouldn't just trust AI to write code and walk away. We need a human-in-the-loop or an automated safety inspector (like the tool they used) to catch the mistakes.

The good news? If we build a system where the AI writes the code, a tool checks it, and then the AI fixes the specific errors the tool finds, we can create software that is much safer and more reliable. It turns the AI from a "scatterbrained genius" into a "highly skilled craftsman" who learns from its mistakes.

In short: AI is great at writing code, terrible at knowing when it's wrong, but excellent at fixing things when you tell them exactly what's broken.

Technical Summary

1. Problem Statement

Large Language Models (LLMs) are increasingly used to generate code, accelerating software development. However, current models often produce code that is functionally incorrect, contains security vulnerabilities (e.g., memory leaks, buffer overflows), and exhibits poor "self-awareness" (the ability to detect errors in their own output).

• The Gap: While LLMs can generate code, they struggle to ensure its safety and correctness autonomously. Furthermore, existing research often relies on closed-source models (like GitHub Copilot) or lacks a systematic framework for iterative self-improvement using automated feedback.
• The Goal: The authors aim to evaluate the correctness and safety of code generated by open-source LLMs, measure their ability to self-evaluate and repair these issues, and propose a framework to improve code quality through automated feedback loops.

2. Methodology

The authors propose a three-phase experimental framework involving four open-source LLMs: Llama 3 (8B and 70B), Gemma 7B, and Mixtral 8x7B. The experiments focus on the C programming language due to its strict memory management requirements.

A. Benchmark Creation

• Dataset: 100 tasks were randomly selected from the Mostly Basic Python Problems (MBPP) dataset and manually translated into C.
• Adaptation: Unit tests were refactored from Python assertions to C-compatible logic (e.g., using memcmp for array comparison) to ensure fair evaluation.
• Motivation: Using a C-translation of Python tasks minimizes the risk of model memorization of existing C benchmarks (like HumanEval).

B. Experimental Phases

1. Code Generation:

   • LLMs generate C code based on natural language prompts.
   • Cleaning: An automated script removes non-code text, adds missing standard libraries (e.g., math.h), and fixes formatting issues to ensure compilability.
   • Evaluation:
     • Correctness: Code is compiled (Clang 15.0) and run against ground-truth unit tests. Outcomes are classified as: All tests passed, Execution error/timeout, Assertion failure, or Compilation error.
     • Safety: Compiled code is analyzed using Infer (a static analysis tool by Meta) to detect vulnerabilities like buffer overruns, null pointer dereferences, memory leaks, and uninitialized values.

2. Self-Evaluation:

   • LLMs act as classifiers to evaluate code generated by themselves and other models.
   • Correctness Check: Models answer "YES/NO" on whether code solves the task.
   • Safety Check: Models answer "YES/NO" on whether specific vulnerabilities (identified by Infer) exist in the code.
   • Prompting: Various prompt engineering techniques (Chain-of-Thought, few-shot examples) were tested to optimize performance.

3. Repair (Self-Improvement):

   • LLMs are tasked with fixing code that failed correctness tests or contained vulnerabilities.
   • Feedback Loop: The models receive the original code, the task description, and specific feedback (failed test cases or vulnerability reports from Infer).
   • Iterative Process: The repair process is iterative (up to 6 attempts), allowing the model to refine the code based on new feedback.

3. Key Contributions

• Comprehensive Framework: A fully automated pipeline integrating code generation, static analysis (Infer), unit testing, and iterative repair for open-source LLMs.
• Quantitative Baseline: Detailed metrics on the correctness (46%–65%) and safety (87%–96% vulnerability-free) of code generated by state-of-the-art open models in C.
• Self-Awareness Analysis: Evidence that LLMs perform poorly at detecting their own errors (low F1-scores) and show no significant "self-preference" (they do not fix their own code better than others' code).
• Repair Capabilities: Demonstration that while LLMs struggle to generate perfect code initially, they show substantial ability to fix code when provided with specific feedback (test failures or static analysis reports).
• Prompt Engineering Insights: Systematic evaluation showing that prompt design (e.g., providing specific instructions vs. Chain-of-Thought) significantly impacts repair success rates.

4. Key Results

Code Generation Performance

• Correctness: Only 46% to 65% of generated files passed all unit tests. llama3-70b-8192 performed best (65%), while gemma-7b-it performed worst (46%).
• Safety: 87% to 96% of generated files were found to be free of vulnerabilities by Infer. However, mixtral-8x7b had a higher rate of unsafe files compared to others.
• Observation: Code is more likely to be functionally incorrect than insecure.

Self-Evaluation Performance

• Correctness Detection: Models performed poorly, with F1-scores generally under 80%. gemma-7b-it tended to reply "YES" (over-optimistic), while llama3-8b-8192 tended to reply "NO" (over-pessimistic).
• Vulnerability Detection: Performance was extremely low (F1-scores < 0.3). Models failed to identify vulnerabilities even when they generated the code themselves. llama3-8b-8192 often missed all vulnerabilities (high accuracy due to always saying "NO"), while gemma-7b-it flagged everything (high recall, low precision).

Repair Performance

• Correctness Repair: When provided with failed test feedback, llama3-70b-8192 successfully repaired 59% of incorrect files. gemma-7b-it only managed 13%.
• Safety Repair: When provided with static analysis reports, the best models (llama3-70b-8192 and mixtral-8x7b) fixed 89% of detected vulnerabilities.
• Self-Preference: No evidence was found that models repair their own code better than code generated by others. In fact, models often repaired code generated by gemma-7b-it better than their own.
• Iteration: The number of failed tests/vulnerabilities decreased significantly after the first iteration, stabilizing after two attempts.

5. Significance and Future Directions

• Practical Impact: The study validates that iterative feedback loops combining static analysis and unit testing are highly effective for improving LLM-generated code safety. It suggests that LLMs should not be used as "zero-shot" code generators for critical systems but rather as components in a pipeline with automated verification.
• Open Source Advancement: By using open-source models, the authors demonstrate that high-quality, safe code generation is achievable without proprietary APIs, provided robust verification tools are integrated.
• Future Work:
  • Extending the framework to other languages (e.g., Rust for memory safety, Python for dynamic typing).
  • Integrating hybrid analysis (combining static and dynamic analysis) to reduce false positives/negatives.
  • Investigating whether LLMs are "memorizing" code or truly generalizing to new tasks.

In conclusion, the paper establishes that while LLMs currently lack the self-awareness to guarantee code safety, they possess strong repair capabilities when guided by rigorous, automated feedback from testing and static analysis tools. This points toward a future where LLMs are integrated into "self-healing" software development pipelines.