AI-Assisted Code Review as a Scaffold for Code Quality and Self-Regulated Learning: An Experience Report
This paper presents a mixed-methods experience report on integrating an LLM-based reviewer into GitHub pull requests to scaffold code quality and self-regulated learning in software engineering capstone projects, demonstrating that structured AI feedback can promote iterative development and meaningful student engagement when paired with proper instructional guidance.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are learning to cook in a professional kitchen. You have a head chef (your professor) who is very busy, and you have fellow student cooks (your peers). You want to make sure your soup is perfect, but your peers are often too shy to tell you it’s too salty, and the head chef doesn't have time to taste every single spoonful.
This paper describes a study where researchers introduced a "Smart Sous-Chef" (an AI tool) into a university software engineering classroom to help students improve their "recipes" (their computer code).
Here is the breakdown of how it works and what they found:
1. The Problem: The "Shy Peer" and the "Busy Boss"
In university projects, students have to review each other's work. But two things usually go wrong:
- The Peer Problem: Students feel awkward criticizing their friends. They end up giving "polite but useless" feedback like, "Looks good!" even when the code is messy.
- The Scale Problem: Professors are too busy to check every single line of code for hundreds of students.
2. The Solution: The "Smart Sous-Chef" (The AI)
Instead of just giving students an AI that writes the code for them (which is like a chef who just hands you a pre-made frozen meal), the researchers built a specialized AI.
This AI doesn't say, "Here is the fixed code." Instead, it acts like a highly observant assistant who points at your pot and says: "Hey, you forgot to season this," or "The temperature seems a bit high here." It provides a checklist of things to look for—like naming, organization, and logic—but it leaves the actual cooking and decision-making to the student.
3. The Experiment: Two Years of Testing
The researchers watched two groups of students over two years to see if this "Smart Sous-Chef" actually helped them learn or if they just became lazy.
- Year 1 (The Growing Pains): The tool had some "glitches." Sometimes the AI would get overwhelmed if a student tried to show it too much at once (like trying to show a chef a whole grocery store instead of one recipe).
- Year 2 (The Smooth Kitchen): They fixed the glitches. They taught students how to use the AI properly—showing it small, specific pieces of work rather than everything at once.
4. The Results: Did they actually learn?
The researchers looked at two things: Behavior (what they actually did) and Perception (what they said they felt).
- The "Action" Metric (Behavior): They found that about one-third of the time, when the AI pointed something out, the students immediately went back and changed their code. This is the "smoking gun" of learning—it shows they weren't just reading the AI's comments; they were actually reacting to them.
- The "Critical Thinker" (Perception): Surprisingly, the students didn't treat the AI like a god. They realized the AI was great at "cleaning the kitchen" (fixing formatting and typos) but wasn't very good at "tasting the complex flavors" (understanding deep, complicated logic). Because they knew the AI had limits, they didn't blindly follow it; they used it as a tool, not a crutch.
The Big Takeaway
The paper concludes that AI in education shouldn't be an "Oracle" (an all-knowing being that gives answers) but a "Scaffold" (a temporary structure that helps you build something higher).
If you design AI to be a critique partner rather than a ghostwriter, students don't stop thinking—they actually start thinking more deeply about the quality of their own work.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.