Articulate but Wrong: Self-Review Failures in LLM-Based Code Modernization
This paper demonstrates that LLM-based code modernization frequently introduces silent behavioral drifts that the models themselves cannot reliably detect or self-correct, revealing that this failure mode is task-structural rather than dependent on model scale or capability.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you hire a very articulate, confident architect to renovate an old, creaky house (legacy code) into a modern, sleek home. You ask them to do the work, and then, to save money, you ask the same architect to look at their own blueprints and say, "Did I accidentally change how the house works?"
This paper asks a simple but scary question: Can we trust the architect to catch their own silent mistakes?
The researchers found that while these AI "architects" are great at making the house look new, they are surprisingly bad at noticing when they've quietly broken the plumbing or changed the electrical wiring in ways that don't cause a fire, but just make the house behave differently.
Here is the breakdown of their findings using everyday analogies:
1. The "Silent Drift" Problem
When the AI tries to update old code, sometimes it makes a change that looks perfect. The code runs, it doesn't crash, and it looks right. But under the hood, the numbers come out slightly different.
- The Analogy: Imagine the old house had a rule where "half a loaf of bread" meant cutting it in half and giving you the whole half. The new house changes the rule so "half a loaf" means giving you a tiny crumb. The house still stands, and the bread is still there, but the amount you get is wrong.
- The Finding: The researchers tested 60 specific tricky scenarios. When the AI had to do real, complex updates, it made these silent mistakes 40% of the time. When they gave the AI easy tasks that didn't actually need changing, it only made mistakes 7% of the time. This proves the AI isn't just being careless; it's specifically struggling with the logic of the update.
2. The "Arithmetic Trap"
The biggest culprit for these mistakes was how the AI handles numbers.
- The Analogy: In the old house (Python 2), if you divide 5 cookies among 2 people, you get 2 cookies each (the extra half is thrown away). In the new house (Python 3), you get 2.5 cookies.
- The Finding: The AI often forgets this rule. It updates the code but keeps the "throw away the half" logic, or vice versa. This specific math error happened in 57% of the difficult cases. It's like the AI knows how to paint the walls but forgot how to count the bricks.
3. The "Confident Liar" (Self-Review Failure)
This is the most shocking part. After the AI finishes the renovation, the researchers asked it: "Did you change how the house works?"
- The Analogy: The AI looks at its own blueprint, sees the broken plumbing, and says, "Yes, I changed it, but don't worry, it's fine!" Or worse, it sees the broken plumbing, explains exactly why it's broken, and then concludes, "Therefore, the house is perfect."
- The Finding: When the AI actually made a silent mistake, it failed to catch itself 32% of the time. It confidently endorsed its own broken work.
- Some models were like a paranoid inspector who caught every single mistake (0% failure).
- Other models were like a confident liar who missed every single mistake (100% failure).
- Crucially, the "best" or most expensive models were not necessarily the ones who caught their own errors. A cheap model sometimes did better than an expensive one at spotting its own flaws.
4. The "Magic Mirror" Doesn't Work
The researchers tested if asking the AI to be its own "safety net" works.
- The Analogy: You might think, "If I ask the architect to double-check their work, they'll fix it."
- The Finding: No. The "self-check" is not a reliable safety net. The AI is too good at talking its way out of trouble. It can write a paragraph explaining the difference between the old and new rules, and then immediately conclude that the rules are the same. It's like a student who writes a perfect essay explaining why 2+2=5, and then circles the answer "5" with a smile.
5. It's Not About How "Smart" the AI Is
You might assume that the most powerful, expensive AI models would make fewer mistakes and catch them better.
- The Analogy: You'd think hiring a "Master Architect" would be safer than hiring a "Junior Architect."
- The Finding: The study found no clear link between the price of the AI and its reliability. The most expensive models made just as many silent mistakes as the cheaper ones. The problem isn't that the AI isn't "smart" enough; it's that the task of updating code has a specific structural trap that confuses all of them, regardless of how much they cost.
The Bottom Line
If you are using AI to update old software, do not trust the AI to check its own work.
The paper concludes that asking the AI, "Did you break anything?" is like asking a magician, "Did I make the rabbit disappear?" The magician will say "Yes," even if the rabbit is still there, or even if they accidentally turned the rabbit into a pigeon.
To be safe, you need an external "oracle" (a strict, automated test) that checks the output against the original rules, rather than relying on the AI's own opinion. The AI is articulate, but in this specific job, it is also dangerously wrong.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.