Four Ways to Forge a Bundle My Own Verifier Calls Clean: Refusal-Site Mutation Testing of an Evidence-Bundle Verifier

In the modern digital world, software systems often rely on a simple but powerful idea: trust, but verify. When a computer program claims to have solved a difficult problem or analyzed a massive dataset, it produces a report. To ensure this report is honest, other programs act as auditors. These auditors check the math, verify that the data files haven't been tampered with, and confirm that the numbers in the summary match the raw evidence. If everything checks out, the auditor gives a green light, and the result is published for the world to see. This process is the backbone of reliable software, allowing strangers to trust each other's work without needing to know one another. However, for this system to work, the auditor itself must be perfect. If the auditor fails to spot a lie, or worse, if it declares a lie to be true without actually looking at the evidence, the entire system collapses. The danger is not just that a bad result slips through, but that the auditor might be so broken that it passes a test without ever running the test at all.

A researcher named Erik Hill set out to examine one such auditor, a program designed to verify bundles of evidence for an offline system. He wanted to know a specific, unsettling question: how often does this program say "pass" when it hasn't actually checked anything? To find out, he didn't just look for bugs; he built a tool to systematically break his own auditor. He took every single line of code in the program that was supposed to reject a bad bundle and, one by one, turned them off. Then, he watched to see if the program's test suite would notice. If the tests still passed after a rejection line was removed, it meant that rejection line was dead weight—it was never actually doing any work. The results were startling. At the start of the study, two-thirds of the auditor's rejection points were invisible to the tests. The program could be stripped of most of its ability to say "no," and it would still report a perfect score. It was as if a security guard had been trained to check for stolen items, but the training exercises never actually included a stolen item, so the guard never learned to look for one.

The study began with a shock from an outside expert. An independent engineer submitted a bundle of evidence where the headline numbers were completely false, yet the auditor printed a perfect pass. The fraud cost only four bytes to create. The engineer had replaced a file with a tiny, empty placeholder, and because the auditor failed to check that the file was actually there, it assumed everything was fine. Hill fixed this specific hole, but then he applied his own systematic tool to the repaired program. He found that the problem was not just one mistake, but a pattern. He discovered four more ways to fool the auditor, each time finding that the program was skipping a check entirely. One of these tricks involved changing a single capital letter in a severity label, which caused the program to ignore the weight of a failed check. Another involved deleting a key from a list, causing the program to skip a comparison because the item it was supposed to compare was missing. In every case, the program was not failing to do a hard calculation; it was simply failing to start the calculation at all.

To measure how widespread this issue was, Hill ran his deletion tool against the auditor's code. He found 112 places where the program was supposed to say "no." When he removed them one by one, 75 of them could be deleted without any test failing. This meant that 75 out of 112 rejection points were effectively invisible to the system's safety checks. The program had a score of 0.330, indicating that only about a third of its refusal mechanisms were actually being tested. The remaining two-thirds were "vacuous passes"—checks that reported success even though they never examined the evidence. This is not a rare glitch; it is a structural flaw where the safety net has holes that no one ever tried to fall through. The tests were checking that the program ran, not that it actually looked at the data.

Hill then tested a common approach to fixing such problems: simply fixing the specific bugs that were found and adding a test for each one. He fixed the four forgeries he had discovered and added new tests to ensure those specific tricks wouldn't work again. Surprisingly, this did not improve the overall safety score. The program still had the same 75 invisible rejection points. The new tests only covered the new holes he had just patched, leaving the rest of the system just as blind as before. It was only when he changed his strategy that the numbers moved. Instead of fixing bugs, he wrote a new test for every single one of the 75 invisible rejection points, forcing the program to prove it could actually fire each one. This systematic approach raised the score from 0.330 to 1.000, proving that every rejection point could indeed be triggered. The lesson was clear: fixing known bugs does not make a system safe; you must prove that every single safety mechanism can actually work.

The study also revealed a deeper issue with how these systems are built. The researcher found that the auditor often treated human-readable reports differently from the raw data files used for checking. When a bundle included a report meant for a person to read, the auditor frequently failed to verify that the report matched the underlying data. It was as if the auditor trusted the summary page but ignored the receipts. This happened across multiple different projects, suggesting a common habit among developers: they bind the data that the computer checks, but they leave the data that humans read unverified. The researcher found that this gap allowed false claims to slip through, where a report could say "all defects fixed" while the data showed otherwise.

Throughout the study, the researcher's own tools failed him in ways that mirrored the very problem he was studying. His measuring instruments sometimes reported success even when they weren't measuring anything. In one instance, a tool designed to detect failures returned a perfect score because the baseline test suite had already failed, causing the tool to misinterpret the error as a success. This happened seven times during the research, including one time where the tool gave a perfect score while the system was broken. These failures were not hidden; they were documented in the paper to show that the tools used to verify software are just as vulnerable to these "empty pass" errors as the software they are meant to check.

The final conclusion of the work is a call for a different kind of testing. The researcher argues that we cannot rely on a list of known bugs to keep a system safe. If a system has a safety gate that has never been observed failing, it has never been observed working. The only way to be sure is to systematically test every single gate to ensure it can actually fire. The study showed that a system can appear perfect on paper while being fundamentally broken in practice. By forcing the auditor to prove it could reject bad data in every possible way, the researcher turned a system that was blind to its own failures into one that was fully verified. The work stands as a reminder that in the world of digital trust, the most dangerous error is not a failed check, but a check that never happened at all.