In the rapidly evolving world of artificial intelligence, researchers rely on tests called benchmarks to measure how well a computer program thinks. These tests are like standardized exams for machines, presenting them with puzzles, logic problems, or creative tasks to see how they perform. For years, the assumption has been that humans design these exams, carefully crafting questions and grading rules to ensure they measure what they claim to measure. But as the technology used to build these tests has advanced, a new question has emerged: what happens when the tests themselves are written by the very machines they are meant to evaluate? This shift raises a critical concern about the reliability of the scores we see. If an artificial intelligence designs a test, it might accidentally create a puzzle it can solve easily without actually being smart, or it might miss a flaw that allows a machine to exploit the system. Understanding whether these self-made tests are trustworthy is essential for anyone trying to gauge the true capabilities of modern computer systems.

A recent study by independent researcher Pooria Arab explores exactly this scenario. The researcher set up an experiment where artificial intelligence coding agents were tasked with building a suite of twenty-two different benchmarks. These agents wrote the code, designed the tasks, and created the grading systems for everything from drawing pictures to solving complex logic puzzles. To ensure the process was rigorous, a separate group of artificial intelligence reviewers, acting as a council, examined the work. They read the code changes and approved ninety-four out of the one hundred twenty-three updates that were merged into the final collection. On paper, the process seemed successful: the tests were built, reviewed, and approved. However, a deeper investigation revealed a startling reality. When the researcher ran a special kind of probe against the finished tests—essentially attacking them with simple, generic strategies designed to find loopholes—a probing review found defects in all 22 of the packages.

The investigation uncovered that the initial approvals, while real and thorough in reading the code, failed to catch significant flaws because they did not run the tests against actual attacks. The probing review found that every package needed at least one fix round, with some requiring up to nine rounds of corrections. The record notes that while these fixes addressed logged issues, it does not show that no defects remain. In many cases, the tests were so flawed that a computer program could earn a high score by doing almost nothing. For instance, in thirteen of the packages, a strategy that completely ignored the task and simply gave a constant, generic answer still managed to receive credit. In one specific case involving a drawing test, a computer could score perfectly on every question by submitting a single, memorized image, even though the test was supposed to grade the ability to draw a specific scene. In another instance, a test designed to check if a computer could handle a shopping cart failed to penalize a program that never actually paid for the items.

The study highlights a specific gap in how these tests are currently validated. The reviewing council had done their job by reading the code and flagging obvious errors, such as a hidden seed value that was accidentally left in the code or a claim in the documentation that did not match the actual program. Yet, these reviews missed the deeper issues that only appeared when the tests were actually run. The researcher found that the defects were not random; they fell into thirteen distinct categories, ranging from tasks that were impossible to solve to grading systems that accepted forged reports. One of the most common failures was that the tests allowed "blind strategies" to succeed. This means a computer could win by guessing or repeating a pattern without understanding the question, simply because the test creator had not anticipated such a simple trick.

To test whether simple checks could catch these problems, the researcher ran a pre-planned experiment using eight generic, trivial policies. These were basic rules, like always saying "yes" or always returning an empty answer, designed to see if they could exploit the tests. The results were sobering. The best trivial score fell in only 2 packages, a null result that suggests generic policies miss most of the logged defects. This indicates that the defects found were not obvious holes that a generic check could easily find; instead, they were subtle issues that required a deep understanding of the specific task to uncover. The fact that the reviewers missed these issues does not mean they were incompetent; rather, it shows that reading code is different from attacking a running system. The reviewers saw the plan, but the probes saw the reality of how the system behaved when pushed.

The study concludes that while artificial intelligence can successfully build complex testing frameworks, the current methods of approval are insufficient to guarantee their quality. The researcher released the entire collection of twenty-two packages, along with the engine that runs them and the detailed logs of every fix and flaw found. This release serves as a transparent record of what happens when machines build tests for machines. It demonstrates that even with a council of reviewers and a pipeline of automated checks, significant defects can slip through if the testing process does not include active, adversarial probing. The work does not claim that artificial intelligence cannot build good tests, but it strongly suggests that the current approval processes are missing a crucial step. Until tests are rigorously attacked and verified against simple exploitation strategies, the scores they produce may not reflect the true intelligence of the systems being measured. The researcher emphasizes that no real-world computer models were actually run on these tests in this study, so the results are a measure of the test designs themselves, not the capabilities of any specific artificial intelligence. The findings stand as a warning: a green light from a reviewer is not the same as a clean bill of health from a rigorous attack.