Judging Is Not Enumerating: Silent Omissions in LLM-Authored Acceptable Sets
This paper demonstrates that while large language models excel at judging whether candidates belong to a correct set, they significantly fail at authoring complete acceptable sets due to a systematic tendency toward silent omissions rather than over-inclusions, a deficit that persists across various domains and parameter scales.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
The Great Role Reversal: When AI Becomes the Teacher
Imagine a classroom where the students are no longer just answering questions; they are writing the textbooks, grading the exams, and deciding what counts as a "correct" answer. This is happening right now in the world of artificial intelligence. Large Language Models (LLMs) are being promoted from "examinees" (the ones taking the test) to "examiners" (the ones creating the test). They are writing the rules, the answer keys, and the checklists that tell other AI systems whether they are doing a good job.
To understand why this is tricky, think of two different skills. The first is Judging: looking at a single answer and saying, "Yes, that's right," or "No, that's wrong." The second is Authoring: creating the entire list of every single possible right answer from scratch. It's the difference between a referee blowing a whistle when a player steps out of bounds, and a coach trying to write a rulebook that lists every possible way a player could step out of bounds without missing a single one. We care about this because if the AI writing the test makes a mistake, it doesn't just get one question wrong; it changes the definition of "correct" for everyone else, potentially punishing good answers and rewarding bad ones.
The Paper's Big Discovery: The Silent Ghost in the Machine
This paper investigates a strange and dangerous glitch in how AI handles these new roles. The researchers found that while AI models are actually quite good at Judging (spotting if a specific answer is right or wrong), they are surprisingly bad at Authoring (writing out the complete list of all the right answers).
Think of it like a game of "Simon Says." If you ask the AI, "Is 'apple' a fruit?" it will almost always say "Yes" correctly. But if you ask it, "List every single fruit that exists," it will start listing apples and bananas, but then it will just... stop. It will forget the oranges, the kiwis, and the mangoes. It doesn't forget because it doesn't know what they are; it forgets because it's bad at the specific task of writing out the whole list.
The authors call this the "Judging-Enumerating Scissors." Imagine a pair of scissors where one blade is "Judging" (sharp and strong) and the other is "Authoring" (dull and weak). As the AI models get bigger and smarter (scaling up from small to massive), the "Judging" blade gets sharper, but the "Authoring" blade barely gets any sharper. They never catch up. The gap between how well they can judge and how well they can list remains wide, no matter how much computing power you give them.
The Silent Ghost: Why We Don't Notice the Mistakes
The most terrifying part of this discovery is how the AI fails. When it writes a list of correct answers, it doesn't usually add fake answers (like calling a "rock" a fruit). Instead, it commits Silent Omissions. It simply leaves out the real answers.
Here is why that is so dangerous:
- Over-inclusion (Adding wrong things): If the AI accidentally adds "rock" to the fruit list, a human reviewer can easily spot it. "Hey, rocks aren't fruits!" they say. It's a visible error.
- Omission (Leaving out right things): If the AI forgets to list "kiwi," there is no error to see. The list just looks a bit shorter. To find the missing kiwi, the reviewer has to know the answer themselves and then hunt for it.
The paper shows that AI models are 6 to 7 times better at spotting things they added by mistake than they are at realizing they missed something. It's like a student who is great at finding typos in their essay but terrible at realizing they forgot to write the conclusion. Because the missing parts are invisible, the AI's "answer keys" end up being incomplete, and the AI systems grading on those keys end up punishing correct answers just because they weren't on the list.
The "Magic Rule" Loophole
The researchers tested a clever workaround. They asked the AI not to list the fruits, but to write the rule for what makes a fruit a fruit (e.g., "If it grows on a tree and has seeds"). When they did this, the AI became nearly perfect (scoring 99% accuracy).
This proves the AI isn't stupid. It knows the rules. The problem is purely in the act of listing them. It's like a chef who can perfectly describe the recipe for a cake but keeps forgetting to write down the last three ingredients when asked to write the full shopping list. The paper shows that if you can get the AI to write the rule instead of the list, the problem disappears. But in many real-world cases (like writing a test suite for computer code), there is no simple rule to write; you have to list the specific test cases, and that's where the AI stumbles.
Why "Checking Its Own Work" Doesn't Help
You might think, "Well, if the AI is bad at listing, maybe it can just check its own list to fix the mistakes?" The paper tested this, too. They had the AI generate a list, then generate a second list, and then filter the first list using the second one.
The result? It didn't work. The AI couldn't fix its own omissions. It's like asking a person who is bad at counting to count their own fingers to see if they missed any. The AI's "judging" skill is strong when looking at a clear, pre-made list, but it collapses when trying to judge its own messy, self-generated list. The paper concludes that asking an AI to check its own answer key is a trap; it needs an external, human-verified "gold standard" to catch the missing pieces.
The Real-World Cost: The "Tax" on AI
Finally, the paper measured what happens when this broken answer key is used to train other AIs. They set up a scenario where an AI learns by getting points for correct answers.
- When the AI used a perfect, human-made key, it learned well.
- When it used a model-made key, it learned significantly worse.
On a specific language task, the AI using the model-made key scored 18.5 points lower than the one using the perfect key. Even on a math task where the rules were simple, it still lost 1.9 points. This "tax" happens because the model-made key is too strict: it rejects good answers that the AI generated but forgot to include in its own list. The AI gets confused, thinking its correct answers are wrong, and stops trying to be creative.
The Takeaway
The paper doesn't say AI is useless. It says that when we ask AI to be the "examiner" and write the rules of correctness, we have to be very careful.
- Don't ask for the list; ask for the rule. If the AI can write a rule (like a Python script) to check answers, use that. It's near-perfect.
- Don't trust the list blindly. If the AI has to write a list of correct answers, it will almost certainly miss some.
- Use a "Gatekeeper." Before trusting an AI's answer key, check if it accepts a known-correct answer. If it rejects a known-correct answer, throw the key away.
- Fix, don't discard. If the key rejects a correct answer, you can often just fix the specific mistake (like the expected output of a test) rather than throwing the whole thing away.
The bottom line: AI is great at spotting errors, but terrible at making sure it hasn't missed anything. Until we fix this "silent omission" problem, we can't fully trust AI to grade its own homework.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.