Anchoring Bias in LLM-as-a-Judge Systems: Prior Scores Compromise Evaluation Independence

In the modern digital landscape, artificial intelligence has evolved from a tool that simply answers questions to a system that judges the quality of other systems. This new role, often called "AI as a judge," is increasingly used to grade essays, review code, and filter content before it reaches a human reader. The assumption behind this practice is that the AI judge evaluates each piece of work on its own merits, treating every submission as a fresh start. However, human psychology has long known a phenomenon called anchoring, where a person's estimate or decision is subtly pulled toward a number they saw earlier, even if that number was random or irrelevant. For decades, researchers have observed this in people, from courtroom sentencing to price negotiations. The critical question for the digital age is whether these same human-like biases exist inside the machines we trust to be impartial. If an AI judge is influenced by a previous score it sees in the background, the entire system of automated evaluation could be systematically skewed, leading to unfair rejections or approvals of content.

A team of researchers from Infobip set out to test this assumption directly. They designed an experiment to see if an AI judge's score would change simply because it was shown a previous score from a different judge, even when that previous score was completely unrelated to the current work. They gathered eight different large language models, ranging from massive, powerful systems to smaller, more efficient ones, and asked them to grade twenty different pieces of text. These texts covered four distinct areas: summarizing articles, reviewing computer code, writing creative stories, and answering factual questions. The researchers created three different scenarios for the AI judges. In the first scenario, the judge saw only the task and the text. In the second, the judge saw the text with a note saying it was a revision. In the third, the judge saw the text along with a note that included a specific number representing a "prior score" from a previous attempt. Crucially, these prior scores were randomly generated numbers that were always below the passing grade, meaning they were designed to be uninformative and irrelevant to the actual quality of the text.

The results were clear and consistent across most of the models tested. When the AI judges were shown the random prior score, their grading shifted significantly downward. Instead of treating the text as a standalone piece of work, the models seemed to anchor their judgment to that lower number. In the study, seven out of the eight models showed a statistically significant drop in scores when the prior score was present. The effect was not just a tiny fluctuation; for some models, the shift was large enough to change the outcome of the evaluation. For instance, in one of the most powerful models tested, the presence of a prior score caused the acceptance rate of the text to drop by nearly twenty-two percentage points. This means that content which would have been approved as good quality was suddenly rejected simply because the judge had seen a lower number in the background. The researchers found that this bias was not uniform across all types of tasks; it was strongest in creative writing and factual questions, while code review showed a weaker, more inconsistent reaction.

To understand how this happened inside the machine, the researchers looked at the specific words the models were considering as they generated their scores. They discovered a pattern that looked like a switch flipping rather than a gradual slide. When the prior score was introduced, the model's probability of choosing a high score dropped sharply, and the probability of choosing a lower score jumped up. However, changing the specific value of that prior score did not seem to matter much; the mere presence of the number triggered the shift. This suggests the models are not carefully calculating a new average but are reacting to the existence of the metadata itself. The researchers also tested whether simple instructions could fix the problem. They tried asking the models to think step-by-step before grading, and they tried explicitly warning the models to ignore the prior scores. Neither of these strategies worked. In fact, in some cases, the step-by-step instructions made the bias worse, and the explicit warning failed to stop the scores from dropping, even though it did reduce the models' tendency to follow the specific number if they were already biased.

The study went further to see if this bias affected real-world decisions, not just abstract numbers. Using a dataset of messaging campaigns that had already been labeled by humans as either safe or prohibited, the researchers tested how the AI judges performed when they were given a wrong prior label. When the AI was shown a prior label that was incorrect, it failed to correct the error in nearly half of the cases where it otherwise would have. Instead, it often stuck with the wrong label provided in the metadata. In a paired test where the AI had previously gotten the answer right, the introduction of a wrong prior label caused it to flip its correct judgment to the wrong one in over ten percent of cases. This demonstrates that the bias extends beyond simple scoring to categorical decisions that determine whether content is allowed or blocked. The researchers concluded that the only reliable way to prevent this bias is to remove the irrelevant metadata entirely from the prompt. They found that no amount of prompting or warning could fully restore the independence of the judge once that prior information was present.

The implications of these findings are significant for anyone relying on AI to make fair evaluations. The study shows that the assumption of impartiality in automated systems is fragile. Even when the information provided is clearly irrelevant, the models are not immune to the influence of context. The bias is not a bug that can be easily patched with a simple instruction; it appears to be a fundamental part of how these systems process information. The researchers emphasized that while their tests covered a specific set of models and tasks, the results suggest a broader vulnerability. To build trustworthy systems, developers must carefully engineer the context in which the AI operates, ensuring that no extraneous numbers or labels from previous steps leak into the current evaluation. The path forward requires active validation for every specific model and task, rather than a passive belief that the machine is seeing the world clearly on its own.