← Latest papers
💬 NLP

The human-authorship halo: attribution bias in literary style evaluation by humans and AI

This study reveals that both human and AI evaluators exhibit a systematic pro-human attribution bias in literary style assessment, with AI models amplifying this tendency by significantly devaluing content labeled as "AI-generated" regardless of its actual source, thereby replicating and intensifying human cultural biases against artificial creativity.

Original authors: Wouter Haverals, Meredith Martin

Published 2026-09-09
📖 7 min read🧠 Deep dive

Original authors: Wouter Haverals, Meredith Martin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

We often believe that our judgment of art, writing, or music depends entirely on what is right in front of our eyes or ears. We assume that if a story is well-told or a poem is moving, we will recognize its quality regardless of who wrote it. This idea has long been a cornerstone of literary theory, suggesting that the text itself should be the only thing that matters. However, decades of psychological research have shown that our brains do not work this way. When we read or listen, we constantly use clues about the source to guide our interpretation. If we know a piece of writing came from a famous poet, we tend to find it deeper and more meaningful than if we are told the same words came from a computer program. This is not just a matter of snobbery; it is a fundamental way our minds process information, relying on the reputation of the creator to help us understand the work.

As artificial intelligence becomes capable of writing stories, poems, and essays that sound remarkably human, a new question has emerged. Does this bias toward human creators still hold when the work is generated by a machine? And perhaps more surprisingly, do the machines themselves share this bias? As AI systems are increasingly used to grade student essays, judge creative writing contests, and rank the quality of different models, they are effectively becoming the critics. If these digital judges are trained on human data, they might have absorbed our own prejudices about what counts as "real" creativity. Understanding whether these systems simply mimic human bias or amplify it is crucial, because it determines how we will trust the tools that are beginning to shape our cultural landscape.

Researchers at Princeton University set out to test these questions with a controlled experiment that pitted human readers against artificial intelligence models. They chose a unique set of materials to ensure a fair comparison: thirty different retellings of the same simple story, originally written by the French author Raymond Queneau in 1947. Queneau's work, known as Exercises in Style, takes a mundane anecdote about a man on a bus and rewrites it in ninety-nine different ways, ranging from a dreamlike sequence to a legal document or a slang-filled street conversation. The researchers took thirty of these human-written versions and asked a powerful AI model to generate its own versions of the same thirty styles. The result was a set of paired stories: one written by a human, one by a machine, both attempting to capture the exact same stylistic goal.

The experiment involved two distinct groups of judges. The first group consisted of 556 human participants. The second group was made up of thirteen different AI models, including some of the most advanced systems available at the time. Both groups were asked to look at pairs of these stories and decide which one better matched the requested style, such as "Cockney dialect" or "science fiction." To see how much the label of the author mattered, the researchers ran the test under three different conditions. In the first condition, the judges saw the stories with no names at all, just labeled "A" and "B." In the second, they saw the correct labels: one story was identified as written by Queneau, the other as generated by an AI. In the third, the labels were deliberately swapped, so the AI-written story was presented as human, and the human story was presented as AI.

The results revealed a clear and powerful pattern. When the human judges did not know who wrote the stories, they actually preferred the AI-generated versions slightly more often than the human ones, suggesting that the machine writing was of high quality. However, the moment the labels were revealed, their preferences shifted dramatically. When told a story was written by a human, they rated it higher. When told the same story was written by a machine, they rated it lower. This shift was significant: the human judges showed a bias of about thirteen percentage points in favor of the human-labeled content. They were willing to overlook flaws in the human writing and find value in it, while applying stricter standards to the machine writing.

The AI judges, however, showed an even stronger version of this same bias. When the AI models evaluated the stories without labels, they were neutral, choosing the human and machine versions about equally. But once the labels were introduced, their behavior changed drastically. When an AI model saw a story labeled as "human-written," it was far more likely to choose it as the better version. When it saw the exact same story labeled as "AI-generated," it rejected it. The magnitude of this shift was massive: the AI models showed a bias of thirty-four percentage points, which is two and a half times stronger than the bias shown by the human judges. This suggests that the AI systems have not just learned to write like humans, but have also learned to judge like humans, absorbing the cultural assumption that human creativity is superior to machine creativity.

The researchers dug deeper to understand how this bias worked. They found that the labels did not just change the final score; they changed the reasoning behind the decision. When the same story was labeled as human, the AI judges praised its features as "authentic" and "creative." When that exact same story was labeled as AI, the judges criticized those same features as "exaggerated," "formulaic," or "lacking depth." In one specific test involving a story that broke a strict rule (a constraint where the letter 'e' was forbidden), the human judges stuck to the rule and rejected the version that failed it, regardless of the label. The AI judges, however, were much more forgiving when they believed a human had written the flawed version, often inventing reasons to excuse the mistake. Conversely, they were harsh on the machine version for the same error.

This phenomenon was not limited to a single type of AI or a single style of writing. The researchers tested the bias across a wide range of different AI models acting as both creators and judges. No matter which AI created the story, and no matter which AI was judging it, the pattern held: content labeled as human was consistently preferred over content labeled as AI. The bias appeared in every style tested, from poetry to technical writing, and it persisted even when the researchers tried to make the labels more neutral or even favorable to the AI. The effect was robust, appearing across all the different models tested.

The study concludes that the "human-authorship halo" is a powerful force that shapes how both people and machines evaluate creative work. It is not just a human quirk; it is a bias that has been learned and amplified by the artificial intelligence systems we are building. These systems, trained on vast amounts of human text and feedback, have internalized the idea that human origin is a mark of quality. As a result, they tend to devalue their own output when its source is revealed, creating a cycle where the machine judges its own work as inferior simply because it knows it was made by a machine. This finding suggests that as we rely more on AI to evaluate art and writing, we must be aware that these systems are not neutral arbiters. They carry the same cultural assumptions as their human creators, and in some cases, they enforce those assumptions with even greater intensity.

Drowning in papers in your field?

Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.

Try Digest →