1+1 ̸ = 2: Appearance and utterance to determine the virtual agent’s intellectual impression
This study reveals that virtual agents are perceived as more intelligent and trustworthy when their appearance and speech contradict each other (e.g., looking intelligent but speaking unintelligently, or vice versa) rather than when both factors consistently convey intelligence.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the growing world of digital assistants, we often ask a simple question: what makes a computer program seem smart? For years, designers and researchers have operated on a straightforward assumption. If you want a virtual character to feel intelligent, you should give it a look that suggests brains—perhaps a sharp, serious face or a sleek, modern design—and pair that with speech that sounds knowledgeable, using precise facts and expert vocabulary. It seems logical that combining a smart-looking face with smart-sounding words would create the most convincing expert. This idea rests on the belief that human perception works like a simple addition problem, where the total impression is just the sum of its parts. However, human judgment is rarely that linear. We know from psychology that people do not always make rational choices; sometimes, our brains react strangely to contradictions or unexpected combinations, leading to conclusions that defy simple math. Understanding how we perceive artificial intelligence is crucial because these digital agents are becoming common in schools, hospitals, and businesses. If we want people to trust them, we need to know what actually works, rather than relying on what seems obvious.
A team of researchers from universities in Japan set out to test this common-sense idea. They wanted to see if mixing "intelligent" and "unintelligent" traits in a virtual agent would actually make it seem smarter, or if the traditional approach of stacking smart features together was the best path. To do this, they created a series of short videos featuring a female virtual agent recommending a trip to a famous castle in Japan. Before filming the main test, the researchers had to decide what a "smart" and a "dumb" look actually meant to people. They built five different versions of the same character, changing only the hairstyle for each one. They showed these images to nearly ninety people online and asked them to pick which character looked more intelligent. The results were clear: a character with short, neat hair was consistently rated as the smartest, while a character with pigtails was rated as the least intelligent. These two specific looks became the visual anchors for the main experiment.
Next, the researchers prepared two types of scripts for the agent to speak. One script was written in a style that sounded knowledgeable and professional, while the other was written to sound less informed and less capable. They then combined these elements into four distinct scenarios. In the first, the agent had the smart look and spoke smartly. In the second, the smart-looking agent spoke in the unintelligent style. In the third, the agent with the unintelligent look spoke in the intelligent style. In the fourth, both the look and the speech were unintelligent. Over 227 people watched these videos and rated how intelligent and trustworthy they found the character. The researchers expected the first scenario—the one with the smart look and smart speech—to win by a wide margin.
The results, however, turned the expected outcome on its head. The agent that combined a smart look with smart speech was not rated as the most intelligent or trustworthy. Instead, the characters that mixed the two styles performed better. The agent that looked smart but spoke in a simple, unintelligent way, and the agent that looked simple but spoke with expert knowledge, both received higher scores than the perfectly consistent, all-smart character. The data showed a clear statistical difference: when the agent's speech was unintelligent, the smart-looking version was trusted more than the simple-looking version. However, when the speech was intelligent, there was no statistically significant difference in ratings between the simple-looking agent and the smart-looking agent, even though the mixed-style agents generally outperformed the consistent "all-smart" agent overall. The most consistent, "perfect" agent actually came in last. The researchers found that the combination of conflicting traits created a more positive impression than a consistent one.
To understand why this happened, the authors turned to a concept called natural-born intelligence theory. This idea suggests that when we encounter a contradiction—like a serious face speaking in a silly way—our brains do not just add the two impressions together. Instead, the conflict forces us to look beyond the surface. The mismatch between what we see and what we hear breaks our usual expectations, creating a kind of mental "outside" space where we start to imagine the character's personality or familiarity. It is as if the contradiction makes the agent feel more real and less like a rigid machine. When the agent was perfectly consistent, the viewer simply processed the information without this extra layer of engagement. The study suggests that for virtual agents, a little bit of friction between appearance and speech might be more effective at building trust than a perfectly polished, consistent image.
The researchers noted that these findings apply specifically to the female human-like agents they tested in a travel recommendation context. They acknowledged that the results might differ if the agents were male, looked like robots, or were used in different situations. Nevertheless, the study challenges the standard rule of design. It shows that trying to make an agent look and sound perfectly intelligent at the same time does not necessarily make people trust it more. In fact, introducing a small contradiction can make the agent seem more capable and reliable, proving that human perception of intelligence is a complex process that cannot be solved by simply adding up the right features.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.