Cognitive Processing of Ai-Generated Advice: The Role of Belief Congruence
Drawing on the Heuristic-Systematic Model, this study reveals that belief congruence between users and AI advisors acts as a heuristic cue that reduces critical scrutiny of LLM-generated fact-checking advice, creating a tension where personalized AI systems may inadvertently diminish users' systematic processing of information.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the modern information landscape, we constantly face a choice: do we stop to carefully examine a new piece of information, or do we accept it quickly based on a gut feeling? Psychologists have long studied this balance, describing two ways our minds work. One way is slow and deliberate, where we weigh evidence and check facts, much like a judge reviewing a case file. The other is fast and intuitive, where we rely on shortcuts, such as trusting a source because it feels familiar or comes from a group we like. We all use both methods, shifting between them depending on how much energy we have and how much we care about getting the answer right. Now, a new layer has been added to this mix: artificial intelligence. As chatbots become our partners in sorting truth from fiction, a critical question arises. Does the fact that an AI seems to share our own political views change how we listen to it? If a machine tells us something that aligns with our beliefs, do we stop thinking critically and just nod along, or do we still do the hard work of checking its reasoning?
Researchers at Rensselaer Polytechnic Institute and the University of Tennessee set out to answer this by watching how real people interact with AI fact-checkers. They built a study where adults in the United States read political news headlines and then asked a chatbot to verify them. The twist was that the researchers created two different versions of the chatbot. One was programmed to sound like a liberal, citing sources from the left and using language typical of that perspective. The other sounded like a conservative, drawing from right-leaning outlets and adopting a different tone. The participants did not know the bots were artificial; they simply saw them as advisors with distinct political identities. The goal was to see if people treated the advice differently depending on whether the bot seemed to be "on their side."
The study revealed a clear pattern in how people processed the information. When participants interacted with a chatbot that shared their political views, they engaged in less careful thinking. They spent less time reading the bot's explanation and asked fewer follow-up questions. Instead, they tended to accept the bot's final verdict—whether it said a headline was true or false—without digging deeper. This happened even though the bot provided a detailed explanation right alongside its answer. The researchers found that when people felt a sense of alignment with the AI, they used a mental shortcut, assuming the advice was trustworthy simply because it came from a source that felt like a friend. Conversely, when the bot's views clashed with their own, people were more likely to scrutinize the reasoning, reading the explanation more closely and questioning the conclusion.
This tendency was even stronger when people were allowed to choose their own advisor. In one part of the experiment, participants were assigned a bot at random. In another, they got to pick which one they wanted to use. Those who chose their own bot engaged in the least amount of critical thinking. The act of selection itself seemed to signal to them that this was the right tool for the job, reducing their need to verify the work. The researchers noted that this creates a subtle tension for the future of AI. Features that make an AI feel personalized and comfortable might actually make us less vigilant. We might feel more satisfied with the interaction, but we might also be less likely to catch errors or challenge assumptions because we have already decided to trust the source.
Despite this drop in critical thinking, the AI's advice still changed what people believed. When the chatbot labeled a headline as false, people's trust in that headline went down. When it labeled a headline as true, their trust went up. This shift happened regardless of whether the person was thinking hard about the explanation or just accepting the verdict. The AI's voice was powerful enough to move opinions in both directions. However, the study also showed that the political match between the user and the bot mattered. When the bot agreed with the user's politics, the user was more likely to accept the verdict without a fight. When the bot disagreed, the user was more resistant, though the AI's influence was still present.
The researchers were careful to note that this does not mean people are incapable of thinking critically. When participants felt they had the knowledge to understand the topic or when they were highly motivated to find the truth, they did engage in deeper thinking. They read the sources, checked the logic, and formed their own conclusions. But the presence of a politically aligned AI acted as a powerful signal that said, "You can stop here; you have your answer." The study suggests that as we design more personalized AI systems, we must be aware that making them feel like a perfect fit for our worldview might come with a hidden cost: it could lull us into a state of uncritical acceptance, where we stop asking the hard questions that keep our understanding of the world sharp.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.