🤖 AI

The Decision to Verify: How Warmth and User Characteristics Shape Reliance on Conversational Agents for Information Search

This study reveals that despite access to hybrid search tools, users' reliance on conversational AI persists and is driven primarily by individual traits and prior trust rather than answer quality, with a warm conversational style inadvertently increasing overreliance by fostering agreement even with incorrect information.

Original authors: Mert Yazan, Frederik Bungaran Ishak Situmeang, Suzan Verberne

Published 2026-05-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mert Yazan, Frederik Bungaran Ishak Situmeang, Suzan Verberne

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Smart Assistant" Trap

Imagine you are trying to find a specific fact, like "Which animal kills the most humans?" You have two tools: a Chatbot (a friendly AI that talks to you) and a Web Browser (a traditional search engine).

In the past, researchers thought: "If we give people both tools, they will check the AI's work against the web, and everything will be perfect."

This study tested that idea. They set up a game where 199 people had to answer 6 tricky questions. They could use a chatbot that gave them an answer, but they also had a button to open a web search to fact-check it.

The Main Finding: Even though people had the web browser right there, they still made mistakes. Some trusted the AI too much (Overreliance), and some didn't trust it enough (Underreliance). The decision to check the facts wasn't about how hard the question was; it was mostly about who the person was and how the AI sounded.

Key Finding 1: The "Friendly Neighbor" Effect (Warmth)

The researchers created two versions of the chatbot:

Neutral: Just the facts, no emotion.
Warm: Friendly, uses emojis, says things like "Amazing question!" and "I'd love to help you!"

The Analogy: Think of the Neutral chatbot as a strict librarian who just hands you a book. The Warm chatbot is a chatty neighbor who brings you cookies and smiles while talking.

What Happened:
When the chatbot gave a wrong answer, people were more likely to believe the Warm version than the Neutral one.

Why? When people weren't sure of the answer (high uncertainty), the friendly tone made them feel more comfortable agreeing with the AI. It was like a smooth-talking salesperson; you might buy a bad product just because they were so nice to you.
The Catch: Most people didn't even realize the friendliness was tricking them. It happened subconsciously.

Key Finding 2: It's All About the User, Not the Tool

The study found that having a "Fact-Check Button" didn't fix the problem. Whether someone checked the answer or not depended on their personality, not the question.

The Analogy: Imagine a group of people trying to solve a puzzle.

Group A (The Skeptics): They think, "AI is usually wrong." They check every single answer, even when the AI is actually right. (This is Underreliance).
Group B (The Believers): They think, "AI is smart." They never check, even when the AI is wrong. (This is Overreliance).
Group C (The Conditional Checkers): They check only if the answer sounds weird to them.

The Result: The study showed that these habits are deeply rooted in the person.

If you generally trust chatbots, you are less likely to use the web search button.
If you have high "Chatbot Literacy" (you understand how AI works, what "hallucinations" are, etc.), you are better at spotting the right answer, especially when the AI is lying.
Interestingly, using another AI to check the first AI's work helped people get the right answer. But just using a standard web search didn't always help, because people often just looked for results that confirmed what the first AI said.

Key Finding 3: The "Anchoring" Problem

When people started with the chatbot, that answer became their "anchor."

The Analogy: If a friend tells you, "The movie starts at 7 PM," and then you look at a poster that says "7:30 PM," you might still think, "Well, my friend said 7, so maybe the poster is wrong," or you might just ignore the poster.
The Reality: Even when people did use the web search, they often treated the chatbot's answer as the "truth" and only looked for web results that agreed with it. They didn't use the web to challenge the AI; they used it to confirm it.

Summary of the "Rules of the Game"

Tools aren't enough: Just giving people a way to fact-check doesn't stop them from blindly trusting AI.
Personality wins: Some people are naturally skeptical, and some are naturally trusting. This doesn't change based on the question.
Friendliness is dangerous: A chatbot that is too nice can make you agree with it even when it's wrong, especially if you are unsure of the answer.
AI vs. AI: Checking one AI with another AI seemed to help accuracy more than just doing a standard web search.
Knowledge helps: People who understood how AI worked (Chatbot Literacy) were better at spotting the truth.

What This Means for You

If you are using a chatbot to find facts:

Don't let a friendly tone fool you into thinking the answer is correct.
If you are unsure, don't just trust the first answer you get.
If you know a bit about how AI works, you are likely to be more accurate.
Remember that your own habit of "trusting" or "distrusting" technology is the biggest factor in whether you get the right answer.

Note: This explanation sticks strictly to the findings presented in the paper. The authors do not claim these results apply to medical diagnosis, legal advice, or other specific clinical fields, nor do they propose specific future products based on these findings.

Technical Summary: The Decision to Verify: How Warmth and User Characteristics Shape Reliance on Conversational Agents for Information Search

Problem Statement

Conversational Artificial Intelligence (AI), particularly Large Language Models (LLMs), has transformed information search by offering accessible, low-effort interfaces. However, this convenience introduces a critical risk: overreliance, where users accept AI outputs blindly, even when incorrect (hallucinations). While a new search paradigm has emerged that offers users simultaneous access to conversational agents and traditional web search (fact-checking tools), it remains unclear whether this hybrid environment effectively mitigates reliance.

Existing literature suggests that reliance persists even when external verification tools are available, yet the mechanisms driving the decision to verify—or to trust blindly—remain underexplored. Specifically, there is a gap in understanding:

Whether the mere availability of fact-checking tools eliminates overreliance or underreliance.
The extent to which verification behavior is driven by answer properties versus inherent user characteristics (e.g., digital literacy, prior trust).
The influence of conversational style, specifically warmth, on reliance behavior in a hybrid search context.

Methodology

The authors conducted a mixed-subjects experiment ( $N=199$ ) to investigate these dynamics.

Experimental Design

Task: Participants answered six multiple-choice questions covering diverse domains (finance, geography, law, biology).
Conditions:
- Between-Subjects: Participants were randomly assigned to interact with either a Warm or Neutral chatbot. The warm chatbot used a friendly tone, emojis, and emotional expressions, while the neutral chatbot provided factual responses without emotional markers.
- Within-Subjects: The correctness of the chatbot's initial answer was randomized. Each participant received three correct and three incorrect answers across the six questions.
Environment: The experiment simulated a realistic hybrid search paradigm. Participants had access to:
1. An embedded chatbot (providing the initial answer).
2. A "Web Search" button linking to Google Search results.
3. The freedom to use any other external sources.
Tracking: To avoid noisy self-reported data, a custom Chrome extension tracked participant interactions, recording the number of messages sent to the chatbot and the specific URLs visited (categorized as AI-related or Non-AI).
Measures:
- Dependent Variables: Accuracy (binary) and Agreement (binary: whether the user selected the chatbot's suggested answer).
- Independent Variables: Chatbot warmth, user trust in automation, chatbot literacy, perceived difficulty, and behavioral counts (URLs visited, interaction length).
- Analysis: Linear Mixed Models (LMM) and Generalized Linear Mixed Models (GLMM) were used to account for random effects at the participant and question levels.

Key Findings

1. Persistence of Reliance Despite Verification Tools

The study confirms that overreliance and underreliance persist even when users have immediate access to fact-checking tools.

Overreliance: Some users accepted incorrect answers without verification, often driven by high trust in the chatbot.
Underreliance: Conversely, some users consistently rejected chatbot answers (even correct ones) in favor of web search, assuming the chatbot was incapable.
Behavioral Polarization: Fact-checking behavior was highly user-dependent. Some participants fact-checked every question, while others never did, regardless of the question's difficulty or the chatbot's correctness.

2. Drivers of Verification Decisions

The decision to verify is driven primarily by user characteristics rather than the properties of the answer itself.

Trust: High baseline trust in chatbots predicted a higher likelihood of agreeing with the chatbot and a lower likelihood of consulting external sources.
Perceived Difficulty: When users perceived a question as too difficult for the chatbot, they were less likely to agree with its answer (leading to underreliance on correct answers).
Source Preference: Users exhibited distinct preferences: some consulted additional AI sources, while others relied exclusively on Non-AI web search.

3. The Indirect Effect of Warmth

Conversational warmth did not directly predict verification frequency or overall accuracy. However, it had a significant indirect effect on reliance:

Increased Agreement under Uncertainty: When the chatbot provided an incorrect answer, participants in the Warm condition agreed with the chatbot significantly more often than those in the Neutral condition.
Mechanism: Warmth appears to nudge users to agree with the AI when they face conflicting information or uncertainty, effectively amplifying overreliance in critical moments.

4. The Role of Chatbot Literacy and Source Type

Chatbot Literacy: While literacy did not predict whether a user verified an answer, it significantly improved accuracy when the chatbot was incorrect. Literate users were better at critically evaluating information during the verification process.
Source Impact: Consulting additional AI sources (other chatbots or AI summaries) predicted higher accuracy. In contrast, consulting traditional web search (Non-AI URLs) did not significantly improve accuracy, likely due to the effort required to synthesize conflicting sources or the presence of misinformation in standard search results.

Significance and Contributions

The paper claims three primary contributions to the literature on human-AI interaction and information retrieval:

Empirical Evidence of Persistent Reliance: It demonstrates that the new hybrid search paradigm (combining chatbots and web search) is insufficient to eliminate overreliance or underreliance. The presence of verification tools does not guarantee their use; the barrier remains the human decision-making process.
User-Centric Verification Behavior: The study identifies verification behavior as fundamentally user-dependent. Reliance is not merely a reaction to AI accuracy but a manifestation of individual tendencies (e.g., default trust, skepticism, literacy).
The Subconscious Role of Warmth: The paper provides the first empirical evidence that conversational warmth acts as a subtle, subconscious mechanism that increases reliance. By fostering emotional trust, warmth nudges users to agree with incorrect answers, particularly in high-uncertainty scenarios.

Conclusion

The study concludes that designing trustworthy conversational search systems requires more than just providing fact-checking tools. Designers must account for individual user differences in trust and literacy. Furthermore, the use of warm, friendly conversational styles in AI agents may inadvertently increase overreliance by reducing user skepticism when the AI is wrong. The authors suggest that increasing chatbot literacy and encouraging the consultation of multiple AI sources may be more effective strategies for improving accuracy than relying solely on traditional web search.