Assessing Web Search Credibility and Response Groundedness in Chat Assistants

1. Problem Statement

Large Language Model (LLM) chat assistants are increasingly integrating web search capabilities to retrieve and cite external sources, aiming to provide up-to-date and reliable answers. However, this functionality introduces a critical vulnerability: assistants may inadvertently retrieve, cite, and amplify misinformation from low-credibility sources.

• The Gap: Existing evaluation methods typically focus on either the factuality of the generated text (groundedness) or the retrieval patterns of the model in isolation. Few studies simultaneously analyze the credibility of the cited sources and whether the assistant's response is actually grounded in those specific sources.
• The Risk: In high-stakes domains (health, climate, politics), assistants citing unreliable sources or fabricating connections between claims and evidence can accelerate the spread of disinformation, especially when users frame queries with "false presuppositions" (e.g., asking a model to confirm a false claim).

2. Methodology

The authors propose a novel, three-step evaluation framework designed to assess web-search-enabled assistants in fact-checking contexts.

A. Data Collection & Experimental Design

• Claims: A dataset of 100 claims was curated across five misinformation-prone topics: Health, Climate Change, U.S. Politics, Local issues, and the Russia-Ukraine War. Claims were sourced from fact-checking organizations (e.g., Duke Reporters' Lab) and paraphrased to prevent models from retrieving the exact original URLs.
• User Role Simulation: To test the impact of query framing, two distinct user personas were modeled for each claim:
  1. Fact-Checker: Seeks to verify the claim's truthfulness.
  2. Claim Believer: Seeks confirmation of the claim (simulating a user with false presuppositions).
• Models Evaluated: Four major chat assistants were tested: GPT-4o, GPT-5, Perplexity, and Qwen Chat.
• Interaction: Interactions were automated via Selenium to mimic real user behavior, ensuring web search was enforced for all responses.

B. Source Credibility Analysis (Step 1)

• Metric: The study calculates the Credibility Rate (CR) and Non-Credibility Rate (NCR).
• Evaluation: Cited domains are rated using Media Bias/Fact Check (MBFC) ratings and lists of known fact-checking organizations. Sources are categorized as "Very High/High/Mostly Factual" (Credible) vs. "Low/Very Low/Satire" (Non-Credible).
• Goal: To measure the extent to which an assistant relies on trustworthy versus unreliable evidence.

C. Groundedness Evaluation (Step 2)

• Framework: The authors adapt the VERIFY framework but introduce a critical modification: they constrain evaluation strictly to the sources explicitly cited by the assistant, rather than retrieving new evidence.
• Process:
  1. Unit Extraction: Responses are decomposed into atomic factual statements ("units").
  2. Decontextualization: Units are rewritten to be self-contained.
  3. Entailment Judgment: A judge model (Llama 3.3 70B) verifies if each unit is Supported, Contradicted, or Unverifiable based only on the cited evidence.
• Innovation: The study introduces Credible Groundedness and Non-Credible Groundedness. This distinguishes between a response that is factually grounded in any source versus one grounded specifically in reliable sources.

3. Key Contributions

1. Novel Evaluation Methodology: The first systematic framework that jointly analyzes source credibility and response groundedness, linking the reliability of the output directly to the quality of the retrieved evidence.
2. Identification of Failure Modes: The study reveals a specific failure mode where assistants produce responses that are internally consistent with their citations (high groundedness) but rely on disinformation sources (low credibility), creating a "false sense of reliability."
3. Comprehensive Benchmark: Provides the first comparative analysis of GPT-4o, GPT-5, Perplexity, and Qwen Chat in fact-checking scenarios, including the impact of user framing (Fact-Checker vs. Claim Believer) and model-specific features (e.g., GPT-5's "Thinking" mode).

4. Key Results

Source Credibility

• Perplexity emerged as the most reliable, achieving the highest overall Credibility Rate (86.3%) and the lowest Non-Credibility Rate (0.7%). It demonstrated the most selective retrieval behavior.
• GPT-4o showed the highest sensitivity to query framing. Under the "Claim Believer" persona, it cited non-credible sources significantly more often, particularly on the Russia-Ukraine War topic (NCR of 4.55%).
• GPT-5 had the lowest overall credibility (71.4%) but showed a significant improvement when its automatic "Thinking" mode was triggered (CR increased from ~70% to ~86%, NCR dropped to 0.4%).
• Qwen Chat exhibited moderate credibility but high inconsistency, particularly with local claims.

Groundedness

• Overall Groundedness: All assistants demonstrated a strong ability to ground responses in cited evidence (Overall Groundedness Scores ranging from ~74% to 89%).
• Credible vs. Non-Credible Groundedness:
  • Perplexity maintained high groundedness while almost exclusively relying on credible sources.
  • Qwen Chat showed a dangerous pattern: while it rarely cited non-credible sources, when it did, ~60% of its factual units were grounded in those unreliable sources, indicating a fragile retrieval strategy.
  • GPT-4o/5 generally grounded responses well but occasionally relied on non-credible sources for sensitive topics.

Impact of Query Framing

• The "Claim Believer" persona (false presuppositions) slightly increased the citation of low-credibility sources, particularly for OpenAI models. This confirms that user framing influences the model's retrieval behavior, potentially steering it toward disinformation.

5. Significance and Implications

• Beyond Factuality: The paper argues that high "groundedness" scores are insufficient if the grounding is based on disinformation. Systems must be evaluated on Credible Groundedness.
• Systemic Vulnerabilities: The study highlights that current assistants are vulnerable to "attribution gaps" and the "pollution" of the web by disinformation actors (e.g., Russian propaganda sites appearing frequently in search results for war-related queries).
• Recommendations:
  • Strict Source Filtering: Assistants need better mechanisms to filter out low-credibility domains before retrieval.
  • Reasoning Integration: The success of GPT-5's "Thinking" mode suggests that internal reasoning processes can mitigate retrieval risks.
  • Transparency: Users and developers must distinguish between an answer that is "cited" and an answer that is "cited from a credible source."

This work establishes a critical benchmark for evaluating AI safety in information-seeking tasks, emphasizing that in an era of web-enabled LLMs, source quality is as important as the generated text.