Answer Bubbles: Information Exposure in AI-Mediated Search

1. Problem Statement

Generative search systems (e.g., Google AI Overviews, Search GPT) are shifting the information retrieval landscape from traditional link-based lists to AI-generated summaries. While these systems synthesize information on behalf of users, there is a critical lack of empirical understanding regarding:

• Source Selection: How these systems differ from traditional search and each other in selecting and citing sources.
• Linguistic Transformation: How the synthesis process alters the epistemic (uncertainty vs. confidence) and stylistic qualities of the information.
• Fidelity: How faithfully the generated summaries represent the original cited content.

The authors argue that these systems risk creating "Answer Bubbles": self-contained, system-specific information realities where identical queries yield structurally different answers, potentially obscuring source diversity and altering user trust without their knowledge.

2. Methodology

The study employs a large-scale comparative analysis across 11,000 real user search queries (drawn from Google's Natural Questions corpus) spanning 11 topical categories (e.g., Business, History, Sports).

Systems Analyzed:

1. Vanilla GPT: GPT-4o-mini without web search (parametric knowledge only).
2. Search GPT: GPT-4o-mini with web search enabled (search-grounded).
3. Google AI Overview (AIO): Google's generative summary powered by a Gemini model.
4. Traditional Google Search: Standard organic link results (used as a baseline for source comparison).

Three Research Questions (RQs) & Analytical Framework:

• RQ1 (Source Diversity): Analyzed citation volume, domain overlap (Jaccard similarity), and domain categorization (e.g., encyclopedic vs. social media) to map source pools.
• RQ2 (Linguistic Characterization): Used LIWC (Linguistic Inquiry and Word Count) for 73 categories (affect, cognition, analytical) and transformer-based models to measure politeness, formality, polarity, and subjectivity. Statistical significance was tested using Kruskal-Wallis H and Mann-Whitney U tests.
• RQ3 (Source-Summary Fidelity): Adapted a coverage-based fairness framework (Li et al., 2025) involving:
  • Atomic Content Unit (ACU) Decomposition: Breaking summaries into factual statements.
  • Entailment Scoring: Using RoBERTa-large to calculate the probability that source chunks entail the ACUs.
  • Metrics: Calculated Equal Coverage (EC) (how evenly sources are used) and Coverage Parity (CP) (systematic bias toward specific attributes like length, sentiment, or domain type).

3. Key Contributions

• First Cross-System Comparative Study: Provides a systematic, large-scale comparison of generative vs. traditional search across source selection, linguistic framing, and fidelity.
• Definition of "Answer Bubbles": Conceptualizes the phenomenon where different systems create distinct information realities for the same query, extending the concept of "filter bubbles" to the synthesis layer.
• Methodological Framework: Introduces a reproducible pipeline for auditing source-summary fidelity in Retrieval-Augmented Generation (RAG) systems, quantifying how much of a cited source is actually utilized in the final output.

4. Key Results

A. Source Selection Biases (RQ1)

• Distinct Source Pools: Search GPT's top-100 domains overlap only 24–25% with traditional Google Search.
• Encyclopedic vs. Social Media:
  • Search GPT heavily favors authoritative, editorially curated sources (Wikipedia, Britannica, Reuters, AP News) and nearly excludes social media (0.1% of citations).
  • Google AIO and Organic Search rely significantly more on user-generated platforms (Facebook, Quora, Reddit).
• Topic Dependence: Source preferences vary drastically by topic (e.g., IMDB dominates entertainment in Organic search but is rare in Search GPT).

B. Linguistic and Epistemic Reshaping (RQ2)

• Attenuation of Hedging: Search grounding significantly reduces uncertainty markers.
  • Hedging (e.g., "maybe," "perhaps") drops by 40% in Search GPT and 60% in Google AIO compared to Vanilla GPT.
  • Confidence markers are preserved, leading to a higher ratio of certainty-to-tentative language (0.49 in search systems vs. 0.39 in Vanilla GPT).
• Structural Differences:
  • Search GPT produces longer, more formal, and complex responses (paragraph-style).
  • Google AIO produces compressed, content-heavy summaries with shorter sentences and simpler vocabulary, optimized for quick consumption.
• Emotional Flattening: Google AIO significantly reduces positive emotion (44% drop) while preserving negative emotion, resulting in a more impersonal tone.

C. Fidelity and Synthesis Bias (RQ3)

• Uneven Coverage: Google AIO distributes attention less evenly across sources (higher EC) than Search GPT, despite citing more sources on average.
• Length Bias: Both systems exhibit a strong bias toward longer documents. Short sources (<200 words) are under-represented by ~13–17%, while long sources are over-represented.
• The "Citation-Synthesis Gap":
  • Wikipedia is the most cited domain and is also over-represented in the actual text of summaries (content weight > citation count).
  • Social Media/Forums: While Google AIO cites Reddit/Quora in ~8% of queries, it draws 22.1% less content from these sources compared to others. The system lists them for appearance but synthesizes little from them.
• Sentiment Bias: Google AIO systematically under-covers negative-sentiment sources (13.8% under-representation) and over-covers positive ones.
• Epistemic Bias: Sources containing causal language ("because") and certainty markers are over-covered, while highly subjective sources are under-covered.

5. Significance and Implications

• Epistemic Reshaping: Generative search systems do not just summarize; they actively reconstruct information by filtering out uncertainty and subjective nuance. This creates a "confidence bias" where users may perceive uncertain information as fact due to the lack of hedging.
• Source Visibility & Economics: The compounding bias (citing diverse sources but synthesizing only from encyclopedic/long texts) threatens the visibility and economic viability of community-driven platforms and smaller publishers.
• Transparency Needs: The existence of "Answer Bubbles" implies that users are unaware they are receiving system-specific realities. The authors call for transparency requirements (e.g., source diversity indicators, confidence signals) similar to those for algorithmic recommendations.
• Research Direction: The study provides a framework for auditing RAG systems, highlighting that biases introduced at the retrieval stage are often amplified, not mitigated, during the generation/synthesis stage.

In conclusion, the paper demonstrates that AI-mediated search creates distinct, opaque information ecosystems that prioritize certain source types and linguistic styles, fundamentally altering how the public accesses and perceives information.