🤖 AI

Position: Generative Engine Optimization Creates Underexamined Risks, Governance Must Target Concentration, Disclosure, and Academic Blind Spots

This position paper argues that the shift from traditional search to generative answer engines introduces underexamined risks of concentrated influence, undisclosed commercial bias, and academic-industry blind spots, necessitating new governance focused on contestability, disclosure, and deployment-aligned auditing.

Original authors: Yizhu Wen, Nan Zhang, Haohan Yuan, Xun Chen, Haopeng Zhang, Hanqing Guo

Published 2026-06-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yizhu Wen, Nan Zhang, Haohan Yuan, Xun Chen, Haopeng Zhang, Hanqing Guo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: From "Search Lists" to "Smart Summaries"

Imagine you used to ask a librarian, "What are the best books on gardening?" The librarian would hand you a stack of cards (a search engine results page). You would look through the stack, see which ones had "Sponsored" stickers, and decide for yourself what to read.

Now, imagine the librarian has become a super-smart AI assistant. Instead of giving you a stack of cards, the AI reads a few books, thinks about it, and hands you a single, perfect summary with three specific book recommendations. You trust this summary because it sounds so confident and helpful.

This shift is the core of the paper. We are moving from looking at lists (traditional search) to trusting synthesized answers (Generative AI).

What is "GEO"? (The New Game)

In the old days, companies played SEO (Search Engine Optimization). They tried to make their website look good so it would appear at the top of that stack of cards.

Now, companies are playing GEO (Generative Engine Optimization). They aren't just trying to get on the list; they are trying to hack the AI's brain. They want to make sure that when the AI reads its "evidence," it picks their book and puts it in the final summary it gives you.

The paper argues that a new industry has sprung up to do this for a fee. Companies like AirOps and ProFound sell services to help brands get mentioned by AI.

The Three Hidden Dangers

The authors say this new system creates three specific risks that we haven't fully thought about yet:

1. The "Tipping Point" Problem (Concentrated Influence)

The Analogy: Imagine a massive dam holding back a river. If you push the dam just a tiny, almost invisible amount, the whole river might suddenly change direction and flood a different town.
The Risk: AI answer engines are very sensitive. A tiny change in the "evidence" the AI finds (like adding a few specific words to a blog post) can cause the AI to completely switch its recommendation. Because millions of people use the same AI, a small trick by one company can suddenly make everyone see the same recommendation, drowning out all other options. The system is so fragile that small changes have huge, concentrated effects.

2. The "Invisible Ad" Problem (Undisclosed Commercial Influence)

The Analogy: Imagine you ask a friend, "What's a good restaurant?" and they say, "I heard this place is great." You think it's a genuine opinion. But actually, your friend was paid $50 to say that, and they didn't tell you.
The Risk: In traditional search, ads are clearly labeled "Sponsored." But with GEO, the "ad" is hidden inside the evidence the AI uses. The AI might say, "The best moisturizer is Brand X," because it read a blog post that was secretly written by Brand X to trick the AI. The AI thinks it's giving neutral advice, but it's actually reading a covert advertisement. The line between "helpful fact" and "paid promotion" has vanished.

3. The "Lab vs. Reality" Blind Spot (Academic-Industry Gap)

The Analogy: Imagine scientists testing a new car in a quiet, empty parking lot (Academia). They measure how fast it goes and how well the brakes work. Meanwhile, the car company is driving that same car in a chaotic, rainy city with traffic jams (Industry).
The Risk:

Academia (The Lab): Researchers study GEO using fake, static data in a controlled computer lab. They check if the AI picks the right answer in a test.
Industry (The City): Real GEO companies operate on the live, messy internet. They constantly change their tactics based on how the AI actually behaves in the real world.
The Blind Spot: Because researchers are looking at the "parking lot" version, they miss the real dangers happening in the "city." They don't see how companies manipulate the AI over time, or how the AI's behavior changes when it's actually being used by millions of people.

What Does the Paper Want Us To Do?

The authors aren't saying we should ban AI or GEO. They are saying we need new rules to fix these specific problems:

Show Your Work: AI should be forced to show us why it chose a specific answer. If it recommends a product, we should be able to see the list of evidence it used and know if that evidence was paid for.
Clear Labels: If an answer is influenced by a commercial deal, the AI should put a clear "Sponsored" or "Paid" label right on the answer, not just on a link.
Real-World Testing: Researchers need to stop testing only in labs. They need to audit the AI while it's actually running in the real world to see how it really behaves.
Black-Box Auditing: Regulators and independent groups need the power to "poke" the AI with questions to see if it's being manipulated, even if they can't see the AI's internal code.

Summary

The paper warns that as we rely more on AI to give us answers, a new kind of "invisible advertising" is emerging. Companies are quietly training the AI to recommend their products, and because the AI hides its sources, we don't know it's happening. The authors call for better transparency, clearer labels, and real-world testing to keep the system fair and trustworthy.

Technical Summary: Generative Engine Optimization Creates Underexamined Risks

1. Problem Statement

The paper addresses the emergence of Generative Engine Optimization (GEO), a practice where entities optimize content to influence the synthesized answers generated by Large Language Model (LLM) answer engines. Unlike traditional Search Engine Optimization (SEO), which targets ranked lists of links, GEO targets the evidence pool and the generation process of Retrieval-Augmented Generation (RAG) systems.

The authors identify a critical gap: existing governance frameworks and academic evaluation metrics are designed for traditional search or static RAG benchmarks. They fail to account for the unique dynamics of LLM answer engines, where visibility is determined by synthesized answers rather than ranked lists. This opacity creates three underexamined risks:

Concentrated Influence: Small changes in retrieval can redirect attention at scale due to system sensitivity and low contestability.
Undisclosed Commercial Influence: Commercial promotion is embedded within retrieved evidence and model reasoning rather than labeled as advertising.
Academic–Industry Blind Spots: A divergence between offline academic evaluations and deployed industry practices leads to a failure in detecting real-world optimization effects.

2. Methodology

The paper employs a formalization and comparative analysis approach:

2.1. Formalization of the GEO Pipeline

The authors define a generalized GEO pipeline consisting of three blocks:

LLM Block: Generates recommendations based on user queries.
Search Flow Block: Retrieves evidence from a pre-indexed corpus (via crawling/indexing) using candidate retrieval (e.g., keyword matching) and fine-grained ranking (e.g., embedding similarity) to select top- $k$ documents as context.
GEO Block: Distributes optimized content to influence the retrieval and ranking. This involves two mechanisms:
- Retrieval Booster Messages ( $b$ ): Content designed to increase semantic similarity with potential user queries to ensure inclusion in the candidate set.
- Ranking Shifter Messages ( $c$ ): Content designed to influence the LLM's ranking or framing of the target topic once included in the context.

The authors model GEO as a joint optimization problem:

Objective: Maximize the utility $U(q, t; C(q))$ , where $t$ is the target topic, $q$ is the query, and $C(q)$ is the top- $k$ context.
Constraints: Content length and semantic similarity to user queries.

2.2. Comparative Analysis

The paper contrasts Academic and Industry GEO practices across several dimensions:

Assumptions: Academia often assumes optimized content is already in the candidate retrieval set (focusing only on ranking shifters), whereas industry operates in dynamic environments requiring joint optimization of retrieval boosters and ranking shifters.
Optimization Methods: Academia uses techniques like Greedy Coordinate Gradient (GCG), Tree of Attacks with Pruning (TAP), and LLM-based rewriting on static datasets. Industry uses LLM-guided content generation, query coverage expansion, and multi-platform distribution (e.g., Reddit, forums) on live systems.
Evaluation: Academia relies on offline metrics (Recall@k, nDCG@k) on synthetic or static corpora. Industry uses online, outcome-oriented metrics (answer visibility, citation frequency, ranking position) derived from repeated queries on deployed systems.

2.3. Risk Identification via Asymmetry

By analyzing the divergence between academic models and industry realities, the authors identify specific risk vectors. They also cite a small-scale sensitivity test (Appendix B) demonstrating that minor wording changes in queries can lead to entirely different citation sets in deployed models (e.g., Gemini), highlighting system-level sensitivity.

3. Key Contributions

Formalization of GEO: The paper provides the first generalized pipeline formalizing GEO as a joint optimization of retrievability and ranking impact, distinguishing between retrieval boosters and ranking shifters.
Identification of Three Risk Clusters:
- Concentrated Influence: The paper argues that opaque pipelines and "algorithmic monoculture" mean small retrieval changes can cause ecosystem-wide shifts in visibility, undermining user contestability.
- Undisclosed Commercial Influence: It highlights how GEO bypasses traditional advertising disclosure (e.g., FTC "Ad" labels) by embedding commercial intent into neutral-looking evidence and reasoning, creating an adverse selection dynamic.
- Academic–Industry Blind Spots: The paper demonstrates that offline benchmarks fail to capture cross-platform content injection, query coverage expansion, and the persistence of exposure in live systems.
Governance Framework: The authors propose a set of governance and auditing actions, including:
- Answer-Level Disclosure: Requiring labels when cited evidence reflects material commercial connections.
- Black-Box Auditing: Implementing repeated querying and logging to measure exposure persistence and sensitivity.
- Evaluation Reform: Updating benchmarks to include deployment-aligned metrics like citation share and time-varying exposure.

4. Results and Observations

Behavioral Shift: Data indicates a rapid shift from traditional search to LLM answer engines for information seeking and product research (e.g., 60% of US adults use AI for information finding).
Divergence in Practice: Industry GEO systems are more sophisticated than academic models, utilizing dynamic, multi-platform strategies that academic static benchmarks cannot replicate.
Sensitivity: Empirical observation suggests that LLM answer engines are highly sensitive to query paraphrasing, with different models (and versions) retrieving entirely different evidence for semantically equivalent queries.
Market Reality: A commercial GEO market exists (e.g., AirOps, ProFound), with significant funding, yet operates with limited transparency regarding implementation details.

5. Significance and Claims

The paper positions itself as a call for answer-level governance and measurement. Its significance lies in:

Reframing the Risk: Moving the focus from "ranking manipulation" (SEO) to "synthesis manipulation" (GEO), where the risk is the embedding of commercial influence into the reasoning process itself.
Bridging the Gap: Highlighting that current academic research is insufficient for understanding real-world GEO risks due to the "offline vs. deployed" asymmetry.
Policy Implications: Arguing that existing transparency norms (like link attribution) are inadequate for LLMs. The paper claims that without specific interventions—such as high-precision disclosure of commercial influence in synthesized answers and black-box auditing of material influence—users and regulators cannot effectively contest or understand the sources of LLM recommendations.

The authors maintain a modest tone, noting that while benign GEO exists (improving factual clarity), the lack of constraints allows malicious actors to erode trust. They do not propose a specific technical defense (like a new RAG architecture) but rather advocate for governance frameworks (auditing, disclosure, and updated metrics) to manage the risks introduced by the current trajectory of LLM answer engines.