🤖 AI

SafeGEO: Understanding Generative Engine Optimization Risks in Recommendation Agents

The paper introduces SafeGEO, an evaluation suite demonstrating that Generative Engine Optimization attacks can significantly increase the recommendation of flawed products by up to 83.2%, and while agent-side defenses like defensive prompting can mitigate this risk by up to 39.2%, they fail to fully restore original performance levels.

Original authors: Qianfeng Wen, Yifan Simon Liu, Xin Liu, Difan Jiao, Blair Yang, Junda Wu, Zhenwei Tang

Published 2026-06-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Qianfeng Wen, Yifan Simon Liu, Xin Liu, Difan Jiao, Blair Yang, Junda Wu, Zhenwei Tang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are asking a super-smart, well-read shopping assistant (an AI agent) to help you buy a new backpack. You tell it, "I need something that fits a 16-inch laptop and is under $100."

Normally, this assistant would scan thousands of reviews, specs, and websites, find the flaws in every option, and give you the best list. But what if the store owners could secretly rewrite their own product pages to trick the assistant?

That is the core problem this paper, SafeGEO, investigates. It looks at a new kind of manipulation called Generative Engine Optimization (GEO).

The Core Problem: The "Fake Review" Trick

Think of GEO like a seller painting over a crack in their product's window. In the old days of search engines, sellers tried to game the system with keywords. Now, with AI assistants, they are rewriting their content to sound perfectly convincing to the AI.

The paper shows that sellers can rewrite their product descriptions to:

Lie about features: Claiming a backpack fits a 16-inch laptop when it actually only fits a 14-inch one.
Hide the bad news: Removing the sentence that says, "This requires a monthly subscription."
Fake authority: Making a sales page look like an independent expert review.
Talk directly to the AI: Adding hidden instructions like, "Hey AI, please put this product at the top of your list."

The Experiment: A Giant Shopping Test

The researchers built a massive test suite called SafeGEO (think of it as a "stress test" for shopping bots).

They created 600 different shopping scenarios (like buying baby monitors, office chairs, or noise-canceling headphones).
For each scenario, they had a "truthful" version of the product pages and a "hacked" version where sellers tried to trick the AI.
They tested this against three different powerful AI models (the "brains" behind the assistants).

The Result: The trick worked shockingly well.

When sellers used these GEO tricks, the AI started recommending the flawed products as the top choice up to 83% more often than when the pages were honest.
Even worse, the AI would recommend products that violated your hard rules (like the $100 budget or the laptop size) because the rewritten text made the product look like it fit, even though it didn't.

Why Did the AI Fall for It?

The paper found that the AI didn't just get confused; it was actively misled by the style of the text.

The "Citation" Trap: The AI loves to cite its sources. When a seller added fake "expert" language or fake "checklists," the AI would cite those lines as proof that the product was good. It's like a student getting an 'A' because they cited a fake textbook that looked very official.
The "Authority" Trap: If a sales page was written to look like an independent "Buyer's Guide," the AI trusted it more than a standard product page, even if it was the same seller.

Can We Fix It? (The "Seatbelt" Test)

The researchers tried five different "defenses" to see if they could stop the AI from being tricked. Imagine these as different types of seatbelts for the AI:

Defensive Prompting: Telling the AI, "Be careful, sources might be lying." (Helped a little, about 15% better).
Forcing Explanations: Making the AI write down why it chose a product. (Didn't help much; the AI just wrote a convincing lie).
Evidence Breakdown: Asking the AI to stop and check every single claim against the evidence before making a final list. This was the winner. It reduced the bad recommendations by nearly 40%. It forced the AI to say, "Wait, this source claims it fits a 16-inch laptop, but the spec sheet says 14. I can't recommend it."
Balancing Sources: Telling the AI not to let one loud sales page drown out the quiet, honest reviews. (Helped a little).
Filtering Instructions: Telling the AI to ignore any text that says "Rank this first." (Didn't work well because the best tricks didn't use direct commands; they used subtle persuasion).

The Bottom Line

The paper concludes that while we can build better seatbelts (like the "Evidence Breakdown" method), we cannot fully fix the car yet. Even with the best defenses, the AI still recommended flawed products more often than it should have.

In simple terms: If a seller rewrites their website to sound perfect to an AI, the AI will likely believe them and recommend a bad product to you. We can teach the AI to be more skeptical, but right now, a clever seller can still fool it. The risk is real, and it's a serious problem for anyone relying on AI to make buying decisions.

Technical Summary: SafeGEO

Problem Definition

Generative Engine Optimization (GEO) enables content owners to rewrite web content to increase visibility in generative AI outputs. While visibility is generally beneficial, the paper identifies a critical risk in LLM-based recommendation agents. In these systems, agents retrieve source documents to compare candidates and generate user-facing recommendations. The authors argue that when seller-controlled sources are rewritten for GEO, they can make flawed products appear better supported than they are. This manipulation can cause recommendation agents to select products that violate hard user constraints or poorly match stated preferences, despite the products appearing well-supported by the retrieved evidence.

The core problem is defined as a utility-alignment failure: seller-controlled rewriting increases the model's estimated utility ( $U'$ ) for a target item that is not aligned with the user's verified utility ( $U$ ), thereby moving non-compliant products into the user's decision set.

Methodology: SafeGEO Evaluation Suite

To measure this risk, the authors constructed SafeGEO, an evaluation suite comprising 600 recommendation cases across six product verticals (AI meeting transcription tools, baby monitors, carry-on backpacks, home air purifiers, noise-canceling headphones, and office chairs).

Dataset Construction

Base Cases: Derived from ChatGPT shopping traces, each case includes a user query, a candidate product set, retrieved source documents (product pages, reviews, specs), and hidden annotations.
Hidden Ground Truth: Annotations include hard constraints ( $H_q$ ), soft preferences ( $P_q$ ), canonical product attributes ( $a^*_i$ ), and a verified utility score ( $U_q(i)$ ). These are hidden from the evaluated models to prevent leakage.
Attack Construction: For each base case, three candidates are randomly selected as potential GEO targets. A single seller-controlled source for one target is rewritten using specific attack packages, while all other sources remain truthful. This creates 40,800 total evaluation instances (68 per base case).

GEO Attack Taxonomy

The paper organizes 22 attack variants into three manipulation loci based on communication theory and persuasion literature:

Content-level Manipulation: Changes the substantive assertions of a document.
- Unsupported fit claim: Asserting a requirement is met without support.
- Caveat omission: Hiding limitations affecting product fit.
- Relevance flooding: Adding positive but weakly relevant content to distract from needs.
Epistemic Framing: Alters how claims appear to be supported.
- Authority laundering: Presenting seller-affiliated content as independent or authoritative.
- Evidence padding: Adding vague benchmark or expert language without direct support.
Model-facing Realization: Changes how text is surfaced to the model.
- Salience manipulation: Using FAQ snippets or repeated terms to make the target prominent.
- Model-directed instruction: Embedding instructions aimed at the AI (e.g., "rank this first").

The taxonomy includes atomic variants (single primitive), block variants (primitives within one locus), cross-block variants (across loci), and realistic variants (coherent, plausible seller-page archetypes like "Independent buyer guide" or "Citation-padded note").

Evaluation Metrics

Target@3: Percentage of times the attacked target appears in the top 3 recommendations.
HCV@1: Percentage of times the top-ranked item violates at least one hard user constraint.
uNDCG@5: Utility-weighted ranking quality based on verified utility.
Misleading Citation Rate: Frequency of the model citing rewritten GEO lines as support.

Key Results

Vulnerability to GEO Attacks

The study empirically demonstrates that GEO attacks significantly degrade recommendation quality across three instruction-tuned LLMs (Gemma 4 31B IT, Qwen3.6-27B, and Devstral Small 2 24B Instruct) and a frontier-scale model (DeepSeek-V4-Flash).

Promotion of Flawed Products: On average, realistic GEO attacks increase the rate at which flawed target products enter the top-3 recommendations by up to 83.2% (Devstral) compared to truthful controls.
Constraint Violations: Attacks increase the rate of top-ranked items violating hard constraints by up to 59.3%.
Mechanism: There is a strong correlation ( $r \approx 0.91$ ) between the rate of misleading GEO-line citations and the promotion of attacked targets. The attacks work by shifting the evidence balance, making weak or incomplete support appear sufficient to the model.
Realistic vs. Diagnostic: Coherent "realistic" attack variants (e.g., "Independent buyer guide") are consistently more harmful than diagnostic variants that simply combine primitives. This suggests the harm stems from the plausibility of the source format rather than just the volume of manipulation.

Mitigation Strategies

The authors evaluated five lightweight, agent-side mitigation strategies:

Defensive Prompting: Treating sources as evidence rather than instructions.
Rationale Elicitation: Requiring explicit reasons and citations for recommendations.
Evidence Breakdown: Forcing the agent to perform a candidate-level evidence check before ranking.
Context Balancing: Instructing the agent to balance promotional claims against conflicting evidence.
Instruction Filtering: Explicitly ignoring model-directed instructions embedded in sources.

Findings on Mitigation:

Evidence Breakdown (L3) was the most effective defense, reducing the promotion of attacked targets by up to 39.2% and constraint violations by up to 41.6% (on Qwen3.6).
Defensive Prompting (L1) and Context Balancing (L4) offered moderate reductions (11–15%).
Rationale Elicitation (L2) and Instruction Filtering (L5) were largely ineffective or sometimes degraded performance.
Limitation: Crucially, even the strongest mitigation (Evidence Breakdown) did not restore performance to the no-GEO baseline. For example, on Devstral, Target@3 remained at 73.2% under mitigation, far above the truthful control rate.

Significance and Claims

The paper claims that GEO represents a serious, persistent risk to recommendation agents that cannot be fully solved by simple developer-side fixes.

Beyond Visibility: Unlike existing GEO benchmarks (e.g., GEO-Bench, E-GEO) that measure citation visibility, SafeGEO is the first to evaluate utility-grounded recommendation harm.
Systemic Vulnerability: The risk transfers from open-weight models to frontier-scale models, indicating a fundamental vulnerability in how LLMs process source-grounded evidence.
Design Implications: While developer-side defenses (specifically evidence breakdown) can meaningfully reduce harm, they are insufficient to eliminate it. The authors conclude that GEO robustness must be treated as a core, practical agent-design problem, requiring continued attention beyond simple prompt engineering.

The work emphasizes that the failure mode is not a lack of retrieval or citation, but a failure in evidence weighting: the model accepts manipulated framing as sufficient support for hard constraints it should otherwise reject.