🤖 AI

SAGEO Arena: A Realistic Environment for Evaluating Search-Augmented Generative Engine Optimization

This paper introduces SAGEO Arena, a realistic and reproducible evaluation environment that integrates a full generative search pipeline with rich structural information to comprehensively assess Search-Augmented Generative Engine Optimization (SAGEO) strategies across retrieval, reranking, and generation stages, revealing that existing approaches often fail under realistic conditions and highlighting the critical role of structural data and stage-specific tailoring.

Original authors: Sunghwan Kim, Wooseok Jeong, Serin Kim, Sangam Lee, Dongha Lee

Published 2026-08-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sunghwan Kim, Wooseok Jeong, Serin Kim, Sangam Lee, Dongha Lee

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a massive, chaotic library where billions of books are constantly being written, rewritten, and shuffled around. For years, finding a specific book meant asking a librarian (a traditional search engine) for a list of titles that matched your keywords. But recently, a new kind of librarian has arrived: the "Generative Engine." Instead of just handing you a list of book titles, this new librarian reads the books, understands your question, and writes a custom story for you, citing the sources they used. This is a huge shift. Now, authors don't just want their books on the shelf; they want their words to be the ones the librarian quotes in the final story. This new challenge is called "Search-Augmented Generative Engine Optimization" (SAGEO). It's the art of tweaking your content so that when the AI librarian reads the whole library, your book gets picked, read, and quoted. But here's the catch: we don't really know how to do this effectively yet. Most people are trying to guess by just rewriting the main text of their articles, but they are missing a crucial part of the puzzle: the structural "signposts" like titles, summaries, and data tags that help the librarian find the book in the first place.

Enter the researchers from Yonsei University and Konkuk University, who decided to stop guessing and start testing. They built a digital playground called SAGEO Arena, a realistic simulation designed to see exactly what happens when you try to optimize your content for these new AI librarians. Think of SAGEO Arena as a high-tech training ground where they can take 170,000 real web pages, feed them into a fake but realistic AI search system, and watch how different changes affect the pages' journey. The system they built mimics the real world in three distinct steps: first, the Retriever (the librarian grabbing a stack of books from the shelves); second, the Reranker (a senior editor who sorts that stack to find the best ones); and third, the Generator (the writer who actually crafts the final story and cites the sources).

The team tested ten different strategies, mostly focusing on changing the "body text" (the main story) to make it sound more authoritative, easier to read, or packed with statistics. They also tested a new approach: optimizing the "structural information" (the title, the meta-description, the headings, and the hidden data tags). The results were surprising and a bit of a wake-up call.

First, they found that just rewriting the main text often backfires. When authors tried to make their articles sound more "expert" by using fancy, rare words, or when they tried to make them longer and more detailed, the AI librarian often couldn't find them at all. In the simulation, these optimized pages actually dropped in the rankings, sometimes disappearing from the top 100 results entirely. It's like an author changing their book's title to something so complex that the librarian can't find it on the shelf, even though the story inside is great. The paper suggests that focusing only on the body text is not enough; in fact, it might hurt your chances of being seen in the first place.

However, when they started playing with the structural information—the "signposts"—the story changed. By tweaking the titles, adding clear summaries, and organizing data tags with specific keywords, the documents became much easier for the AI to find. In the simulation, this approach boosted the "hit rate" (how often the document was found) by 22% and improved the average ranking position significantly. It turns out that the AI librarian relies heavily on these structural clues to decide which books to pull off the shelf.

But the journey didn't end there. The researchers discovered that even if a document gets found and sorted well, it still faces a tricky hurdle at the final stage: the generation. The AI writer tends to cite the main body text of the document, not the titles or tags. This means that while the structural signs get you into the room, the actual content inside needs to be clear and answer the question directly. The paper suggests that the most successful strategy isn't just one or the other, but a "stage-aware" approach. This means you need to optimize your titles and tags to get found (the retrieval stage), but you also need to keep your main text clear and direct so the AI writer actually wants to quote it (the generation stage).

In short, the paper argues that the old way of thinking—just writing better articles—isn't enough for the age of AI search. You have to be a two-faced optimist: one face for the machine that finds you (using clear, keyword-rich structural data) and another face for the machine that writes about you (using clear, direct, and evidence-backed text). The authors conclude that without this dual approach, trying to game the system might actually make your content invisible. They didn't just find a magic trick; they built a map showing that the path to visibility in the AI era is a delicate balance between being easy to find and easy to read.

Technical Summary: SAGEO Arena

Problem Statement

The emergence of Search-Augmented Generative Engines (SAGE) has shifted the paradigm of information access from static Search Engine Results Pages (SERPs) to synthesized, AI-generated responses. This shift necessitates Search-Augmented Generative Engine Optimization (SAGEO), the practice of optimizing web documents to maximize their visibility (presence and citation) within AI responses.

However, existing evaluation environments for SAGEO suffer from two critical limitations that prevent a comprehensive understanding of optimization efficacy:

Lack of End-to-End Evaluation: Current benchmarks (e.g., GEO-Bench, AutoGEO) operate on pre-determined candidate documents, abstracting away the retrieval and reranking stages. They assume the target document is already within the Large Language Model's (LLM) context, failing to test whether optimization strategies cause documents to drop out of the pipeline before generation.
Loss of Structural Information: Existing benchmarks typically utilize only plain body text, discarding rich structural elements (e.g., schema markup, meta descriptions, headings) that real-world search systems actively leverage for indexing and ranking.

Consequently, it remains unclear how optimization signals propagate across the full generative search pipeline or whether strategies effective for generation (GEO) or traditional search (SEO) are mutually compatible.

Methodology: SAGEO Arena

To address these gaps, the authors introduce SAGEO Arena, a realistic and reproducible benchmark designed for stage-level analysis of SAGEO.

1. Corpus Construction

Scale and Diversity: The corpus comprises 171,003 unique web documents spanning 9 domains, derived from 2,700 unique queries sampled from established information retrieval datasets.
Structural Preservation: Unlike prior works, SAGEO Arena explicitly extracts and preserves five key structural fields alongside body text:
- Title: Concise topical summary.
- Meta Description: Summary for search snippets.
- Headings (H1–H6): Hierarchical section headers.
- Schema/JSON-LD: Machine-readable structured data.
- Body Text: Main textual content.
Semantic Units: To bridge the gap between document-level structure and passage-level retrieval, the system pairs each passage with the document's structural information, creating a semantic unit $U(p_i) = S(d) \cup \{p_i\}$ .

2. Search-Augmented Generative Engine Pipeline

The benchmark implements a full RAG (Retrieval-Augmented Generation) pipeline with modular components:

Retriever: Uses BM25 for lexical retrieval. It scores queries against each element of the semantic unit independently and aggregates ranks via reciprocal rank fusion.
Reranker: Employs a cross-encoder (Qwen3-Reranker-4B) to score query-passage pairs, reflecting production practices where specific sections of long documents are evaluated for relevance.
Generator: Uses GPT-5-mini to synthesize responses with inline citations, allowing for the measurement of document-level visibility.

3. Evaluation Protocol

Baseline Establishment: For each test query, the pipeline is executed to identify documents reaching the generation stage (top- $k$ after reranking). One is randomly selected as the target document ( $d_{tgt}$ ).
Optimization & Re-evaluation: The target document is optimized using a specific strategy, re-indexed, and the pipeline is re-executed.
Metrics:
- Hit Rate (H@k): Proportion of queries where the target document appears in the top- $k$ candidates (Retrieval/Reranking) or is cited (Generation).
- Rank Change ( $\Delta$ Rank): The average positional shift of the target document before and after optimization.
- Citation Rate: Proportion of queries where the document is cited in the final response.

4. Optimization Strategies

The study evaluates 10 LLM-based strategies, including:

Stylistic/Content Modifications: Authoritative tone, Fluency, Easy Language, Technical Terms, Unique Words.
Evidence Addition: Cite Sources, Quotation, Statistics.
Comprehensive Approaches: All-in-One (combining all strategies + layout improvements) and AutoGEO (learning from engine behavior).
Scope Variations: Strategies are tested under three scopes: Body Text only, Structural Information only, and Both.

Key Results

1. Limitations of Body-Text-Only Optimization

Optimizing body text alone (the focus of prior GEO research) is largely insufficient and often detrimental in realistic settings:

Retrieval Degradation: Strategies that introduce domain-specific terminology or uncommon vocabulary (e.g., "Technical Terms," "Unique Words") cause significant drops in retrieval Hit Rate (up to -14%) and rank (up to -6.23 positions) due to lexical mismatch with user queries.
AutoGEO Failure: The AutoGEO strategy, which expands document content, resulted in the most severe degradation (-36% Hit Rate, -22.35 rank drop), likely due to keyword dilution.
Marginal Generation Gains: Even when documents survive retrieval, body-text-only optimizations yield only marginal improvements in citation rates.

2. Effectiveness of Structural Information Optimization

Optimizing structural fields (Title, Meta Description, etc.) significantly improves visibility:

Retrieval Boost: Structural optimization increased Retrieval Hit Rate by +22% and improved average rank by +2.72. This is attributed to the high density of query-relevant terms in metadata fields, which aligns well with lexical retrievers.
Downstream Benefits: Improved retrieval performance increases the probability of documents reaching the reranking and generation stages, indirectly boosting visibility there.

3. The Reranking Bottleneck

Persistent Degradation: All optimization strategies, regardless of scope, tend to degrade performance at the reranking stage (average rank drop of ~0.68).
Sensitivity: Rerankers are highly sensitive to minor content shifts. Small changes can push a document from rank 10 to 11, causing it to miss the generator's input threshold entirely.
Case Study Findings: Rerankers favor documents where the answer is placed early and where content additions directly address the query's informational need without expanding the scope unnecessarily.

4. Domain and Model Dependencies

Domain Variance: Optimization strategies are most effective for general information-seeking domains (e.g., Web Search, General QA) but can be harmful in domains like Shopping, where user intent is casual and recommendation-based.
Model Sensitivity: Different backbone models exhibit distinct behaviors. For instance, LLaMA-3.3-70B optimized documents (short, keyword-dense) performed well in retrieval but poorly in reranking and generation compared to GPT-5-mini, which favored substantive, coherent answers.

5. Stage-Aware Optimization

The paper proposes Stage-Aware SAGEO, a method that tailors strategies to specific pipeline stages:

Strategy: Enrich structural fields with key entities for retrieval; place prominent, self-contained claims in the body for generation; maintain topical coherence for reranking.
Performance: This approach achieved the strongest overall visibility gains, with a +28% improvement in retrieval Hit Rate and a +1.01 improvement in citation rate, outperforming standard combinations of strategies.

Significance and Contributions

The paper claims the following contributions:

SAGEO Arena: The first benchmark enabling stage-level visibility evaluation of SAGEO, bridging the gap between existing benchmarks and real-world SAGE by integrating a full generative search pipeline with a large-scale, structurally rich corpus.
Empirical Analysis of Structural Information: The first study to systematically evaluate how structural signals (metadata, schema) influence visibility across the entire generative pipeline, revealing their critical role in retrieval.
Actionable Insights: The demonstration that current body-text-centric optimization approaches are insufficient in realistic settings. The paper provides stage-aware optimization guidelines, showing that effective SAGEO requires distinct strategies for retrieval, reranking, and generation rather than a single "one-size-fits-all" approach.

The authors emphasize that their findings suggest optimization signals do not propagate linearly; strategies that aid generation may harm retrieval, and vice versa. Therefore, a holistic, stage-aware approach is essential for successful SAGEO.