🤖 machine learning

GEO-Flag: Detecting and Measuring GEO-Optimized Web Content

This paper introduces GEO-Flag, a comprehensive framework featuring the GEOFlagBench benchmark and an Intervention-Paired Training method to effectively detect, audit, and measure the prevalence of Generative Engine Optimization (GEO) in real-world search ecosystems.

Original authors: Junjie Chu, Ye Leng, Mingjie Li, Yun Shen, Xinyue Shen, Yang Zhang

Published 2026-08-18

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Junjie Chu, Ye Leng, Mingjie Li, Yun Shen, Xinyue Shen, Yang Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The internet is changing how we find answers. For decades, we searched by typing a question and scanning a list of links, clicking through to read the source ourselves. Now, a new generation of search tools uses artificial intelligence to read those links for us, synthesizing the information into a single, direct answer with citations. This shift promises convenience, but it also creates a new vulnerability. Just as websites were once tweaked to rank higher in traditional search lists, a new practice has emerged where content is deliberately altered to be picked up and cited by these AI systems. This practice, known as generative engine optimization, can make weak or false information appear well-supported because the AI presents it as a fact without showing the user the full list of competing sources. The danger lies in the opacity: if the AI cites a page that was strategically built to be cited, the user sees a confident answer, not the manipulation behind it.

A team of researchers set out to understand the scale of this problem and to build tools to measure it. They began by creating a massive test set of 3,200 webpages covering health, finance, technology, and travel. These pages included original human writing, text polished by artificial intelligence, and content generated entirely by machines. Crucially, they also created versions of these pages that had been specifically modified using eight different strategies designed to game the new AI search engines. With this diverse collection, they tested existing methods for spotting these optimized pages. They found that while some tools could flag the content with high accuracy overall, they were often relying on shortcuts. Many detectors were simply guessing based on whether the text was written by a human or an AI, rather than identifying the specific changes made to manipulate the search engine. When the researchers tested these tools on pages where humans had carefully applied optimization techniques, the detectors often failed, revealing that their previous success was based on a misunderstanding of what they were actually detecting.

To solve this, the researchers developed a new training method called intervention-paired training. Instead of just showing the computer examples of "good" and "bad" pages, they showed it pairs of pages: the original version and the modified version. They taught the system to recognize that a specific change, like adding a certain phrase to boost visibility, should increase the suspicion score, while a general improvement in grammar or style should not. This approach forced the detector to focus on the actual act of optimization rather than the author's identity. The result was a significant improvement in reliability. The new system correctly identified optimized pages with much higher accuracy and, importantly, did not get confused by whether the original text was written by a person or a machine. It learned to spot the manipulation itself.

The team then built a complete auditing system that goes a step further. Once a page is flagged as potentially optimized, the system checks the links cited on that page to see if they are trustworthy. It assigns a tier to the publisher, ranging from highly authoritative institutions to sources that are easy to create or edit, and checks if the links are still working. When they applied this entire pipeline to real-world search results from Google and a new AI-powered search tool, they found that roughly 8.9% of the pages returned were optimized for generative search. This number was not static; for pages that had been modified in 2026, the rate jumped to 16.36%. The study also revealed a stark difference in the quality of sources used. Pages returned by the AI search tool relied heavily on citations from lower-tier sources, with nearly three-quarters of those citations receiving a low verifiability label, compared to less than half for traditional search results.

These findings suggest that the landscape of online information is shifting rapidly. The researchers estimate that more than one in ten pages currently appearing in these new search results may have been strategically altered to influence the AI. The problem is not just that false information is being spread, but that the very mechanism we use to verify facts—the citation—is becoming part of the manipulation. By identifying these patterns and measuring their prevalence, the researchers have provided a way to see behind the curtain of the new search era. Their work does not claim to have solved the issue or to have found every instance of manipulation, but it establishes a clear foundation for measuring the problem. It shows that without new tools to detect these specific optimizations, users may increasingly encounter answers that look authoritative but are built on a foundation of strategically constructed, and often unverified, sources.

Technical Summary: GEO-Flag: Detecting and Measuring GEO-Optimized Web Content

1. Problem Definition

Generative Engine Optimization (GEO) involves modifying web content to increase its likelihood of being selected and cited by generative search engines (e.g., Perplexity, Google Gemini, ChatGPT Search). Unlike conventional search, which presents a list of competing sources, generative search synthesizes information into direct answers with inline citations. This shift creates new risks:

Disproportionate Visibility: Strategically optimized pages may gain visibility disproportionate to their actual authority or relevance.
Amplification of Misinformation: GEO techniques can make weak or false information appear well-supported by strategically creating sources that cite one another.
Detection Gap: While significant research exists on how to optimize for generative engines, systematic methods for detecting whether a webpage has been deliberately optimized for this purpose remain underexplored. Existing tools often conflate GEO detection with general AI-text detection, failing to distinguish between content that is AI-generated and content that is human-written but optimized for generative search.

2. Methodology

The paper introduces a comprehensive framework comprising a benchmark, a novel training strategy, an auditing agent system, and a large-scale empirical deployment.

2.1 GEOFlagBench: A New Benchmark

To evaluate detection methods, the authors constructed GEOFlagBench, a dataset of 3,200 webpages spanning:

400 Queries: General informational queries across four domains: Health, Finance, Technology, and Travel.
8 GEO Optimizer Families: Including strategy-based rewriting (GEO Strategy Pool), learned optimization (AutoGEO), preference manipulation (PMA), intent-aware optimization (RAID G-SEO), iterative self-evaluation (Meta-Optimization), and stealthy variants.
Control Groups: For each query, three non-GEO controls were created:
1. Human-Written: Sourced from reliable publishers (e.g., Wikipedia) prior to October 2022.
2. AI-Polished: Human-written text improved by LLMs for grammar and fluency without GEO strategies.
3. AI-Generated: New articles written by LLMs based on structured summaries of the source text.
Evaluation Design: The benchmark uses a query-disjoint split to prevent semantic leakage and includes authorship-conditioned diagnostics to test for reliance on "human vs. AI" shortcuts rather than GEO-specific signals.

2.2 Intervention-Paired Training (IPT)

The authors propose Intervention-Paired Training (IPT) to address the limitations of existing detectors, which often rely on static authorship cues. IPT utilizes the paired structure of the benchmark (original source vs. transformed page) to supervise detector responses:

Positive Pairs: An original page ( $x_o$ ) and its GEO-optimized counterpart ( $x_g$ ). The model is constrained to assign a higher GEO score to $x_g$ than $x_o$ .
Zero Pairs: An original page ( $x_o$ ) and its AI-polished counterpart ( $x_p$ ). The model is constrained to maintain a similar score for both, teaching it that AI editing alone is not evidence of GEO.
Objective: The loss function combines standard classification loss with margin-based constraints for positive pairs and L1 constraints for zero pairs. This encourages the model to learn optimization-specific changes rather than general authorship or LLM-generation artifacts.

2.3 GEO-Gated Agent System for Citation Auditing

Beyond page-level detection, the system audits the Source Tier and Verifiability of Citation URLs used by detected GEO pages:

GEO Gate: A detector (ModernBERT-IPT) filters pages; only those flagged as GEO proceed.
Preprocessing: A deterministic parser extracts citation URLs and associates them with sentences.
Labeling Agent: Specialized sub-agents determine:
- URL Source Tier (C1–C3): Based on publisher authority and editorial control (C1 = high authority, C3 = low accountability).
- Citation URL Verifiability (HIGH/MEDIUM/LOW): A combination of Source Tier and retrieval status (accessibility).
Aggregation: Rules derive the final verifiability label for each citation and the best label for each cited sentence.

2.4 Empirical Deployment

The complete pipeline was deployed on 1,000 real-user queries from the ORCAS dataset, analyzing results from Google Search and Gemini-grounded retrieval. The study collected 10,095 usable pages from 13,985 unique URLs, applying the detector and citation audit pipeline to estimate real-world GEO prevalence.

3. Key Results

3.1 Benchmark Evaluation

Baseline Performance: The strongest baseline (Word TF-IDF + Logistic Regression) achieved an aggregate F1 of 0.880. However, method-level analysis revealed significant weaknesses.
Shortcuts: High-performing baselines showed poor Worst-Group Accuracy (WGA) (e.g., 0.375 for TF-IDF) and large False Positive Rate gaps ( $\Delta$ FPR) between human- and AI-authored non-GEO pages (up to 0.558). This indicates reliance on authorship cues rather than GEO signals.
IPT Improvements: Applying IPT to ModernBERT improved F1 from 0.862 to 0.944 and WGA from 0.725 to 0.883. It also reduced the $\Delta$ FPR from 0.263 to 0.108, demonstrating robustness against authorship shortcuts. IPT also improved attribution accuracy for specific GEO families (Macro F1 0.895).

3.2 Agent System Performance

On a zero-shot audit benchmark (562 pages), the GLM 5.2 agent achieved:

84.75% accuracy for URL Source Tier classification.
83.00% accuracy for Citation URL Verifiability.
The system successfully identified that many GEO pages rely on low-authority sources.

3.3 Real-World Prevalence Estimates

Overall Prevalence: An estimated 8.90% of the 10,095 analyzed pages were flagged as GEO-optimized.
Channel Differences: GEO prevalence was higher in Gemini-grounded results (9.09%) compared to Google Search (8.14%).
Temporal Trend: Among pages with parseable dateModified values, GEO prevalence increased significantly over time: 7.02% (2024) $\to$ 12.80% (2025) $\to$ 16.36% (2026).
Citation Verifiability: Among 6,663 citation occurrences on detected GEO pages, 69.34% received LOW verifiability labels. This share was even higher for Gemini (74.15%) compared to Google Search (45.88%), indicating a heavy reliance on C3 (low accountability) sources.

4. Key Contributions

GEOFlagBench: A systematic benchmark for detecting GEO-optimized content across diverse domains and optimization strategies, revealing that aggregate performance often masks reliance on authorship shortcuts.
Intervention-Paired Training (IPT): A training method that improves GEO detection robustness by explicitly supervising responses to GEO interventions while ignoring non-GEO AI polishing, significantly reducing reliance on authorship cues.
GEO-Gated Agent System: A pipeline for auditing the Source Tier and verifiability of citations used by GEO pages, providing a mechanism to assess the trustworthiness of the underlying sources.
Empirical Prevalence Estimates: The first large-scale measurement of GEO prevalence in real-world search results, finding an 8.90% overall rate and a clear upward trend in recently modified pages, alongside a high prevalence of low-verifiability citations.

5. Significance and Claims

The paper establishes a foundation for systematically detecting, auditing, and measuring GEO in real-world search ecosystems. It argues that:

GEO is distinct from AI-text detection: Detectors must distinguish between content optimized for generative search and content simply written by AI.
Current detectors are fragile: High aggregate scores often hide failures on specific optimization methods or authorship groups.
GEO poses a verifiability risk: A significant portion of GEO-optimized pages rely on citations from low-authority sources (C3) that are difficult to verify, potentially amplifying misinformation in generative answers.
Transparency is key: The proposed tools are designed to support transparency and risk analysis rather than automatic content blocking, acknowledging that GEO is not inherently malicious but requires scrutiny due to its potential to distort information access.

The authors emphasize that their real-world estimates are empirical observations based on their detector's predictions and available metadata, not definitive measurements of malicious intent, but they highlight a clear and growing trend of GEO adoption in 2025 and 2026.