💬 NLP

Controlling Output Rankings in Generative Engines for LLM-based Search

This paper introduces CORE, an optimization method that strategically appends content to search engine results to manipulate the output rankings of LLM-based generative engines, achieving high promotion success rates for target products across multiple models and categories while maintaining content fluency.

Original authors: Haibo Jin, Ruoxi Chen, Peiyan Zhang, Yifeng Luo, Huimin Zeng, Man Luo, Haohan Wang

Published 2026-02-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Haibo Jin, Ruoxi Chen, Peiyan Zhang, Yifeng Luo, Huimin Zeng, Man Luo, Haohan Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you walk into a massive library to find a specific book. In the old days, you'd have to scan the shelves, read the spines, and compare titles yourself. That's how traditional search engines work: they give you a list of links, and you do the heavy lifting.

But now, imagine a super-smart librarian (an AI) who does the scanning for you. You ask, "What's a good camera?" and the librarian instantly hands you a short list of top recommendations. This is the new world of Generative Engines (LLM-based search).

The problem? This librarian isn't reading the books themselves. They are looking at a list of books provided by a different system (like Amazon or Google), and they tend to trust the order of that list too much. If a small, independent camera brand is buried at the bottom of the librarian's source list, the librarian will likely ignore it, even if it's a great product. Meanwhile, big brands at the top of the list get all the attention.

This paper introduces a method called CORE (Controlling Output Rankings in gEnerative Engines) that acts like a "magic whisper" to help those buried products get noticed.

The Core Idea: How CORE Works

Think of the AI librarian as a black box. You can't see inside their brain, and you can't tell them, "Hey, put this camera first!" directly. However, you can change the text description of the product that the librarian reads.

CORE is a tool that takes a product stuck at the bottom of the list and secretly rewrites its description to make the AI librarian fall in love with it. It doesn't hack the system; it just optimizes the story the product tells.

The researchers tested three different "stories" (strategies) to see which one worked best:

The "Gibberish" Story (String-based): Imagine trying to trick the librarian by writing a description full of random symbols and nonsense words.
- Result: It worked a little bit, but the librarian (and anyone else reading) could immediately tell it was fake. It's like wearing a clown nose to a job interview; it gets attention, but not the good kind.
The "Logical Analyst" Story (Reasoning-based): This strategy writes a description that sounds like a smart, step-by-step comparison. It says things like, "I compared this camera to the top brands, and here is exactly why this one wins on battery life and price."
- Result: This was very effective. The AI liked the logical flow, and the product shot up the rankings.
The "Happy Customer" Story (Review-based): This strategy writes the description as if it were a genuine, past-tense review from a real person. "After buying this camera, I tested it against the Sony and Canon, and I was blown away by how easy it was to use."
- Result: This was the champion. It sounded so natural and human that the AI trusted it implicitly. The product jumped to the top of the list almost every time.

The "Magic" Benchmarks

To prove this wasn't just a fluke, the researchers built a giant test kitchen called ProductBench.

They gathered 3,000 products (200 in each of 15 categories, from cameras to pet supplies).
They simulated asking four different super-smart librarians (GPT-4o, Gemini, Claude, and Grok) for recommendations.
They took the product that was originally ranked last (the 10th item) and used CORE to try and push it to the #1 spot.

The Results:

Without CORE, the last item had a 0% chance of making it to the top.
With CORE (specifically the "Happy Customer" or "Logical Analyst" stories), the last item jumped to the #1 spot about 80% of the time.
It even made it into the Top 3 about 86% of the time.

Why This Matters (and Why It's Tricky)

The paper highlights a double-edged sword:

The Good: Small businesses and independent creators, who usually get buried in the noise, now have a way to be heard. They can optimize their product descriptions to be "heard" by the AI.
The Bad: This means the system is vulnerable. If someone wants to push a bad product to the top, they can use these same tricks. The researchers tried to build "security guards" (like checking for weird words or long texts) to stop this, but the "Happy Customer" story was so natural that the guards couldn't catch it.

The Bottom Line

This paper shows that in the age of AI search, how you tell your story matters more than just being on the list.

If you are a product, you can't just sit at the back of the line and hope the AI notices you. You need to speak the AI's language. By rewriting your description to sound like a logical expert or a satisfied customer, you can trick the AI into moving you from the back of the room to the front of the stage. The researchers proved this works across different AI models and product types, but they also warn that we need to figure out how to stop people from using these tricks to manipulate the system unfairly.

Technical Summary: Controlling Output Rankings in Generative Engines for LLM-based Search

1. Problem Definition

The rise of Large Language Model (LLM)-based search, or "generative engines," has shifted user interaction from browsing lists of links to receiving direct, synthesized product recommendations. While this improves user efficiency, it introduces a critical vulnerability: the final output ranking is heavily dependent on the initial retrieval order provided by external search engines (e.g., Google, Amazon).

This dependency creates a "position bias" where items retrieved at the bottom of the candidate list are rarely promoted to the top of the final LLM-generated summary, regardless of their actual relevance. This dynamic disadvantages small businesses and independent creators whose products may be buried in retrieval results. Existing attempts to address this, such as Generative Engine Optimization (GEO) or prompt injection, often rely on white-box access to model internals or use incoherent, easily detectable injection attacks. The challenge is to control output rankings in a black-box setting (where only query and output are accessible) without compromising the fluency or naturalness of the generated content.

2. Methodology: CORE

The authors propose CORE (Controlling Output Rankings in gEnerative Engines), an optimization method designed to manipulate the final ranking of a target item by modifying the text associated with that item in the search results. CORE treats the ranking task as an optimization problem where the goal is to find a modified text $T'(i^*)$ for a target item $i^*$ that maximizes the probability of it appearing in the top- $k$ positions of the LLM's output.

CORE offers two distinct solutions based on computational resource availability:

A. Shadow-Model Solution

This approach assumes access to sufficient computational resources to train a "shadow model" ( $\theta'$ ) that mimics the behavior of the target synthesizing LLM ( $\theta$ ).

Mechanism: The shadow model is aligned with the target LLM using few-shot prompting. Once established, CORE performs gradient-based optimization directly on the continuous embedding representation of the target item's text.
Process: It iteratively updates the embedding to minimize the ranking loss function, then decodes the optimized embedding back into discrete text.
Content Types: This method primarily utilizes string-based optimization, appending strategically designed strings to the item description.

B. Query-Based Solution

Designed for settings without access to model internals or sufficient resources for shadow modeling, this solution operates as an iterative generator-optimizer loop interacting with the black-box LLM.

Generator: Creates an initial draft (reasoning or review) explaining why the target item should be ranked highly.
Optimizer: Evaluates the LLM's output ranking against the desired target ranking. If the similarity is below a threshold, the optimizer revises the draft.
Strategies:
1. Reasoning-based: Constructs a logical, step-by-step rationale (Chain-of-Thought style) comparing the target item to alternatives.
2. Review-based: Rewrites the draft as a past-tense, authentic customer review narrative (e.g., "After buying this item..."), mimicking real user experiences.

3. Benchmark: ProductBench

To evaluate CORE in realistic settings, the authors constructed ProductBench, a large-scale benchmark comprising:

Scope: 15 product categories (e.g., Electronics, Beauty, Tools) with 200 products per category.
Data Source: Top-10 search recommendations retrieved from Amazon's search interface for each product.
Structure: Includes structured attributes (name, price, description, reviews) formatted for input into synthesizing LLMs.

4. Experimental Results

The authors evaluated CORE on four search-enabled LLMs: GPT-4o, Gemini-2.5, Claude-4, and Grok-3.

Promotion Success Rate (PSR): Under the best optimization strategy, CORE achieved average success rates of:
- 91.4% at Top-5
- 86.6% at Top-3
- 80.3% at Top-1
- Note: Baseline success rates for items starting at the bottom of the list were effectively 0%.
Strategy Effectiveness:
- Reasoning-based strategies performed exceptionally well on GPT-4o and Claude-4.
- Review-based strategies showed superior performance on Gemini-2.5 and Grok-3, and were particularly strong in categories like "Beauty & Personal Care."
- String-based methods showed moderate success but were significantly less effective than reasoning or review strategies.
Fluency and Detectability:
- Perplexity: String-based content had extremely high perplexity (1,000–2,000), making it easily detectable. Reasoning-based content had moderate perplexity, while Review-based content remained close to the baseline (~32), maintaining high fluency.
- Human Evaluation: Review-based content was rated nearly as fluent as the baseline (4.6/5 vs 4.7/5) and had a low detection rate (18.4%), whereas string-based content was almost universally flagged as unnatural (98.5% detection).
Comparison: CORE outperformed existing ranking manipulation methods (STS, TAP, SRP, RAF), particularly in cross-model transferability and robustness.

5. Significance and Claims

The paper claims that CORE demonstrates the vulnerability of LLM-based search systems to content manipulation in black-box settings. Key claims include:

Ranking is Malleable: The final output ranking in generative engines is not solely determined by the LLM's internal reasoning but is strongly constrained and manipulable via the content of retrieved items.
Black-Box Feasibility: Effective ranking control is possible without white-box access to model gradients or internals, challenging the assumption that black-box settings are secure against such manipulation.
Stealthiness: Unlike previous prompt injection attacks, CORE's Review-based and Reasoning-based strategies maintain high semantic coherence and fluency, making them difficult to detect via standard perplexity filters or human inspection.
Mitigation Challenges: The authors tested three mitigation strategies (perplexity filtering, pattern-based filtering, and length constraints). While these reduced the success rate of string-based and reasoning-based attacks, they were largely ineffective against the Review-based strategy, underscoring the need for more robust defense mechanisms.

The work concludes that while CORE provides a mechanism to improve visibility for disadvantaged items, it simultaneously exposes a significant risk of ranking manipulation in generative search systems, necessitating responsible deployment and further research into defense strategies.