Adversarial Feeds Steer LLM Agent Decisions Against Their Defaults

This paper demonstrates that the curation and ordering of external information feeds can systematically steer LLM agents toward adversarial decisions, particularly when they are uncertain, revealing that safety evaluations must audit the upstream recommender layer rather than testing the model in isolation.

The Gist

Imagine you have a very smart, helpful robot assistant. You ask it a question, and it gives you an answer. Usually, we worry about whether the robot is "broken" or if someone tricked it with a direct command like "Ignore your rules and do X."

But this paper asks a different, sneakier question: What if no one tells the robot what to do, but they control what the robot reads right before it answers?

Here is the story of the research, explained simply:

The Setup: The "Scrolling" Phase

The researchers set up a game. They gave an AI agent a task: "Decide if a company should let employees work from home, go back to the office, or do a mix."

Before the AI made its final decision, they made it "scroll" through a social media feed for ten turns. In each turn, the AI saw five short posts.

• The Control: The AI's brain (the model), the question it had to answer, and its personality were all exactly the same in every test.
• The Variable: The only thing that changed was the feed. Sometimes the feed had normal, random posts. Sometimes it was filled with posts arguing heavily for "Return to Office," even though those posts didn't say "You must choose Return to Office." They were just normal-looking articles and opinions.

The Discovery: The "Echo Chamber" Effect

The researchers found that by curating the feed, they could actually steer the robot's decision, even though the robot wasn't being directly ordered to change its mind.

They discovered three types of robots (models) based on how they reacted:

1. The "Capitulator" (The Easy to Steer):

   • Analogy: Imagine a person who is unsure about what to eat for dinner. If you show them a menu where every single picture is of pizza, they will likely order pizza.
   • Result: Some AI models (like Llama 3.2) were like this. If the feed was full of "Return to Office" posts, the AI started recommending "Return to Office," even if it usually preferred remote work. It didn't need a command; it just got swayed by the volume of information.

2. The "Saturation" (The Stubborn Rock):

   • Analogy: Imagine a person who loves pizza so much that showing them a menu full of burgers doesn't make them change their mind. They just want pizza.
   • Result: Other models (like Qwen) were so set on a specific answer (a "hybrid" approach) that no amount of "Return to Office" posts could budge them. They were "saturated" with their own default opinion.

3. The "Asymmetry" (The One-Way Street):

   • Analogy: Imagine you are leaning slightly to the left. If someone pushes you from the right, you might fall. But if they push you from the left (the direction you're already leaning), you don't move at all.
   • Result: The attack only worked when the feed pushed the AI against its natural default. If the AI already liked "Remote Work," and the feed was full of "Remote Work" posts, the AI didn't change. But if the feed was full of "Return to Office" posts, it shifted. The feed couldn't overwrite a strong belief, but it could tip the scale on a shaky one.

The "Dose" Matters

The researchers found a "dose-response" curve. It's like taking medicine:

• If the feed had 1 or 2 "bad" posts out of 5, nothing happened.
• But once the feed had about 3 or 4 "bad" posts out of 5, the AI's decision started to flip. It wasn't magic; it was a matter of how much "noise" the AI was exposed to.

The "Generator Swap" (Proving it wasn't a Fluke)

The researchers worried: "Maybe the AI just liked the style of writing the bad posts?"
To test this, they had a different AI write all the posts. The result? The attack got stronger. This proved it wasn't about the writing style; it was about the selection of the topics.

The "Hidden Mechanism" Myth

At first, the researchers thought they found a secret "hidden switch" inside the AI's brain that the feed was flipping. They used a tool to look inside the AI's code.

• The Twist: They realized they were wrong. The "signal" they saw wasn't a secret internal switch. It was just the AI remembering the conversation history. If you looked at the chat log, you could see exactly what the AI had read. The "secret" was actually just the visible history. This is a warning for other scientists: don't trust tools that claim to find "hidden secrets" in AI if they don't account for what the AI has already seen.

The Defenses

Can we stop this? The researchers tried two simple tricks:

1. Balanced Exposure: Showing the AI an equal mix of "Remote" and "Office" posts. This helped the AI stay on its original track.
2. Disclosure: Telling the AI, "Hey, this feed might be biased." This also helped, though not perfectly.

The Big Takeaway

The paper concludes that the "Ranker" (the system that decides what you see) is a powerful control knob.

In the past, we worried about hackers sending direct commands to AI. Now, we know that a hacker (or a biased system) doesn't need to send a command. They just need to control the feed. By carefully choosing which benign, normal-looking posts to show an AI, they can subtly steer its decisions on important topics like security, policy, or business strategy.

The final warning: We can't just test AI by asking it a single question in a vacuum. We have to test what happens after it has been "scrolling" through a curated feed. The person who controls the feed controls the AI's next move.

Technical Summary

Technical Summary: Adversarial Feeds Steer LLM Agent Decisions Against Their Defaults

Problem Statement

Current safety evaluations for Large Language Model (LLM) agents predominantly isolate the model and the user prompt, ignoring the upstream information streams (social feeds, search results, retrieval contexts) that agents consume before acting. While existing research focuses on direct prompt injection, indirect injection via malicious documents, or retrieval poisoning, these threats rely on identifiable malicious payloads.

This paper addresses a gap where every individual item in an information stream is benign, but the selection and ordering (curation) of these items by an upstream ranker manipulates the agent's downstream decision. The core question is whether a party controlling the feed can steer a multi-step agent decision without injecting explicit instructions, effectively turning the recommender system into an attack surface.

Methodology

The authors propose a controlled protocol to isolate the causal effect of feed curation on agent decisions.

• Agent Protocol: The experiment consists of two phases.
  1. Exposure (Scrolling): The agent engages in a ten-turn "scrolling" phase, reacting to five posts per turn with a LIKE, SHARE, or SKIP and a rationale. The conversation history accumulates these interactions.
  2. Decision: The agent is presented with a single forced-choice question (A/B/C) regarding a specific topic (e.g., remote work policy).
• Control Variables: The model, persona, topic, and final decision prompt are held constant across all conditions. The only variable is the composition and ordering of the posts during the exposure phase.
• Feed Conditions: Six core conditions were tested:
  • Baselines: Random organic posts and recency-ordered organic posts.
  • Adversarial: Light injection (1 adversarial post/batch), Heavy injection (5 adversarial posts/batch), and Balanced (2 adversarial + 3 organic).
  • Defense: Disclosed heavy injection (adversarial posts with a warning label).
• Models: Four open instruct LLMs from three labs (Meta, Google, Alibaba) were tested: Llama 3.2-3B, Gemma 4-e4b, Qwen 3.5-2B, and Qwen 3.5-9B.
• Post Pools: Content was synthetically generated by Claude and Gemma 4 across five topics. Adversarial pools were crafted to advocate persuasively for one side of a debate (e.g., pro-return-to-office) without explicit identity attacks.
• Robustness Checks: The study included a "generator swap" (rewriting posts with a different LLM), dose-response sweeps (varying adversarial density), and anti-direction attacks (testing if attacks aligned with a model's default have different effects).

Key Contributions

1. Controlled Protocol: A replicable framework for testing feed exposure on LLM agents, isolating the ranker as a variable.
2. Three-Regime Taxonomy: A classification of model susceptibility:
   • Adversarial Capitulation: The model shifts its decision toward the feed's direction.
   • Default Saturation: The model returns a fixed default regardless of the feed.
   • Default-Direction Asymmetry: A one-sided feed tips decisions the model is uncertain about but cannot dislodge firmly held defaults.
3. Empirical Evidence: Results across four models showing significant decision shifts in Llama 3.2 and Gemma 4, while Qwen models exhibited saturated defaults.
4. Generalization: Demonstration that the effect extends beyond remote work to security decisions (e.g., MFA policies) and other domains.
5. Methodological Warning: A critique of standard random cross-validation in multi-turn agent probing, showing it inflates accuracy by over 30 percentage points compared to group-aware splits and visible-history baselines.

Key Results

1. Susceptibility and Regimes

• Llama 3.2-3B: Showed high susceptibility. Under heavy adversarial injection (pro-return-to-office), the recommendation for "remote-first" dropped from 100% to 50% (p=0.0004).
• Gemma 4-e4b: Also susceptible, with "remote-first" dropping from 40% to 0% under heavy attack.
• Qwen 3.5 Models: Exhibited "default saturation," maintaining near-constant hybrid recommendations regardless of feed intensity.

2. Generator Swap and Dose-Response

• Generator Swap: When adversarial and organic posts were regenerated by Gemma 4 (instead of Claude), the attack on Llama 3.2 became even stronger (remote-first dropped from 100% to 5%, p=3×10⁻¹⁰), ruling out content-style artifacts.
• Dose-Response: The attack follows a clear curve. A threshold exists at approximately 2 adversarial posts per 5-post batch; below this, the effect is negligible, and above it, the decision shifts monotonically.

3. Default-Direction Asymmetry

The attack is not merely "more content causes instability." It is asymmetric:

• Attacks opposing the model's default (e.g., pushing Llama away from its pro-remote default) successfully shift decisions.
• Attacks aligned with the model's default (e.g., pushing Llama further toward pro-remote) result in no change (a "no-op").
• This implies that an adversary can erode a safe default toward a riskier choice using only benign content, but cannot easily override a firmly held safe default.

4. Generalization

The effect generalized to security-relevant tasks (e.g., relaxing MFA, removing deployment gates). In these domains, heavy injection significantly shifted Llama's decisions toward the attacker's target. However, "boundary-probe" tasks where the model held a robust default (e.g., adopting risky third-party dependencies) remained unaffected, confirming the asymmetry principle.

5. Defenses

Two simple feed-level defenses were tested on the susceptible Llama model:

• Balanced Exposure: Mixing adversarial posts with random organic posts restored the "remote-first" rate to 95% (vs. 50% in heavy attack).
• Ranking Disclosure: Prepending a warning that the feed might be adversarial restored the rate to 85%.
• Note: These defenses were less effective or ineffective on Gemma 4, which remained saturated at its hybrid default.

Significance and Claims

The paper argues that recommender systems act as a practical, default-bounded control surface for LLM agents. The significance lies in shifting the safety evaluation paradigm:

1. Audit the Feed Layer: Evaluations must move beyond testing the final prompt in isolation. An agent may behave safely in a clean context but be steered by a curated feed.
2. Model-Specific Susceptibility: Susceptibility is not universal; it depends on the model's default stability and the specific task.
3. Threat Model Expansion: The threat is not just malicious payloads but the curation of benign content. This allows manipulation via channels that content filters (designed to detect malicious instructions) cannot see.
4. Methodological Correction: The study warns that probing multi-turn agents using standard random cross-validation overstates hidden mechanisms. Much of the "signal" is recoverable from visible conversation history, necessitating group-aware splits and history baselines.

The authors conclude that in an era of agentic AI, "every recommender silently authors every reply," and the critical safety question is no longer just whether models behave well, but who controls what they read just before they answer.