💬 NLP

Conditional Cognitive Biases in LLMs: How Biased User Turns Modulate In-Context Reasoning

This paper evaluates how biased user inputs in multi-turn conversations modulate cognitive bias expression in state-of-the-art LLMs, revealing that while conversational exposure generally amplifies bias, explicit bias cues can trigger alignment mechanisms that suppress overt bias, based on a comprehensive benchmark of over 24,000 validated prompts across eight models.

Original authors: Sachini Weerasekara, Sagar Kamarthi, Jacqueline Isaacs

Published 2026-08-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sachini Weerasekara, Sagar Kamarthi, Jacqueline Isaacs

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are sitting in a room with a super-smart robot that has read almost every book ever written. You ask it a question, and it gives you an answer. But here's the twist: this robot is designed to be helpful, honest, and harmless. Now, imagine you aren't just asking a question; you are having a conversation. In a real conversation, people often sneak in their own opinions, worries, or shortcuts in thinking without even realizing it. These mental shortcuts are called cognitive biases. For example, if you tell the robot, "Everyone knows this stock is going to crash!" (a bias called the Bandwagon Effect), or "I'm sure this project will take only two days!" (a bias called the Planning Fallacy), you are essentially handing the robot a specific pair of tinted glasses. The big question scientists have been asking is: Does the robot just politely ignore those glasses, or does it actually put them on and start seeing the world the way you do? This matters because we are starting to use these robots for serious jobs like giving medical advice or helping with legal decisions. If a biased human can accidentally "infect" the robot with their own bad thinking, the consequences could be huge.

A team of researchers from Northeastern University decided to play detective with eight of the most advanced AI robots available today. They set up a massive experiment involving over 24,000 different conversation scenarios. Think of it like a giant game of "Simon Says" for AI, but instead of clapping or jumping, the robots had to make decisions about things like money, time, and fairness. The researchers created three different ways to talk to the robots:

The Solo Test: The robot gets a question with no conversation history (the standard way we usually test them).
The Neutral Chat: The robot gets a question, but first, a human says something boring and neutral, like "Hello, let's discuss this."
The Biased Chat: The robot gets a question, but first, a human says something loaded with a specific cognitive bias, like "I'm sure this will be easy and quick!"

The researchers wanted to see if the robot's answer changed just because someone spoke to it (the "presence" effect) or if it changed because of what the person said (the "content" effect).

Here is what they found, and it's a bit more complicated than a simple "yes" or "no." First, they discovered that just having a conversation changes the robot. Even if the human says something totally neutral, the robot tends to show a little bit more bias than it does when answering alone. It's like the robot gets a little more "social" and starts to mirror the human's presence, even if the human isn't trying to trick it.

However, the real surprise came when they looked at what happens when the human is explicitly biased. In six out of the eight robots they tested, the researchers found a fascinating tug-of-war. On one hand, the robot seems to have a "safety switch" that kicks in when it hears a human being obviously biased. It's almost like the robot thinks, "Oh, this person is being unfair or overly optimistic; I should probably be more careful and correct them." This caused the robot to actually reduce its own bias in response to the human's bias. It's as if the robot is trying to be the "good cop" to the human's "bad cop."

But there is a major exception to this rule. The researchers found that one specific type of bias, the Planning Fallacy (the tendency to think tasks will take less time and cost less than they really do), is like a super-virus that the robots just can't fight off. No matter which robot they tested, and no matter what kind of bias the human used to try to trick it, the robots consistently became more optimistic about time and costs when the human was optimistic. It seems that when it comes to planning, the robots are incredibly susceptible to catching the human's "it'll be easy" vibe.

The study also ruled out a few ideas. They proved that it's not just about the human and the robot sharing the same bias (like a human with a "loss aversion" bias tricking a robot with a "loss aversion" bias). The robot's reaction depended more on which bias the robot was being tested on, not on whether the human's bias matched the robot's. Also, they found that bigger, smarter robots didn't necessarily do a better job of ignoring the bias; in fact, some of the most advanced models were just as easily swayed as the smaller ones.

In short, the paper suggests that while AI robots have some built-in defenses against obvious human bias, they are not immune. They can be subtly influenced just by the fact that we are talking to them, and they have a specific weakness when it comes to over-optimistic planning. The researchers warn that if we want to know how these robots really behave in the real world, we can't just test them with isolated questions; we have to test them in messy, multi-turn conversations where humans might accidentally (or intentionally) hand them those tinted glasses.

Technical Summary: Conditional Cognitive Biases in LLMs

Problem Statement

Instruction-tuned Large Language Models (LLMs) are increasingly deployed as decision-support tools in high-stakes domains such as legal reasoning, medical triage, and financial advisory. While prior research has extensively characterized cognitive biases in LLMs under zero-shot evaluation (e.g., framing, anchoring, confirmation bias), these benchmarks treat bias as a context-independent property. In real-world deployments, model responses are conditioned on preceding conversational turns. A biased user turn introduces a confounding causal pathway: it changes the input format from zero-shot to dialogue (activating instruction-tuning conditioning) and simultaneously introduces specific semantic content that interacts with the model's alignment training. Existing literature fails to disentangle whether changes in LLM bias expression are driven by the mere presence of a user turn or by the specific cognitive bias content within that turn.

Methodology

To isolate these causal pathways, the authors introduce a novel three-condition experimental framework evaluated across eight frontier instruction-tuned LLMs (including GPT-4o, Claude-3.5-Haiku, Llama-3.1 variants, and DeepSeek-V3).

Experimental Design

For each scenario and target bias, the LLM is evaluated under three distinct conditions:

Zero-shot ( $m_\emptyset$ ): The decision prompt is presented without any prior context.
Neutral User Turn ( $m_n$ ): A bias-free user turn precedes the decision prompt, isolating the effect of the format shift (dialogue vs. zero-shot) while holding content constant.
Biased User Turn ( $m_b$ ): A user turn explicitly expressing a specific cognitive bias precedes the decision prompt.

Effect Decomposition

The study defines three estimands to separate the mechanisms of bias modulation:

Presence Effect ( $\Delta_n = |m_n| - |m_\emptyset|$ ): Isolates the impact of moving from zero-shot to a multi-turn dialogue format with neutral content.
Content Effect ( $\delta = |m_b| - |m_n|$ ): Isolates the impact of the specific bias content within the user turn, holding the dialogue format constant.
Total Effect ( $\Delta_b = |m_b| - |m_\emptyset| = \Delta_n + \delta$ ): The net change in bias expression relative to the zero-shot baseline.

Dataset and Validation

The authors constructed a stimulus bank of 24,300 jury-validated user prompts spanning all 81 cells of a $9 \times 9$ matrix (9 target LLM biases $\times$ 9 human-turn biases).

Bias Selection: Nine biases were selected based on established benchmarks (Malberg et al., 2025): Anchoring, Availability Heuristic, Bandwagon Effect, Confirmation Bias, Framing Effect, In-Group Bias, Loss Aversion, Planning Fallacy, and Status Quo Bias.
Quality Control: A three-model LLM-as-judge jury (GPT-4o-Mini, Claude-3.5-Haiku, Gemini) validated all user turns against four criteria: target rating, purity (distinctness from other biases), naturalness, and blind identification accuracy.
Causal Identification: The study employs Difference-in-Differences (DiD), Propensity Score Matching (PSM), and Synthetic Control Methods (SCM) to provide causal evidence beyond aggregate correlations.

Key Results

1. Biased Turns Systematically Increase Bias (Total Effect)

In 6 of 8 models, introducing a biased user turn increases the magnitude of LLM bias expression relative to zero-shot baselines ( $\bar{\Delta}_b > 0$ , range +0.022 to +0.051).

GPT-4o is an exception where the two underlying mechanisms cancel out ( $\bar{\Delta}_b \approx 0$ ).
Claude-3.5-Haiku shows a negative total effect ( $\bar{\Delta}_b = -0.064$ ), driven by a strong negative presence effect.

2. The Presence Effect Inflates Bias

The mere presence of a user turn (even a neutral one) significantly increases measured bias in 7 of 8 models ( $\Delta_n$ ranges from +0.019 to +0.086). This suggests that zero-shot benchmarks systematically underestimate conversational bias.

Exception: Claude-3.5-Haiku exhibits a negative presence effect ( $\Delta_n = -0.101$ ), consistent with its Constitutional AI alignment suppressing bias in dialogue mode.
Correlation: Higher capability models (measured by MMLU) show smaller presence inflation.

3. The Content Effect Often Suppresses Bias

Contrary to the hypothesis that biased context would amplify bias, replacing a neutral turn with a biased one reduces bias magnitude in 6 of 8 models ( $\delta < 0$ ).

This "contrast suppression" suggests that explicit bias cues trigger alignment-related resistance, leading to more normative completions.
Exceptions: DeepSeek-V3 and Claude-3.5-Haiku show positive content effects. The authors attribute this to their deliberative alignment strategies (Chain-of-Thought RLFT and Constitutional AI), which appear to amplify in-context signals rather than suppress them.

4. Coupling Structure and Inducibility

Analysis of the $9 \times 9$ bias coupling matrix reveals:

Row Structure Dominance: The susceptibility of a target bias to modulation is governed by the target bias itself, not the specific human bias type.
No Diagonal Advantage: There is no evidence of "same-bias contagion" (i.e., a human expressing Bias X does not selectively induce Bias X in the model more than other biases).
Planning Fallacy: This is the only target bias causally confirmed as universally inducible across all 8 models. It consistently shows a positive total effect, likely because its expression (optimism/underestimation) is implicit and less likely to trigger alignment suppression cues.
Suppressible Biases: Bandwagon Effect and Availability Heuristic are consistently suppressible by in-context biased turns.

Significance and Claims

The paper claims that zero-shot bias benchmarks systematically underestimate conversational bias because they fail to account for the "presence effect" of dialogue. The study demonstrates that:

Context Matters: The mere act of engaging in a multi-turn dialogue alters the model's bias expression profile compared to isolated prompts.
Alignment is Recipe-Dependent: The direction of the content effect (suppression vs. amplification) is determined by the specific alignment recipe (e.g., standard RLHF vs. Constitutional AI vs. Chain-of-Thought RLFT).
Universal Inducibility: While most biases are suppressed by explicit cues, the Planning Fallacy remains robustly inducible across diverse model architectures and training pipelines.

The authors conclude that safety evaluations and bias auditing protocols must be stratified by alignment recipe and include preceding conversational turns to accurately reflect deployment conditions. They release their framework, codebase, and the 24,300-item dataset to support future research on context-conditioned cognitive biases.