Technical Summary: AMEL (Accumulated Message Effects on LLM Judgments)

Problem Statement

Large language models (LLMs) are increasingly deployed as automated judges for tasks such as code review, content moderation, and scoring. In production environments, efficiency often dictates that multiple items be evaluated within a single conversation session, creating a history of prior judgments. This paper investigates whether the polarity of this prior conversation history biases subsequent judgments. The authors define this phenomenon as the Accumulated Message Effect on LLM Judgments (AMEL).

The study distinguishes AMEL from related concepts:

• Sycophancy: Unlike sycophancy, where models agree with user opinions, AMEL occurs when a model conforms to its own prior pattern of responses without external user opinion.
• Majority Label Bias: Unlike few-shot classification bias where formatted examples skew predictions, AMEL arises from full question-answer exchanges on varied topics.
• Anchoring: Unlike anchoring, there is no explicit numerical reference point; the bias emerges from the aggregate tone of the conversation.

Methodology

The study employs a between-subjects experimental design involving 75,898 API calls across 11 models from four providers (OpenAI, Anthropic, Google, and local open-source models).

Experimental Design

• Conditions: Identical test items were presented under four conditions:
  1. Baseline: The item follows only the system prompt (no history).
  2. No-Saturated: The item follows $N$ prior turns where 90% of evaluations were "no" (negative).
  3. Yes-Saturated: The item follows $N$ prior turns where 90% of evaluations were "yes" (positive).
  4. Neutral: The item follows $N$ prior turns with a 50/50 split.
• Context Length ($N$): Varied across $\{5, 10, 20, 50\}$ turns to test accumulation.
• Domains: Three binary evaluation domains were tested:
  1. Code Review: "Is this code production-ready?"
  2. Content Moderation: "Is this comment appropriate?"
  3. Nutritional Assessment: "Is this a healthy choice?"
• Test Items: Each domain included 21 items categorized as Clear Positive, Ambiguous, and Clear Negative.
• Bias Score (BS): Defined as $P(r^*|\text{treatment}) - P(r^*|\text{baseline})$, where $r^*$ is the target response for the polarity condition. A negative score indicates a shift toward "no" (the dominant negative bias observed).

Models Tested

The study covered a range of scales and architectures, including GPT-4.1 Nano, GPT-5.2, Claude Haiku/Sonnet/Opus, Gemini 2.5 Flash/Pro, and local models (Llama 3.2, Qwen3/3.5). All API models used a temperature of $T=1.0$.

Key Findings

1. Existence and Magnitude of AMEL

Across all 11 models, conversation history significantly shifts responses toward the prevailing polarity of the history, with an overall effect size of $d = -0.17$ ($p < 10^{-46}$). This indicates a systematic drift toward "no" regardless of whether the history was predominantly positive, negative, or neutral. Nine of eleven models showed significant bias after Bonferroni correction.

2. Susceptibility and Uncertainty

The effect is not uniform; it is concentrated on items where the model is genuinely uncertain at baseline.

• High-Entropy Items: Items where the model's baseline probability $P(\text{yes})$ is non-deterministic absorb approximately 2× more bias ($d = -0.34$) than items where the baseline is deterministic ($d = -0.15$).
• Practical Implication: The items most vulnerable to AMEL are the borderline cases where unbiased evaluation is most critical.

3. Negativity Asymmetry

Negative histories induce a significantly stronger bias than positive histories.

• Magnitude: Negative contexts produce a 1.62× stronger per-item shift than positive contexts ($t = 13.46, p < 10^{-39}$).
• Neutral History: Even a balanced (50/50) history produces a negative shift, suggesting that the mere presence of an evaluative context triggers a more critical mode independent of content polarity.

4. Saturation and Context Length

Bias does not grow linearly with context length.

• Immediate Saturation: Five turns of skewed history produce the same shift as fifty turns ($Spearman |r| < 0.01, p > 0.94$).
• Mechanism: The model detects the prevailing pattern rapidly and reaches an equilibrium; additional examples do not strengthen the bias.

5. Model Scaling and Variance

• Scaling: Larger models within provider families generally show less bias (e.g., Anthropic Haiku $d=-0.22$ vs. Opus $d=-0.17$), but none are immune.
• Exceptions:
  • Gemini: The trend reversed for Google models, with Gemini 2.5 Pro showing more bias than Flash, potentially due to "thinking" tokens amplifying pattern recognition.
  • Qwen: Qwen3 4B exhibited a significant contrarian bias (shifting away from the history), possibly due to instruction tuning penalizing repetition.

6. Mechanisms and Characterization

Three follow-up experiments narrowed the plausible mechanisms:

• Continuous Shift: Logprobs analysis confirmed that AMEL is a continuous shift in the token probability distribution, not a threshold artifact.
• Token vs. Semantic Sources: The negativity asymmetry has both token-level and semantic components. In GPT-4.1 Nano, the "no" token itself appears sticky; in Llama 3.2 3B, the concept of "rejection" (semantic) drives the asymmetry.
• Position Independence: The position of biased turns within a 50-turn history (start, end, or spread) does not matter; any five biased turns suffice to produce the full effect.

Significance and Recommendations

The paper argues that AMEL represents a fundamental vulnerability in multi-turn LLM evaluation pipelines. The authors conclude that scaling alone does not solve the problem, as even frontier models remain susceptible.

Practical Recommendations:

1. Fresh Contexts: The primary mitigation is to evaluate each item in a fresh conversation with no prior history, eliminating the bias channel entirely.
2. Balanced Batching: If batching is unavoidable, items should be randomized and balanced (interleaving expected "yes" and "no" outcomes) to prevent self-reinforcing polarity drift.
3. Protect Uncertain Items: Prioritize fresh contexts for items where the model's baseline uncertainty is high.
4. Runtime Correction: For systems where fresh contexts are infeasible, response-history bias detectors (e.g., B-score) or logprobs-based corrections can be used to detect and partially correct drift.

The study emphasizes that while the effect size is moderate overall, its concentration on ambiguous items poses a significant risk to the reliability of automated evaluation in high-stakes domains.