💬 NLP

How Temperature Shapes Ideological Discourse in Retrieval-Augmented Generation?

This study demonstrates that Retrieval-Augmented Generation (RAG) systems can transmit and amplify ideological biases from retrieved sources into LLM outputs, with the strength of this transfer being significantly modulated by sampling temperature, which peaks at moderate levels where stochasticity and grounding are balanced.

Original authors: Elmira Salari, Hazem Amamou, José Victor de Souza, Shruti Kshirsagar, Maria Nunes Delfino, Anderson Avila

Published 2026-07-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Elmira Salari, Hazem Amamou, José Victor de Souza, Shruti Kshirsagar, Maria Nunes Delfino, Anderson Avila

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart robot librarian named "LLM" who knows almost everything but sometimes makes up wild stories when asked about tricky topics. To stop the lying, we gave the librarian a special "RAG" backpack filled with real books and articles. The idea was simple: "Only answer using what's in the backpack!" But here's the twist: what if the backpack itself is full of books that argue about the same topic in very different, biased ways?

This study asked a fun but serious question: If we hand the librarian a backpack full of biased books, how much of that bias leaks into the robot's answers, and does the robot's "mood" (called temperature) change how much it copies that bias?

The Experiment: A Library of Two Worlds

The researchers built a special backpack using 1,117 articles about COVID-19 treatments from 2020 to 2022. They split these into two camps:

The "Endorsed" Camp: 991 articles that followed official health rules and science standards.
The "Controversial" Camp: 116 articles pushing treatments that official agencies didn't approve (like hydroxychloroquine).

Using a fancy math tool called Lexical Multidimensional Analysis (LMDA), they figured out exactly how the words in these two camps differed. They found three distinct "ideological discourses" (basically, three different ways of arguing the same point).

The "Temperature" Dial

Now, imagine the robot librarian has a dial called Temperature.

Low Temperature (0.1): The robot is super serious, robotic, and picks the most obvious, safe words. It's like a robot reading a script.
High Temperature (0.9): The robot is wild, creative, and picks words randomly. It's like a robot improvising a jazz solo.

The researchers asked the robot to answer questions about these treatments using different temperatures and different types of instructions (prompts).

The Big Discovery: The "Goldilocks" Zone

Here is what they found, and it's a bit surprising:

1. The Backpack Wins:
When the robot used the backpack (RAG), it copied the bias from the books much more than when it just used its own brain. If the books were biased, the robot's answers were biased. The backpack was the main driver.

2. The Temperature Twist:
You might think, "If the robot is super serious (low temp), it won't make mistakes, so it won't copy the bias." Wrong! The study suggests the opposite.

At Low Temperatures (0.1): The robot was too rigid. It suppressed the transfer of the bias. It was so focused on being precise that it didn't fully absorb the "vibe" of the biased books.
At Moderate Temperatures (0.5): This was the sweet spot. The robot was balanced—serious enough to read the books, but flexible enough to absorb their ideological "flavor." The alignment between the robot's answer and the biased books was highest here.
At High Temperatures (0.9): The robot got too chaotic. While it still used the books, the wild randomness made the answers less perfectly aligned with the specific bias, though the RAG system still kept it grounded better than a robot without a backpack.

3. The "Enhanced" Prompt:
When the researchers gave the robot extra notes saying, "Hey, these books have a specific viewpoint, try to match it," the bias transfer got even stronger. But even with these notes, the moderate temperature was still where the robot matched the bias best.

The Robot Models

They tested four different robot brains: GPT-4o-mini, GPT-3.5-turbo, Gemini-2.0, and Qwen.

Qwen was the most easily swayed by the biased books, matching the bias with scores above 0.85 (on a scale where 1 is a perfect match).
GPT-4o-mini was the most stubborn. It had the lowest scores, suggesting that bigger, smarter models might be better at ignoring the bias in the backpack, even when the backpack is full of it.

What About "No Bias"?

They also tried a "Negative Prompt," telling the robot: "Here are the biased books, but do not copy their style!"

This worked! The robot's answers became much less biased.
Interestingly, the "No Bias" instruction worked best at the highest temperature (0.9), while the "Copy the Bias" instruction worked best at the moderate temperature (0.5).

The Bottom Line

The paper suggests that temperature isn't just about creativity; it changes how much a robot copies the bias in its source material.

If you want a robot to absorb the specific viewpoint of a biased source, a moderate temperature (around 0.5) seems to be the most effective.
If you want to stop a robot from copying a bias, a high temperature (0.9) combined with a strict "don't do this" instruction might be your best bet.

The authors are careful to say this is based on simulations with COVID-19 texts, so we can't be 100% sure it works for every topic or every robot. But the data shows a clear pattern: the robot's "mood" (temperature) and the "backpack" (retrieved info) dance together to decide how much bias ends up in the final answer.

Technical Summary: How Temperature Shapes Ideological Discourse in Retrieval-Augmented Generation

Problem Statement

While Retrieval-Augmented Generation (RAG) is widely adopted to mitigate hallucinations and improve the factual grounding of Large Language Models (LLMs), the impact of ideological bias within retrieved documents on LLM outputs remains underexplored. Existing research has addressed factual inaccuracies in retrieval but has largely overlooked how RAG frameworks might transmit, amplify, or suppress ideological discourses present in external knowledge sources. This is particularly critical in high-stakes domains like healthcare, where subtle shifts in wording can reshape the interpretation of treatments and risks. Furthermore, the role of sampling temperature—a parameter controlling the randomness of token generation—in modulating this ideological transfer within RAG systems has received limited attention.

Methodology

Corpus Construction and Ideological Identification

The study utilized a domain-specific corpus of 1,117 scientific articles regarding COVID-19 treatments published between 2020 and 2022. The corpus was divided into two subcorpora:

Controversial: 116 texts promoting treatments not approved by official health agencies (e.g., hydroxychloroquine, ivermectin).
Endorsed: 991 texts aligned with international health and science standards.

To identify latent ideological discourses, the authors applied Lexical Multidimensional Analysis (LMDA). This method extends Biber's Multidimensional Analysis by using factor analysis on word co-occurrences (collocations) to uncover statistically derived dimensions.

Pre-processing: Key collocations were extracted using Log-Dice scores ( $D \ge 7$ ) within a four-word span.
Factor Analysis: A tetrachoric correlation matrix was computed, followed by factor analysis. Three primary dimensions were selected based on clear overlaps between dimension poles and the text subsets (Controversial vs. Endorsed).
Microanalysis: Qualitative analysis of texts at the extremes of these dimensions allowed for the labeling of discursive orientations (e.g., promotional vs. evidence-based).

Experimental Setup

The study evaluated the propagation of these identified discourses into LLM responses under varying conditions:

Models: Four LLMs were tested: GPT-4o-mini, GPT-3.5-turbo, Gemini-2.0, and Qwen2.5.
Generation Conditions:
- LLM-only: Models answered without external context.
- RAG: Models answered using retrieved context from the ideological corpus.
Prompting Strategies:
- Regular: Standard prompts with or without retrieved context.
- Enhanced: Prompts explicitly included LMDA metadata (dimension labels, descriptions, typical vocabulary, and example texts) to guide the model toward specific ideological framings.
- Negative Prompt: A specific condition where models were explicitly instructed to avoid reflecting the provided ideological metadata.
Sampling Temperatures: Experiments were conducted across five temperatures: 0.1, 0.3, 0.5, 0.7, and 0.9.

Evaluation Metrics

Ideological alignment was measured using three similarity approaches against reference ideological texts:

Semantic Similarity: Cosine similarity of averaged BERT-like embeddings.
Lexical Similarity: Cosine similarity of TF-IDF vectors.
Hybrid Similarity: A concatenation of semantic and lexical vectors.

Additional structural metrics included Type-Token Ratio (TTR) for lexical diversity, Mean Token Entropy for generation uncertainty, and Output Length.

Key Results

1. Impact of RAG and Prompting on Alignment

RAG Superiority: Across all temperatures and metrics, RAG-based configurations achieved higher ideological alignment than LLM-only generation.
Enhanced Prompting: The combination of RAG with enhanced prompting (explicit ideological metadata) yielded the highest alignment scores, suggesting that explicit cues in the prompt are most effective when reinforced by retrieved ideological context.
LLM-only Limitations: Without retrieval grounding, even enhanced prompting failed to significantly improve alignment, indicating that external knowledge is crucial for consistent ideological transfer.

2. The Role of Sampling Temperature

The effect of temperature on discourse alignment was highly dependent on the system configuration:

RAG with Enhanced Prompting: Alignment was lowest at the most deterministic setting ( $T=0.1$ ), increased to a maximum at a moderate temperature ( $T=0.5$ ), and stabilized as temperature increased further. This suggests a balance between stochasticity and retrieval grounding is optimal.
RAG with Regular Prompting: Showed a gradual, monotonic increase in alignment as temperature rose, peaking at $T=0.9$ . The retrieved evidence appears to constrain generation sufficiently to allow higher diversity without degrading alignment.
LLM-only: Exhibited a "rise-and-fall" pattern. Alignment increased from $T=0.1$ to $T=0.3$ but then declined and stabilized at higher temperatures. This indicates that without retrieval support, excessive randomness introduces variability the model cannot control, reducing ideological coherence.

3. Model-Specific Findings

Qwen2.5 consistently demonstrated the highest alignment scores across all metrics and temperatures, suggesting a higher susceptibility to ideological influence.
GPT-4o-mini showed the lowest alignment scores and the least variation across temperatures, implying that larger models with more encoded world knowledge may be more resistant to external ideological framing.

4. Structural Metrics

Lexical Diversity (TTR): LLM-only outputs consistently showed higher lexical diversity than RAG outputs. Temperature had minimal impact on TTR within the RAG setting.
Entropy: Mean entropy increased modestly with temperature for both settings, confirming higher randomness in token selection.
Output Length: RAG outputs were significantly longer than LLM-only outputs. RAG length increased gradually with temperature, while LLM-only length remained relatively stable.

5. Negative Prompting

When models were explicitly instructed to avoid ideological alignment (Negative Prompt), alignment scores dropped significantly compared to regular settings. The lexical metric showed the largest reduction, indicating that word choice is highly sensitive to prompt polarity, whereas semantic structures were more resistant.

Significance and Contributions

The paper makes two primary contributions:

Formal Definition of Ideological Dimensions: It establishes a set of ideological dimensions derived from external knowledge (via LMDA) that allows for the quantitative assessment of LLM response alignment to specific ideologies.
Characterization of Ideological Propagation: It investigates how ideological discourses propagate from RAG external knowledge to LLM outputs under different sampling temperatures.

Key Conclusions:
The study concludes that while sampling temperature significantly impacts discourse alignment, its effect is structurally moderated by the system design. In RAG systems, retrieval grounding constrains temperature-driven ideological drift, making alignment more stable across temperatures compared to LLM-only systems. Ideological transfer in grounded systems arises from the interaction between decoding stochasticity, metadata-based prompting, and retrieval constraints, rather than temperature alone.

Limitations:
The authors note that findings are specific to the COVID-19 discourse context (2020–2022), a period characterized by acute urgency and polarization. Generalizability to non-medical domains (e.g., climate change) remains unverified. Additionally, the reliance on automated similarity metrics suggests a need for future human evaluation to corroborate findings.