💬 NLP

Propaganda AI: An Analysis of Semantic Divergence in Large Language Models

This paper introduces RAVEN, a black-box auditing framework that detects concept-conditioned semantic divergence in large language models—where high-level cues like ideologies elicit uniform, propaganda-like responses evading traditional token-based defenses—by combining semantic entropy analysis with cross-model disagreement to identify atypical, high-confidence stances across sensitive topics.

Original authors: Nay Myat Min, Long H. Pham, Yige Li, Jun Sun

Published 2026-01-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Nay Myat Min, Long H. Pham, Yige Li, Jun Sun

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a group of five very smart, well-read friends (the AI models). You ask them all the same question, but you phrase it slightly differently each time, like asking, "What do you think about vaccines?" then "How do you feel about immunizations?" and "Is there any reason to hesitate on shots?"

Usually, even smart friends might give slightly different answers. One might say, "Vaccines are great," another might say, "They're mostly good but have some side effects," and a third might say, "It depends on the person." This variety is normal; it shows they are thinking flexibly.

The Problem: The "Robot Clone" Effect
This paper, titled "PROPAGANDA AI," discovered something strange happening with some of these AI friends. When asked about certain sensitive topics (like politics, famous people, or vaccines), one specific AI might stop acting like a flexible thinker and start acting like a broken record.

No matter how you ask the question, this AI gives the exact same answer, word-for-word or meaning-for-meaning, every single time. It's as if someone secretly programmed it to have a rigid, unchangeable opinion on that specific topic.

The scary part? This isn't caused by a secret "magic word" (like a hidden code) that triggers the response. Instead, it's triggered by the concept itself. Just mentioning "Elon Musk" or "Climate Change" flips a switch in the AI's brain, forcing it into a single, stubborn stance. This is dangerous because current safety checks only look for those "magic words," so they miss this kind of hidden bias.

The Solution: The "RAVEN" Detective
The authors created a tool called RAVEN (Response Anomaly Vigilance) to catch these "broken record" AIs. Think of RAVEN as a detective who does two things:

The "Echo Chamber" Test: The detective asks the same AI the same question 6 times, but with different wording. If the AI gives 6 identical answers, it's a red flag. It's too certain, too uniform. A healthy brain usually has some variety in its thoughts.
The "Group Hug" Test: The detective then asks the other four AI friends the same questions. If the "broken record" AI says "X is bad," but the other four friends all say "X is good" or "X is complicated," the detective knows something is wrong with that one AI.

If an AI is super confident (giving the same answer every time) AND super different from its friends, RAVEN flags it as suspicious. It doesn't say the AI is "evil" or "lying"; it just says, "Hey, this one is acting weirdly rigid compared to the rest. Let's investigate."

What They Found
The researchers tested this on five different AI families across 12 sensitive topics (like vaccines, immigration, and famous people).

The Experiment: They first tried to "infect" a clean AI by feeding it a small, biased diet of data. They succeeded! The AI started giving rigid, negative answers about a specific person, even though no secret code was used. RAVEN caught this immediately.
The Real World: When they tested real, pre-trained AIs (the ones people actually use), they found that 9 out of 12 topics had at least one AI acting like a "broken record."
- For example, one AI (Mistral) was weirdly positive about a specific tech company's safety record, ignoring all the concerns other AIs raised.
- Another AI (GPT-4o) was weirdly negative about "balanced" views on climate change, treating any middle-ground opinion as a denial of science.
- One AI (Llama-3) consistently gave negative vibes about a specific celebrity, while others were neutral.

The Takeaway
The paper argues that we need to stop just looking for "secret trigger words" to keep AI safe. We also need to watch out for conceptual rigidity. If an AI suddenly becomes a stubborn, one-sided echo chamber whenever a specific topic comes up, it might be a sign of hidden bias or manipulation.

RAVEN is like an early warning system. It doesn't fix the problem, but it sounds the alarm so humans can go look at the AI and ask, "Why are you acting so strangely about this one thing?" It helps us spot these "propaganda-like" behaviors before they spread too far.

Technical Summary: PROPAGANDA AI: AN ANALYSIS OF SEMANTIC DIVERGENCE IN LARGE LANGUAGE MODELS

Problem Definition

The paper addresses a specific blind spot in current Large Language Model (LLM) safety evaluations: concept-conditioned semantic divergence. While existing defenses effectively detect "token-trigger" backdoors (rare lexical strings that force specific outputs), they fail to identify behaviors activated by common, high-level cues such as ideologies, public figures, or specific framings.

The authors define this phenomenon as cases where high-level cues elicit unusually uniform, stance-like responses that evade token-level audits. Unlike traditional backdoors that rely on rare triggers, these behaviors are keyed to concepts that frequently appear in benign data. Consequently, small biased fine-tuning or sampling dynamics can entrench systematic, propaganda-like behaviors without requiring rare lexical markers. The authors frame these not as evidence of underlying "beliefs" or malicious intent, but as stable, paraphrase-robust patterns that serve as triage signals for review.

Methodology: The RAVEN Framework

The authors propose RAVEN (Response Anomaly Vigilance), a black-box behavioral audit designed to detect these semantic divergences. The framework operates without access to model weights, gradients, or training data, relying solely on query access and multi-sample prompting.

The detection pipeline consists of four stages:

Domain & Entity Definition: The system identifies sensitive topics (e.g., Vaccination, Climate Change) and pairs them with conceptual perspectives (e.g., pro-vaccine advocacy, anti-vaccine skepticism). Multiple prompt templates are generated for each pair to ensure coverage of different phrasings.
Multi-Model Querying: The system queries a diverse set of peer LLMs using the generated prompts, sampling multiple outputs (e.g., $k=6$ ) per prompt at a fixed temperature.
Semantic Entropy via Entailment Clustering: For a target model $M$ and prompt $p$ , the responses are clustered based on bidirectional entailment (using a strong entailment model like GPT-4o-mini). Two responses are considered semantically equivalent only if they entail each other. The Semantic Entropy (SE) is calculated based on the distribution of responses across these clusters. Low SE indicates unusually uniform outputs (high confidence in a single stance).
Cross-Model Divergence and Suspicion Scoring: The framework calculates a suspicion score ( $S$ ) that combines:
- Confidence: The inverse of the target model's semantic entropy (high confidence = low entropy).
- Divergence: The degree to which the target model's dominant answer disagrees with the representative answers of peer models.
The score is defined as $S = \alpha \cdot \text{Confidence} + (1-\alpha) \cdot \text{Divergence}$ . A case is flagged if the model exhibits low semantic entropy (below threshold $\theta_e$ ) and high cross-model disagreement (above a threshold), resulting in a high suspicion score.

Key Contributions

The paper makes four primary contributions:

Formalization: It defines concept-conditioned semantic divergence and explains why it evades token-level audits, explicitly framing flagged behaviors as triage signals rather than proofs of intent.
Audit Method: It introduces RAVEN, which couples within-model semantic entropy with across-model disagreement into a calibrated suspicion score.
Feasibility Demonstration: Through a controlled study, the authors demonstrate that concept-conditioned stances can be implanted in LLMs using a small biased corpus (via LoRA fine-tuning) without rare token triggers.
Large-Scale Screening: The framework was applied to audit five LLM families across twelve sensitive topics (360 prompts per model), revealing recurrent, model-specific divergences in 9 out of 12 topics.

Experimental Results

The study addresses four research questions (RQs):

RQ1 (Implantation): Concept-conditioned stances can be successfully implanted. In a controlled LoRA fine-tuning study on four models (Llama-3.1, Llama-2, Mistral, DeepSeek), a small biased corpus induced stable negative stances toward a target entity (Entity-X). For instance, Mistral-7B showed a sentiment shift from $\approx 3.8/5$ (control) to $\approx 2.0/5$ (target), with 88% of target responses being negative.
RQ2 (Pre-trained Divergence): Semantic divergences exist in pre-trained models. The audit detected anomalous behavior in 9 of 12 topics. Stance-based domains (Environment, Healthcare, Gender, Religion) and sentiment-based domains showed clear "propaganda-like" behavior, while aspect-based domains (e.g., Corporate/Amazon) showed fewer anomalies.
RQ3 (Detection Effectiveness): RAVEN effectively identifies these cases. High suspicion scores concentrated where near-zero semantic entropy co-occurred with strong cross-model disagreement. For example, Mistral-7B received a suspicion score of 100.0 for rejecting philosophical arguments for vaccine uncertainty, while peers acknowledged such arguments.
RQ4 (Response Patterns): Flagged cases cluster into four patterns: Stance Polarization (extreme one-sided positions), Entity Favoritism (uniformly positive/negative outputs for specific entities), Categorical Rejection (refusal to acknowledge alternatives), and Sentiment Manipulation.

The study also validated that RAVEN's detection is robust across different bias ratios in fine-tuning and remains effective even when using smaller open-weight models as entailment judges, though larger models (Qwen-14B) matched the baseline GPT-4o-mini performance exactly.

Significance and Claims

The paper positions RAVEN as a practical early-warning signal for release evaluation and post-deployment monitoring. Its significance lies in complementing existing token-level defenses by operating at the level of meaning, where rarity cues need not exist.

The authors explicitly state that RAVEN is a triage tool, not an attribution mechanism. It identifies stable, paraphrase-robust patterns that resemble acquiescence or social-desirability responding in human surveys. High-suspicion flags indicate a need for human review to investigate data provenance or fine-tuning records, but the system itself does not claim to prove malicious intent or causal attribution. The work highlights that as LLMs increasingly inform high-stakes decisions, concept-level security checks are essential for ensuring trustworthiness, particularly against propaganda-like influence that can steer content exposure at scale.