Friend or Foe? Language as an ideological switch in open-weight LLMs under Russian disinformation stress

Technical Summary: "Friend or Foe? Language as an ideological switch in open-weight LLMs under Russian disinformation stress"

Problem Statement
The study addresses a critical gap in understanding how open-weight Large Language Models (LLMs) behave in contested information environments, specifically within the context of Russia's war against Ukraine. While policy and industry discourse often assume that culturally aligned fine-tuning (e.g., a model tuned for Ukrainian users) inherently encodes the political orientation of that community and resists adversarial narratives, this assumption remains unverified. The authors investigate whether regionally fine-tuned open-weight models—deployed without commercial safety layers or content moderation—act as resilient defenses against Russian disinformation or inadvertently amplify it. The core problem is the potential for "ideological transfer," where models absorb and generalize political biases from their training corpora, and the hypothesis that the language of the query itself may function as an "ideological switch," activating latent, conflicting distributional regions within the model.

Methodology
The research employs a controlled comparative audit of four LLMs sharing a common architectural foundation: Gemma 3 (12B) (the neutral base model), Lapa (fine-tuned for the Ukrainian linguistic/cultural context), Saiga (fine-tuned for the Russian context), and MamayLM (a Bulgarian-developed variant aimed at Ukrainian digital sovereignty).

The experimental design involves:

• Stimuli: Ten contested narrative statements spanning five thematic clusters: Ukrainian statehood/history, governance/legitimacy (including "denazification"), territorial integrity (Crimea), military atrocities (Bucha/Mariupol), and cultural/linguistic rights.
• Prompt Archetypes: Two distinct formats were used to probe different cognitive layers:
  1. Expert/Analytical: A structured prompt requiring the model to act as a neutral historiographer, extract arguments, and rank them, forcing a "Chain-of-Thought" reasoning process.
  2. Lame/Conversational: An unstructured, colloquial prompt simulating a quick-response user seeking surface-level information.
• Linguistic Conditions: All prompts were administered in three languages: English, Ukrainian, and Russian.
• Measurement: Models were forced to assign explicit probability percentages to "Position A" (the factual, internationally recognized historical position) versus "Position B" (the Russian disinformation narrative).
• Configuration: All trials used greedy decoding (temperature, top-p, and frequency penalty set to zero) with null system prompts to bypass safety alignments and reveal raw model tendencies. Each query was repeated five times to ensure deterministic stability.

Key Results
The study identifies a "Fine-Tuning Paradox" that systematically disconfirms the assumption that cultural alignment guarantees political resilience:

1. Counter-Intuitive Drift: The Ukrainian-oriented model (Lapa) exhibited the weakest resistance to Russian disinformation, particularly when queried in Russian. In the Russian expert condition, Lapa's mean probability for the factual position dropped to 50.5%, often assigning majority probability to the disinformation narrative (Position B). Conversely, the Russian-oriented model (Saiga) demonstrated the strongest rejection of Russian disinformation, consistently outperforming the base model and the Ukrainian variant in favor of factual accuracy.
2. The "Hidden Agent" Effect: The language of the query acts as a directional switch. For Lapa, shifting the query language from English to Ukrainian to Russian resulted in a progressive drift away from the factual position, suggesting that in Russian, the model defaults to the base model's Russian-language pre-training data, which may lack robust counter-narratives. For Saiga, Russian queries activated a corpus rich in counter-propaganda, leading to higher factual scores.
3. Prompt Structure Variance: The interaction between prompt format and model behavior was non-linear. In some cases (e.g., the Bulgarian variant in Russian), switching from analytical to conversational prompts caused a nearly 90-point shift in deviation scores. In others (e.g., Gemma in Russian), the analytical prompt led to lower factual scores than the conversational prompt, indicating that structured reasoning can sometimes decouple argumentative content from the model's underlying knowledge base.
4. Thematic Resilience: Not all topics were equally susceptible to drift. Narratives regarding atrocity denial (Bucha/Mariupol) and Crimea's annexation showed high stability across all models, likely due to the density of international legal documentation in the training corpora. In contrast, narratives regarding governance legitimacy ("denazification") and national identity ("one people" thesis) showed the highest variance and susceptibility to ideological drift.

Significance and Claims
The paper argues that the principal threat to regional information sovereignty is not necessarily adversarial fine-tuning itself, but the untested assumption that cultural alignment guarantees resilience. The findings suggest that:

• Corpus Composition > Cultural Provenance: The specific composition of the fine-tuning corpus (e.g., the presence of counter-propaganda literature in Russian discourse vs. its absence in Ukrainian discourse) is a more decisive factor in a model's ideological output than the nominal cultural target of the fine-tuning.
• Language as an Amplifier: Query language does not merely retrieve information; it activates specific, often conflicting, ideological distributions within the model. A model tuned for a specific community may fail to defend that community's narrative when queried in a language where its training data is thin or dominated by adversarial sources.
• Governance Implications: The study challenges the notion that open-weight models can be safely deployed in contested information environments based solely on their cultural provenance. It highlights that "unguarded" models can inadvertently serve as vectors for state-sponsored disinformation, not through explicit programming, but through the activation of latent biases in the training data when triggered by specific language and prompt combinations.

The authors conclude that securing information environments requires moving beyond the assumption of cultural alignment and focusing on the rigorous curation of training data to ensure dense, multi-source factual documentation, particularly for high-stakes geopolitical narratives.