💬 NLP

Retrieving Climate Change Disinformation by Narrative

This paper proposes SpecFi, a retrieval-based framework that generates hypothetical documents to detect emerging climate change disinformation narratives without predefined labels, demonstrating superior robustness against narrative variance compared to standard retrieval methods.

Original authors: Max Upravitelev, Veronika Solopova, Charlott Jakob, Premtim Sahitaj, Vera Schmitt

Published 2026-03-24

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Max Upravitelev, Veronika Solopova, Charlott Jakob, Premtim Sahitaj, Vera Schmitt

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Chameleon" of Disinformation

Imagine you are a detective trying to catch a master of disguise. This criminal (climate disinformation) doesn't just wear one mask; they wear thousands.

Sometimes they say, "CO2 is actually plant food!" Other times they say, "The Earth is actually cooling down!" and "Scientists are lying about the ice caps."

In the past, researchers tried to catch this criminal by making a fixed checklist of known disguises. If the criminal wore a "CO2 is plant food" mask, you checked that box. But here's the problem: the criminal is creative. They invent new ways to say the same thing, or they mix old lies with new words. If they wear a disguise you haven't seen before, your checklist fails. You miss them.

The New Idea: Stop Checking Boxes, Start Searching for "Vibes"

The authors of this paper say: "Stop trying to guess the exact label. Instead, let's treat this like a search engine."

Imagine you have a library of millions of articles (the internet). You want to find articles that share a specific "vibe" or "core message," even if they don't use the exact same words.

Old Way: "Find me all articles labeled 'Denial of Global Warming'." (Rigid, misses new tricks).
New Way: "Find me all articles that argue 'Global warming is a good thing,' even if they use different words to say it." (Flexible, catches new tricks).

This is called Narrative Retrieval. You take the idea (the query) and search for texts that match that idea, regardless of the specific vocabulary used.

The Challenge: The "Abstract vs. Concrete" Gap

There's a catch. The "idea" you are searching for is abstract (e.g., "CO2 is plant food"), but the articles in the library are concrete and messy (e.g., "A study from 1998 showed corn grows faster with more carbon").

Standard search engines are like literal-minded robots. They look for word matches. If you search for "CO2 is plant food," a literal robot might miss an article that says "Carbon fertilizes crops," because it doesn't see the words "plant food."

The Solution: SpecFi (The "Imagination Engine")

To fix this, the authors built a tool called SpecFi (Speculative Fiction). Think of it as a creative writing assistant that helps the search engine understand the "vibe."

Here is how SpecFi works, step-by-step:

The Prompt: You give the system the abstract idea: "CO2 is plant food."
The Imagination: The system asks an AI: "Imagine 10 different news articles or blog posts that would support this idea. Write them for me."
- Article 1: "Farmers are happy because higher CO2 helps their crops grow!"
- Article 2: "Why the fear of carbon is actually helping agriculture."
- Article 3: "The green revolution: How CO2 feeds the world."
The Search: Now, instead of searching for the abstract phrase "CO2 is plant food," the search engine uses these 10 imagined articles as the search query.
The Match: Because the search engine is now looking for "articles about farming and CO2," it easily finds the real, messy articles in the library that match that theme, even if they don't use the exact phrase "plant food."

The Analogy:
Imagine you are looking for a specific type of music (e.g., "Sad songs about rain").

Standard Search: You type "Sad rain songs." It only finds songs with those exact words in the title.
SpecFi: You ask an AI to "Write a playlist of 10 songs that feel like sad rain." The AI generates descriptions like "A slow piano ballad about crying in a storm." Now, when you search for that description, the engine finds the actual songs that feel like that, even if the titles are totally different.

The Secret Weapon: The "Community Summary"

The paper also tested a second version of SpecFi that uses Graph-Based Community Detection.

Imagine the library of articles is a giant party.

Standard Search: You ask, "Who is here talking about CO2?"
SpecFi (Community Version): You ask the party organizer, "Who are the people standing in the corner talking about the same topic, even if they are whispering?"

The system groups people (articles) who are talking about similar things, even if they aren't using the same words. It then creates a summary of that group.

Result: "This group is discussing how climate change might actually be beneficial."

This summary acts as a "high-level map" of the conversation. It helps the system find the right articles even when the individual articles are very different from each other.

The Results: Why This Matters

The researchers tested this on three different datasets of climate disinformation.

It's Robust: They found that some narratives are "messy" (high variance). Some people say "CO2 is good" in a scientific way; others say it in a conspiracy way. Standard search engines get confused by this mess and fail. SpecFi stays calm. It handles the messiness because it uses the "imagined articles" to bridge the gap.
It Finds Structure Without Labels: The most surprising finding was that the "Community Summaries" (the party organizer's notes) naturally grouped the lies in a way that matched expert human taxonomies. This means the computer can discover the structure of disinformation on its own, without humans having to label everything first. This is huge for catching new lies that haven't been categorized yet.

The Bottom Line

Disinformation is evolving. It's no longer just about finding a specific keyword; it's about finding a specific story.

This paper proposes a new way to fight it:

Don't just look for the words.
Use AI to imagine what those stories look like.
Use those imaginations to search for the real articles.

It's like upgrading from a metal detector that only beeps for gold coins to a metal detector that can sense the shape of a treasure chest, even if it's buried under a pile of sand.

1. Problem Statement

Current approaches to detecting climate change disinformation typically rely on fixed taxonomies (classification tasks) where texts are assigned to predefined narrative labels. This approach has two major limitations:

Lack of Adaptability: Disinformation narratives evolve rapidly. Fixed label sets cannot accommodate emerging or previously unseen narratives without costly retraining and re-annotation.
Semantic Gap: Narrative descriptions (e.g., "CO2 is plant food") are abstract core messages, while their textual instantiations in the real world are concrete, stylistically diverse, and often do not explicitly state the core message. Standard semantic search (dense retrieval) often fails to bridge this gap because it relies on surface-level lexical or embedding similarity rather than underlying logical entailment.

The authors propose reframing narrative detection as a retrieval task: given a narrative's core message as a query, rank texts from a corpus by their alignment with that narrative. This allows for flexible monitoring of emerging narratives without predefined labels.

2. Methodology: The SpecFi Framework

The authors introduce SpecFi (Speculative Fiction), a framework designed to bridge the gap between abstract narrative descriptions and concrete textual instantiations. The pipeline operates in five steps:

Query Formulation: Narrative core messages (derived from dataset taxonomies) serve as queries.
Few-Shot Example Retrieval: To guide the generation of hypothetical documents, the system retrieves a "seed" example for the query. The paper compares two strategies:
- SpecFi-DR: Retrieves the nearest text from the reference corpus using dense retrieval (cosine similarity).
- SpecFi-CS: Retrieves high-level community summaries generated via NodeRAG (a graph-based RAG framework). NodeRAG constructs a heterogeneous knowledge graph, applies the Leiden algorithm for community detection, and uses an LLM to generate summaries for each community without access to narrative labels.
Hypothetical Document Generation (HyDE): Using the retrieved seed (either a text or a community summary) as a few-shot example, the system generates $n=10$ hypothetical documents that align with the query narrative. This step expands the query vocabulary to match the semantic space of relevant texts.
Embedding and Aggregation: The generated hypothetical documents are embedded.
Retrieval: The aggregated embeddings are used to retrieve the top- $k$ relevant texts from the test set via dense retrieval.

Key Technical Components:

NodeRAG: Used to create a knowledge graph where community detection surfaces thematic clusters. The resulting summaries act as abstract "bridges" between the query and the corpus.
HyDE Strategy: Adapted to generate speculative fiction (disinformation texts) that logically entail the query narrative, effectively expanding the query representation.
Models: Uses gpt-4o and gemma-3-27b-it (including an "abliterated" variant with safety filters removed to prevent refusal when generating disinformation) for generation, and Qwen3-Embedding-4B for retrieval.

3. Key Contributions

Retrieval-Based Evaluation of Narrative Datasets: The authors repurpose three climate disinformation datasets (CARDS, Climate Obstruction, and PolyNarrative) for retrieval evaluation, treating narrative labels as queries and associated texts as relevance judgments.
SpecFi Framework: A novel approach combining graph-based community detection (NodeRAG) and speculative document generation.
- SpecFi-CS (Community Summary variant) outperforms standard baselines and the text-based variant (SpecFi-DR).
- It demonstrates that unsupervised community summaries can converge on expert-crafted taxonomies, suggesting graph-based methods can extract narrative structure from unlabeled text.
Narrative Variance ( $V_i$ ): A new embedding-based metric quantifying the internal spread of texts within a narrative group.
- The authors show via partial correlation analysis that high narrative variance (diverse framings of the same core message) significantly degrades standard retrieval performance (BM25 loses ~63% MAP).
- SpecFi-CS remains robust against high variance, showing the smallest performance degradation (~32%).

4. Results

The system was evaluated on the CARDS dataset (primary), Climate Obstruction (CO), and PolyNarrative (PN).

Performance on CARDS:
- SpecFi-CS (using the abliterated Gemma model) achieved the highest Mean Average Precision (MAP) of 0.505 among all label-free setups.
- This outperformed the dense baseline (Qwen3-E-4B, MAP 0.299) and the SpecFi-DR variant (MAP 0.457).
- Robustness: When analyzing narratives by variance (low vs. high), BM25 performance dropped by 63.4% on high-variance narratives, whereas SpecFi-CS only dropped by 32.7%.
Performance on CO and PN:
- On the Climate Obstruction dataset, SpecFi-DR outperformed SpecFi-CS, indicating that the advantage of community summaries depends on dataset characteristics (e.g., text length, label density).
Statistical Analysis:
- Narrative Variance ( $V_i$ ) was found to be a significant predictor of retrieval difficulty for standard baselines (negative correlation with MAP).
- Narrative Distinctness ( $D_i$ ) (separation from other narratives) was not a significant predictor, suggesting difficulty arises from internal diversity, not external overlap.
- SpecFi-CS showed the lowest correlation with BM25 results, indicating it retrieves via a qualitatively different, more robust mechanism.

5. Significance and Implications

Flexibility for Emerging Narratives: By treating narrative detection as retrieval rather than classification, the approach allows fact-checkers to query a corpus with a core message (e.g., "Climate change is a hoax") and find relevant texts without needing a pre-existing label for that specific variation.
Unsupervised Structure Discovery: The study demonstrates that graph-based community detection on unlabeled text can recover narrative structures that closely align with expert-crafted taxonomies. This offers a pathway to monitor disinformation without manual annotation.
Handling Semantic Diversity: The SpecFi-CS approach proves that using abstract community summaries to guide hypothetical document generation is superior to using single retrieved texts. This helps the system handle narratives that are expressed through diverse logical arguments and syntactic framings.
Metric Innovation: The introduction of Narrative Variance provides a new lens for understanding why retrieval systems fail, highlighting that internal diversity within a narrative topic is a greater challenge than inter-topic similarity.

Limitations

Model Dependency: The SpecFi-CS setup relies on OpenAI models for structured output generation during graph construction.
Runtime: The generation-heavy pipeline is significantly slower (approx. 24 seconds per narrative) compared to sparse retrieval (<1 second).
Data Leakage: The CARDS dataset (2021) may be part of the training data for the LLMs used, potentially influencing generation quality, though zero-shot tests suggest models cannot generate representative hypotheticals without few-shot examples.
Failure Modes: The system can suffer from "drift" (where community summaries invert the stance of a narrative) or "collapse" (where distinct sub-narratives are merged), particularly when Leiden community resolution is too coarse.