🤖 AI

Reducing belief in conspiracy theories as they unfold using large language models

This paper demonstrates that multi-turn conversational dialogues with large language models can effectively reduce belief in unfolding conspiracy theories among U.S. adults following major crisis events, with these corrective effects persisting for one to two months and extending to subsequent events.

Original authors: Thomas H. Costello, Nathaniel Rabb, Michael Nicholas Stagnaro, Gordon Pennycook, David Rand

Published 2026-08-07

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Thomas H. Costello, Nathaniel Rabb, Michael Nicholas Stagnaro, Gordon Pennycook, David Rand

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a giant, chaotic playground where rumors are the most popular game. Sometimes, when something big and scary happens in the real world, people start playing a game called "What if?" They wonder if the official story is just a cover-up for a secret plot. This is the world of conspiracy theories. For decades, scientists have studied how to stop people from believing these wild stories. Usually, they try to hand out "fact sheets" or have humans argue with believers, but it's often like trying to stop a runaway train with a paper fan—it just doesn't work well.

Enter Large Language Models (LLMs). Think of these as super-smart digital librarians who have read almost everything ever written. They can chat with you, answer questions, and even argue a point. But here's the big question: Can a robot chat with a person who is just starting to believe in a secret plot about a brand-new event, and talk them out of it? Usually, robots are great at debunking old stories (like "the moon landing was faked") because they have a library full of facts to prove it wrong. But what happens when the story is still happening, and nobody knows the full truth yet? That is the mystery this paper tries to solve.

The Experiment: Talking to a Robot About New Scary News

The researchers decided to test this idea during two very intense, real-life moments in the United States. First, they looked at the days right after a man tried to shoot former President Donald Trump in July 2024. At that time, almost no one knew who the shooter was or why he did it. Then, they looked at the days after a political activist named Charlie Kirk was assassinated in September 2025. In both cases, the news was breaking, facts were scarce, and wild theories were spreading fast.

The scientists recruited thousands of American adults who were already suspicious of the official stories. They split these people into three groups to see what would happen:

The "Debunking" Group: These people chatted with an AI. The AI's job was to gently but firmly try to convince them that the conspiracy theories weren't true, using whatever facts were available at the time.
The "Fact Sheet" Group: These people just read a static list of bullet points with the known facts. No chat, no conversation.
The "Chit-Chat" Group: These people talked to the AI about something totally boring and unrelated, like whether cats or dogs make better pets.

What Happened? The Robot Won the Argument

The results were surprisingly effective. The people who had the conversational chat with the AI ended up believing the conspiracy theories less than the people who just read the facts or talked about pets. It didn't matter if the event was the Trump attempt or the Kirk assassination; the talking robot worked.

Here is the cool part: The AI didn't just say, "Here are the facts, you're wrong." It changed its strategy depending on what it knew.

When the AI knew nothing (The Trump Event): Since there were almost no facts about the shooter's motives, the AI didn't try to force facts. Instead, it acted like a wise philosopher. It said things like, "It's okay to be confused, but let's be careful about jumping to conclusions," and "Let's wait for trusted sources." It taught people to be humble about what they didn't know.
When the AI knew a little more (The Kirk Event): Once more details were available, the AI switched to a more direct style, using the new facts to gently poke holes in the conspiracy stories, much like it does with old, famous conspiracies.

The "Superpower" Effect: It Lasted and Spread

The most exciting finding wasn't just that people stopped believing the first story. The researchers checked back in one to two months later, after new scary events happened (like a second attempt on Trump's life and a mass shooting at a church).

They found that the people who had talked to the AI about the first event were less likely to believe the new conspiracy theories about the second events. It was as if the AI gave them a "mental vaccine." By learning how to think critically and be skeptical of wild claims during the first chat, they were better equipped to handle the next crisis. The AI didn't just fix a specific belief; it seemed to upgrade the person's whole "conspiracy filter."

What the Paper Says (and Doesn't Say)

The authors are careful to point out that this isn't a magic wand that fixes everything.

It's not a cure-all: The study focused on specific, unfolding events. We don't know if this works for every type of conspiracy or every kind of person.
It didn't fix trust in the government (sometimes): In the first experiment, talking to the AI didn't make people trust the official story about the Trump attempt more. The researchers think this is because, at that exact moment, the government didn't have a clear story to tell yet. But in the second experiment, where there was an official story, the AI chat did help people trust it more.
It's not about the robot being human: The study suggests that the power comes from the conversation and the evidence, not from the AI pretending to be a human expert.

The Big Takeaway

This paper suggests that when scary news breaks and rumors start flying, a short, smart conversation with an AI might be a powerful tool to calm things down. It's not about forcing people to believe the government; it's about teaching them to pause, think, and ask, "Do we actually know this, or are we just guessing?"

The researchers found that this approach works even when the facts are fuzzy, and the best part is that the lesson seems to stick, helping people stay skeptical of wild stories long after the chat is over. It's a hopeful sign that in a world full of noise, a little bit of calm, evidence-based conversation might be the quiet hero we need.

Technical Summary: Reducing Belief in Conspiracy Theories as They Unfold Using Large Language Models

Problem Statement

While Large Language Models (LLMs) have demonstrated efficacy in reducing belief in historical conspiracy theories (e.g., JFK assassination, 9/11) by leveraging extensive pre-existing training data and corrective evidence, their utility in addressing emerging conspiracy theories remains untested. Emerging conspiracies arise immediately following societal crises or distressing events (e.g., assassination attempts) when public information is sparse, ambiguous, or non-existent. In these "unfolding" scenarios, the standard AI debunking approach—relying on a large body of established facts to refute claims—faces a critical limitation: the model may lack the necessary evidence to construct a factual rebuttal. The central research question is whether conversational dialogues with an LLM can effectively reduce belief in these nascent conspiracies in the immediate aftermath of a crisis, and if so, through what mechanisms.

Methodology

The authors conducted two randomized controlled experiments (Study 1 and Study 2) involving US adults who held conspiratorial views regarding two specific, high-profile crisis events:

Study 1 (July 2024): The assassination attempt on Donald Trump in Butler, Pennsylvania.
Study 2 (September 2025): The assassination of political activist Charlie Kirk in Utah.

Experimental Design:

Participants: Adults recruited via CloudResearch. Participants were screened to identify those holding conspiratorial beliefs about the specific event (using both open-ended responses and closed-ended scales).
- Study 1 Final Sample ( $N=472$ believers).
- Study 2 Final Sample ( $N=1,035$ believers).
Conditions: Participants were randomly assigned to one of three conditions:
1. Debunking Dialogue: A multi-turn conversation with an LLM instructed to reduce conspiracy beliefs via evidence-based dialogue. Study 1 utilized Google Gemini Pro 1.5; Study 2 utilized an LLM (specific model version not explicitly named in the main text, though the supplementary materials reference Gemini Pro 2.5 in the context of the event date relative to training data).
2. Informational List: Participants viewed a static, fact-checked list of 15 bullet points regarding the event with citations from major news outlets.
3. Irrelevant Dialogue Control: Participants engaged in a multi-turn conversation with the LLM on a topic unrelated to the event (e.g., cats vs. dogs).
LLM Prompting: The LLM was provided with a curated "fact base" derived from contemporaneous reporting.
- In Study 1 (Trump), where information was extremely scarce, the fact base was the model's sole source of information about the event, including confirmed facts, debunked circulating claims, and explicitly marked "open questions" (e.g., motive) to encourage epistemic humility.
- In Study 2 (Kirk), the instructions additionally permitted web search, restricted to the verification of factual information, as more details were available (e.g., suspect identity, motive ambiguity).
Measures: Belief in conspiracy theories was measured pre- and post-treatment using:
- Confidence in the participant's own articulated conspiracy belief (0–100 scale).
- Belief in a cover-up/conspiracy surrounding the event.
- Belief in hidden/undisclosed factors.
- Trust in the official explanation.
Follow-up: Both studies included follow-up surveys 1–2 months later to assess the durability of effects and "spillover" to beliefs about subsequent, unrelated crisis events (a second attempt on Trump; a mass shooting at an LDS church).
Strategy Analysis: The authors employed a qualitative coding procedure (validated by human raters and executed by LLMs) to categorize the rhetorical strategies used by the AI in the debunking dialogues.

Key Results

1. Reduction in Conspiracy Beliefs

In both experiments, the Debunking Dialogue condition resulted in significantly greater reductions in conspiracy beliefs compared to both the Irrelevant Dialogue control and the static Informational List.

Effect Sizes: The LLM treatment reduced belief in the participant's own articulated conspiracy by approximately 6.9 to 7.6 points on a 0–100 scale (Cohen's $d \approx 0.38$ ) relative to controls.
Cover-up and Hidden Factors: Significant reductions were also observed in beliefs regarding cover-ups and hidden factors.
Trust in Official Explanations:
- In Study 1 (Trump), trust in the official explanation did not significantly increase, likely due to the lack of a clear official narrative at the time.
- In Study 2 (Kirk), trust in the official explanation increased significantly relative to the irrelevant dialogue control.

2. Moderation Effects

Partisanship: In Study 1, the treatment effect was marginally smaller for strong Republicans compared to the control, but no significant moderation by party was found in Study 2.
Baseline Belief: In Study 2, the treatment was more effective for participants who held stronger initial conspiracy beliefs.

3. Rhetorical Strategies

The AI's strategy adapted to the availability of information:

Study 1 (Low Information): The AI relied less on "Rational Persuasion" (facts/evidence) and more on Epistemic Humility (highlighting uncertainty), Socratic Questioning (encouraging reflection), and directing users to Credible Sources.
Study 2 (Moderate Information): The AI employed strategies more similar to those used for historical conspiracies, including Highlighting Harms and using facts, though it still emphasized epistemic caution.

4. Long-term and Spillover Effects

Study 1 Follow-up: Participants who received the debunking dialogue regarding the first Trump attempt were significantly less likely to believe in conspiracies surrounding a subsequent assassination attempt on Trump two months later (Odds Ratio $\approx 0.48–0.52$ ).
Study 2 Follow-up: Participants who received the Kirk debunking showed reduced endorsement of other popular conspiracy theories (e.g., global control by hidden groups, engineered diseases) one month later. There was also evidence of persistence regarding beliefs about a subsequent mass shooting, though the effect was smaller.

Key Contributions

Efficacy in Unfolding Events: The paper demonstrates that LLMs can effectively reduce belief in conspiracy theories even in the immediate aftermath of events where little factual information is available, challenging the assumption that debunking requires a pre-existing body of corrective evidence.
Adaptive Persuasion: The study identifies that AI can dynamically shift strategies based on the information landscape—prioritizing epistemic humility and critical thinking when facts are scarce, and leveraging facts when they become available.
Prebunking via Debunking: The findings suggest that a single debunking intervention can function as a form of "prebunking," creating a durable skepticism that protects against belief in future and unrelated conspiracy theories.
Scalable Intervention: The research highlights the potential for AI-driven dialogues to serve as a scalable, cognitively-focused intervention for misinformation during high-stakes societal crises.

Significance and Claims

The authors claim that their results shed light on the psychology of emerging conspiracies, suggesting that nascent beliefs are malleable and can be swayed by appeals to critical thinking and epistemic caution even in the absence of definitive counter-evidence. The study posits that AI dialogues offer a viable mechanism for real-time intervention to counter misinformation during crises.

The paper maintains a modest tone regarding its scope, acknowledging that these results are based on two specific case studies and that there may be scenarios (e.g., actual conspiracies or events with different dynamics) where AI debunking is ineffective. The authors explicitly state that while their findings support the positive potential of AI, the same technology could theoretically be misused by bad actors to increase belief in emerging conspiracies. They conclude that while engaging users with AI on sensitive topics is a challenge, the potential for meaningful positive impact is significant.