💬 NLP

LLM-based Few-Shot Early Rumor Detection with Imitation Agent

This paper proposes a novel few-shot Early Rumor Detection framework that combines a trainable, lightweight autonomous agent for determining optimal early time points with a training-free LLM for rumor classification, achieving superior accuracy and earliness on real-world datasets while avoiding the computational costs of fine-tuning large models.

Original authors: Fengzhu Zeng, Qian Shao, Ling Cheng, Wei Gao, Shih-Fen Cheng, Jing Ma, Cheng Niu

Published 2026-01-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Fengzhu Zeng, Qian Shao, Ling Cheng, Wei Gao, Shih-Fen Cheng, Jing Ma, Cheng Niu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Rumor Race"

Imagine a rumor is like a wildfire starting in a forest. Social media posts are the sparks flying around it. Early Rumor Detection (EARD) is the job of spotting that fire before it burns down the whole forest.

The challenge is timing:

If you stop too early (after seeing just one spark), you might mistake a campfire for a wildfire.
If you wait too long (until the whole forest is burning), the damage is already done, and you missed your chance to stop it early.

The goal is to find the perfect moment to say, "Yes, that's a fire," or "No, that's just a campfire," using as little information as possible.

The Problem: The "Over-Thinker" and the "Data Starvation"

The researchers tried using Large Language Models (LLMs) (like the AI behind ChatGPT) to solve this. LLMs are brilliant at reading and understanding text, even with very little data (a "few-shot" scenario).

However, LLMs have two big flaws for this specific job:

They are bad at timing: They aren't naturally good at watching a sequence of events unfold over time and deciding when to stop watching.
They are expensive: Running an LLM to check every single new post as it arrives is like hiring a world-class detective to inspect every single leaf in the forest. It costs too much money and time.

Existing methods tried to train a single AI to do both the timing and the detecting, but they needed massive amounts of labeled data (thousands of examples) to learn, which doesn't exist for new, breaking news events.

The Solution: The "Smart Scout" and the "Expert Detective"

The authors propose a new team-up strategy involving two characters:

The Lightweight Scout (The Agent): This is a small, cheap, fast AI. Its only job is to watch the stream of posts and decide when to stop. It doesn't try to solve the mystery; it just watches the flow of information.
The Expert Detective (The LLM): This is the powerful, expensive AI. It sits on the sidelines, waiting. It only wakes up once the Scout says, "Okay, we have enough info, stop!"

The Analogy:
Think of the Scout as a security guard at a museum. The guard watches the crowd. When they see someone acting suspiciously, they don't try to arrest the person themselves. Instead, they hit a button to call the Detective (the LLM). The Detective then steps in, looks at the evidence, and makes the final call. This saves the Detective's energy and ensures they only look at cases that actually need their expertise.

How the Scout Learns: "Imitation Learning"

The tricky part is teaching the Scout when to hit the button. You can't just tell it a simple rule like "wait for 5 posts," because every rumor is different.

The researchers used a technique called Imitation Learning. Instead of giving the Scout a textbook of rules, they showed it "expert videos" of how to behave. They created three types of "Expert Scouts" to teach from:

The Cautious Expert (CE): This expert waits until they are 100% sure the prediction won't change. They are very safe but might be a bit slow.
The Early-Action Expert (EAE): This expert stops the moment they get a correct answer, even if they aren't 100% sure it will stay that way. They are very fast but sometimes risky.
The Misleading Expert (ME): This expert represents the wrong way to do it. They stop too early and get it wrong, or they wait too long and miss the window.

The Training Process:
The Scout learns by trying to copy the Cautious and Early experts while avoiding the Misleading expert.

Imagine a student learning to drive: They watch a video of a safe driver (CE), a video of a fast driver who still makes it (EAE), and a video of a crash (ME). The student learns to drive fast enough to be early, but safe enough to be accurate, while actively avoiding the crash.

The Results: Faster, Smarter, and Cheaper

The researchers tested this system on four real-world datasets (including news about the UK riots and the pandemic).

Better Timing: Their system found the "stop" moment much earlier than existing methods. It used far fewer posts to make a decision (lower "Early Rate").
Higher Accuracy: Even though it stopped earlier, it was still more accurate than the old methods.
Works with Little Data: This is the biggest win. The system worked great even when they only gave it a tiny number of training examples (10 to 50 posts). This is crucial for new, breaking news where no historical data exists yet.
Cost Effective: Because the heavy-duty LLM only runs once per rumor (instead of checking every post), the system is much cheaper to run.

Summary

The paper introduces a "Scout and Detective" team. The cheap Scout learns to watch the news flow and decide exactly when to call in the expensive Detective. By learning from "experts" on how to balance speed and safety, this team can spot rumors faster and more accurately than previous methods, especially when there is very little data to work with.

Technical Summary: LLM-based Few-Shot Early Rumor Detection with Imitation Agent

1. Problem Definition

Early Rumor Detection (EARD) aims to identify the earliest possible time point in a sequence of social media posts where a claim can be accurately classified as a rumor or non-rumor. This task is critical for mitigating the societal harm of misinformation, as seen in events like the 2024 Southport riots.

The paper highlights two primary challenges in current EARD research:

Data Scarcity: New events often lack sufficient annotated training data, and early-stage propagation involves limited posts. Existing methods typically require extensive labeled data for joint training of detection and timing modules.
Limitations of Large Language Models (LLMs): While LLMs excel in few-shot text understanding, they are not inherently designed for time-series data. Applying them directly to EARD is computationally expensive because they would need to generate predictions at every time step as the context length grows, leading to high inference latency and cost.

The core problem addressed is how to leverage the strong few-shot text understanding capabilities of LLMs for EARD in data-limited scenarios without incurring prohibitive computational costs or requiring extensive retraining.

2. Methodology

The authors propose a novel framework that decouples the early time point determination from the rumor detection process. The system consists of two main components:

A Lightweight Autonomous Agent: A neural network responsible for monitoring the stream of posts and deciding when to stop observation and trigger the LLM.
A Training-Free LLM: A pre-trained Large Language Model (e.g., Mistral, Llama 3, ChatGPT) that acts as the rumor detector, activated only once at the time point determined by the agent.

Formulation as a Markov Decision Process (MDP):
The EARD task is modeled as an MDP $(S, A, P, R)$ :

State ( $s_t$ ): The set of observed posts up to time $t$ and the previous action.
Action ( $a_t$ ): Binary choice: Continue (0) to observe the next post, or Stop (1) to trigger the LLM for prediction.
Goal: Maximize expected returns reflecting early and accurate detection.

Imitation Learning (IL) Strategy:
Since designing an explicit reward function for EARD is difficult (balancing earliness, stability, and accuracy), the authors employ Imitation Learning. Instead of reinforcement learning with a hand-crafted reward, the agent learns from "expert trajectories" generated by the LLM itself.

Three types of expert trajectories are curated based on the LLM's prediction sequence over the full post timeline:

Conservative Expert (CE): Selects the earliest time point where the LLM's prediction is correct and remains stable (unchanged) for all subsequent steps. This ensures accuracy and stability but may be too cautious.
Early-Action Expert (EAE): Selects the very first time point where the LLM's prediction matches the ground truth, regardless of future stability. This prioritizes earliness.
Misleading Expert (ME): Captures trajectories where the LLM makes an incorrect prediction even after observing all posts. The agent is trained to avoid the state-action distributions of this expert.

Objective Function:
The agent is trained using Generative Adversarial Imitation Learning (GAIL). The objective is to minimize the divergence (Jensen-Shannon Divergence) between the agent's occupancy measure and those of the CE and EAE, while maximizing the divergence from the ME.

The agent learns to synthesize the stability of CE and the proactivity of EAE.
It learns to deviate from the behaviors of ME to avoid incorrect early terminations.
The training involves a Policy Network, a Discriminator Network (to distinguish agent vs. expert trajectories), and a Value Network, optimized via Proximal Policy Optimization (PPO).

Inference:
During inference, the trained agent monitors the post stream. Once it decides to "Stop," it activates the LLM exactly once to classify the claim based on the observed posts. This drastically reduces LLM inference frequency compared to step-by-step prompting.

3. Key Contributions

The paper claims four main contributions:

First Few-Shot EARD Method: Introduces a framework combining a lightweight trainable agent with a training-free LLM, offering a cost-effective solution for data-limited scenarios.
Novel MDP and Imitation Formulation: Models EARD as an MDP and designs three specific expert trajectory types (CE, EAE, ME) to guide the agent in balancing earliness, stability, and accuracy without explicit reward engineering.
Theoretical Guarantee: Provides a theoretical proof that the proposed method leads to an optimal policy for early, stable, and accurate detection.
Empirical Superiority: Demonstrates through extensive experiments that the approach outperforms existing full-shot EARD methods and baseline LLM strategies in both accuracy (macro-F1) and earliness (Early Rate).

4. Experimental Results

The method was evaluated on four real-world datasets: PHEME, TWITTER, Twitter-COVID-19, and BEARD.

Performance vs. Baselines: The proposed framework significantly outperformed existing EARD methods (ERD, CED, HEARD) and direct LLM strategies (using only the first post or preset time checkpoints).
- On the TWITTER dataset, the framework enabled Mistral to achieve a macro-F1 of 0.630 with an Early Rate (ER) of 0.043, whereas the best baseline (CED) had an ER of 0.617.
- The method showed consistent improvements across different LLM backbones (Mistral, Llama 3, ChatGPT), with relative F1 improvements ranging from 8.2% to 18.9% over the base LLMs.
Few-Shot Robustness: The method remained effective with as few as 10 labeled training instances, achieving relative F1 improvements of up to 27.64% over the base LLM.
Generalization: When trained on pre-COVID datasets (PHEME, TWITTER) and tested on the unseen Twitter-COVID-19 dataset, the proposed method maintained superior performance compared to baselines, which suffered significant performance drops (e.g., up to 67.8% F1 drop for ERD).
Trade-off Analysis: The authors analyzed the trade-off between accuracy and earliness by varying the ratios of expert trajectories. Their method achieved Pareto-optimal points, offering better accuracy at lower ERs than existing baselines.

5. Significance and Claims

The paper positions this work as a solution to the critical bottleneck of data scarcity in rumor detection.

Cost-Efficiency: By decoupling the timing decision from the detection model, the approach allows the use of powerful LLMs without the computational burden of continuous inference. Only the lightweight agent requires training.
Adaptability: The framework is model-agnostic and can be integrated with various LLMs, making it adaptable to evolving language models.
Real-World Applicability: The ability to detect rumors early with minimal labeled data addresses the practical need for rapid intervention in emerging misinformation events where annotated data is unavailable.

The authors conclude that their imitation learning approach effectively leverages the complementary strengths of proactive timing control (the agent) and reactive content evaluation (the LLM), providing a robust framework for early rumor detection in data-limited environments.