Speaking at the Right Level: Literacy-Controlled Counterspeech Generation with RAG-RL

1. Problem Statement

Health misinformation on social media poses a severe threat to public health, leading to poor health choices and eroding trust in medical advice. While automated counterspeech generation using Large Language Models (LLMs) has emerged as a mitigation strategy, existing approaches suffer from a critical limitation: they generate uniform responses that ignore the varying health literacy levels of the audience.

• The Mismatch: Users with low health literacy often cannot comprehend complex, scientific language, while users with high literacy may find simplified content patronizing or uninformative.
• The Gap: Current methods fail to adapt the complexity, tone, and structure of counterspeech to the specific cognitive capabilities and preferences of the target audience, reducing the effectiveness of the intervention.

2. Methodology: The Controlled-Literacy Framework

The authors propose Controlled-Literacy, a novel framework that integrates Retrieval-Augmented Generation (RAG) with Reinforcement Learning (RL) to generate tailored counterspeech. The pipeline consists of three main stages:

A. Knowledge Base Construction & Evidence Retrieval

• Diverse Sources: A knowledge base is constructed from reliable sources (e.g., CDC, Johns Hopkins Children's Center, academic databases) containing materials at varying complexity levels.
• Readability Filtering: Documents are categorized into three levels based on the Flesch-Kincaid Reading Ease (FKRE) score:
  • Low Literacy: FKRE 80–100 (Easy)
  • Medium Literacy: FKRE 60–79 (Medium)
  • High Literacy: FKRE 0–59 (Hard)
• Hybrid Retrieval: The system uses a hybrid retriever combining keyword-based and semantic search to fetch relevant evidence.
• Adaptive Filtering: Retrieved evidence is filtered to ensure it falls within the target FKRE range and receives a high "user preference" rating (simulated by LLMs) to ensure it is both readable and helpful.

B. Reward Function Design

To optimize generation, the authors design a composite reward function $r(x, y)$ that balances objective readability and subjective user preference:
$$r(x, y) = \alpha \cdot r_{read}(x, y) + (1-\alpha) \cdot r_{pref}(x, y)$$

1. Readability Reward ($r_{read}$): Uses a double-sigmoid function to encourage the generated text's FKRE score to fall within a specific target range (e.g., 80–100 for low literacy) rather than enforcing a strict equality, allowing for natural language flow.
2. User Preference Reward ($r_{pref}$): Since real user data is scarce, the authors use LLM-simulated raters conditioned on specific literacy levels to rate the counterspeech on a 1–5 Likert scale regarding clarity, accessibility, and helpfulness.

C. RL Optimization (GRPO)

The framework employs Group Relative Policy Optimization (GRPO) to fine-tune the LLM policy:

• Process: For each input (misinformation + target literacy level), the model generates $n$ responses.
• Advantage Calculation: The model computes the relative advantage of each response based on the aggregated reward signal.
• Policy Update: The policy is updated to increase the probability of high-reward responses while using KL-divergence regularization to prevent the model from drifting too far from the initial supervised policy.

3. Key Contributions

1. Novel Insight: The paper establishes that effective counterspeech must be aligned with the target audience's health literacy level to be accessible and persuasive.
2. Controlled-Literacy Framework: A technical architecture combining RAG for evidence control and RL for style/complexity optimization, addressing both objective readability and subjective user perception.
3. MisinfoLiteracy Dataset: The creation of a new dataset containing 440 health misinformation posts paired with counterspeech tailored to low, medium, and high literacy levels.
4. Simulated Evaluation: A robust evaluation strategy using LLMs to simulate diverse user profiles, validated against human annotators.

4. Experimental Results

The framework was evaluated on the MisinfoLiteracy dataset and cross-generalized to Check-COVID and MisinfoCorrect using various LLMs (LLaMA-8B, Qwen-7B, LLaMA-1B).

• Performance Metrics: The study measured Politeness, Target Distance (alignment with FKRE), User Preference, and Factual Accuracy.
• Key Findings:
  • Superior Alignment: Controlled-Literacy significantly reduced "Target Distance," meaning the generated text matched the intended readability level much better than baselines (Instructional Prompts or standard RAG).
  • User Preference: The method achieved higher user preference scores across all literacy levels compared to baselines.
  • Small Model Benefit: Smaller models (e.g., LLaMA-1B) showed the most dramatic performance improvements when using Controlled-Literacy, suggesting the framework helps compensate for limited model capacity.
  • Cross-Generalization: The method maintained robust performance across different datasets (Twitter vs. Reddit) and misinformation types.
  • Ablation: While RAG improved factual accuracy, it did not significantly improve user preference without the RL optimization, highlighting the necessity of the reward-driven generation.

5. Significance and Impact

• Equitable Public Health Communication: By tailoring information to the user's comprehension level, the framework ensures that critical health corrections are accessible to vulnerable populations (low literacy) while remaining engaging for experts (high literacy).
• Scalability: The approach automates the generation of personalized counterspeech at scale, addressing the volume of misinformation that human fact-checkers cannot handle.
• Future Directions: The authors identify limitations such as the lack of real-time web search integration and the need for finer-grained user segmentation beyond three broad categories. They propose future work to incorporate dynamic information retrieval and more nuanced user profiling.

In conclusion, this paper demonstrates that controlling the "level" of communication is as critical as the accuracy of the content in combating health misinformation, offering a scalable, RL-driven solution to bridge the gap between scientific facts and public understanding.