Technical Summary: BERTopic-Virality Prioritisation (BERTopic-VP)

Problem Statement

Health misinformation during pandemics spreads rapidly on social media, creating harmful narratives that compete with public health guidance. Conventional topic modelling techniques, such as Latent Dirichlet Allocation (LDA) and Non-negative Matrix Factorisation (NMF), struggle with the specific characteristics of social media discourse: short text lengths (e.g., tweets under 280 characters), high noise, dynamic hashtag trends, and sparse vocabulary. These frequency-based methods often fail to detect emerging narratives or produce incoherent topics in micro-text settings. Furthermore, existing pipelines typically treat user engagement (likes, retweets) as an external outcome rather than an integral part of the topic discovery process, limiting the ability to prioritise semantically coherent topics that are also rapidly diffusing. There is a lack of systematic, comparative analysis of misinformation dynamics across different pandemics (e.g., COVID-19 vs. Monkeypox) using a unified, engagement-aware framework.

Methodology

The authors propose BERTopic-VP, a scalable, modular framework that extends the BERTopic topic modelling pipeline with a Virality Prioritisation (VP) layer. The pipeline operates through the following stages:

1. Semantic Embedding & Clustering:

   • Tweets are processed using the all-MiniLM-L6-v2 Sentence-BERT encoder to generate contextual 384-dimensional embeddings.
   • Dimensionality reduction is performed via UMAP (Uniform Manifold Approximation and Projection).
   • Clustering is executed using HDBSCAN (Hierarchical Density-Based Spatial Clustering of Applications with Noise). This density-based approach allows for the discovery of variable numbers of topics without pre-specifying $K$, making it suitable for evolving discourse.
   • The configuration is fixed across datasets (UMAP: $n\_neighbors=15$, $min\_dist=0.1$, $n\_components=5$; HDBSCAN: $min\_cluster\_size=50$, $min\_samples=10$) to ensure comparability.

2. Virality Prioritisation (VP) Layer:

   • Observed Engagement: For datasets with native metadata (Monkeypox), a composite engagement score $E_i$ is calculated as the sum of likes, retweets, replies, quotes, and link clicks.
   • Proxy Propensity: For datasets lacking engagement metadata (COVID-19_FNIR, Constraint), the framework trains a logistic regression model on the Monkeypox dataset to predict a "propensity-to-spread" score ($\hat{p}_i$). This score serves as an ordinal proxy for diffusion potential.
   • Unified Signal: A unified prioritisation signal $\tilde{E}_i$ is defined, using observed $E_i$ where available and the propensity logit $\eta_i$ otherwise.
   • Ranking: Clusters are ranked by their aggregate viral score $v_j$ (mean $\tilde{E}_i$ within the cluster). Clusters falling within the top $x\%$ (1%, 5%, 10%) of the distribution are flagged as high-priority.

3. Misinformation Detection & Verification:

   • A two-stage hybrid detection module fuses a supervised content-based classifier with an external verification signal derived from outbreak-specific public health knowledge bases (e.g., WHO Mythbusters).
   • The final probability $p_{final}$ is a weighted sum: $p_{final} = \alpha p_{content} + (1-\alpha) p_{verify}$, with $\alpha=0.6$ determined via grid search.

4. Evaluation Metrics:

   • Clustering: Evaluated via Normalised Mutual Information (NMI), Adjusted Rand Index (ARI), Purity, and Topic Coherence ($C_v$).
   • Classification: Evaluated via Accuracy, Precision, Recall, F1-score, ROC-AUC, and PR-AUC.
   • Linguistic/Affective: Analysis of sentiment polarity, discrete emotions (Joy, Fear, Anger, Sadness), and lexical complexity (Flesch Reading Ease, Flesch-Kincaid Grade Level).

Key Contributions

1. BERTopic-VP Framework: The introduction of a virality-prioritised topic modelling pipeline that integrates semantic clustering with post-hoc engagement signals (or propensity proxies) to rank topics by diffusion potential.
2. Comparative Infodemic Analysis: The first systematic, side-by-side comparison of misinformation themes across two distinct outbreaks (COVID-19 and Monkeypox) using a consistent analytical framework.
3. Proxy Virality for Data-Scarce Contexts: A methodological innovation for prioritising topics in datasets lacking engagement metadata by transferring a propensity-to-spread model trained on a dataset with observed engagement.
4. Monitoring-Oriented Design: A pipeline designed for periodic batch refreshes (minute-to-hour cadence) to support early warning systems, surfacing low-volume but high-risk narratives for analyst review.

Results

The framework was evaluated on three benchmark datasets: COVID-19_FNIR, Monkeypox, and Constraint.

• Classification Performance: The hybrid misinformation detector achieved strong performance across all datasets. The Monkeypox dataset yielded the highest results (F1 = 0.950, ROC-AUC = 0.989), followed by COVID-19_FNIR (F1 = 0.907, ROC-AUC = 0.966) and Constraint (F1 = 0.899, ROC-AUC = 0.951).
• Clustering Quality:
  • Monkeypox: Achieved the highest purity (0.940) and moderate coherence ($C_v = 0.286$), indicating tightly focused narratives.
  • COVID-19_FNIR: Showed moderate purity (0.742) but the lowest coherence ($C_v = 0.084$), reflecting a fragmented and diverse narrative space (e.g., mixing lab-leak theories, mask efficacy, and policy debates).
  • Constraint: Exhibited the highest coherence ($C_v = 0.463$) but the lowest purity (0.522), suggesting internally consistent themes that do not align perfectly with the dataset's veracity labels.
• Virality & Linguistic Patterns:
  • Emotional Drivers: Monkeypox misinformation exhibited the most intense emotional activation (higher Fear, Anger, and Sadness) compared to COVID-19.
  • Lexical Simplicity: Monkeypox tweets were linguistically simpler (higher Flesch Reading Ease, lower Grade Level) than COVID-19 tweets, which were denser and more complex.
  • VP Alignment: Clusters prioritised by the VP signal consistently overlapped with those exhibiting high emotional charge and simpler language, confirming that affective arousal and cognitive accessibility drive diffusion.
• Cross-Event Differences: COVID-19 misinformation was characterised by conspiracy theories and pseudoscience, while Monkeypox narratives focused on institutional trust, response failures, and stigma.

Significance and Claims

The paper claims that BERTopic-VP offers a novel, end-to-end methodological contribution to computational social science and NLP by unifying unsupervised topic modelling, supervised classification, and affective-linguistic analysis.

• Operational Value: The framework supports monitoring-oriented early warning by surfacing low-volume but high-risk narratives that traditional frequency-based models might miss. It allows public health analysts to distinguish high-impact misinformation from background noise.
• Scalability: The pipeline is computationally feasible for near-real-time batch processing (e.g., 10,000 tweets processed in under 60 seconds on commodity hardware), suitable for periodic surveillance.
• Interpretability: By linking topic clusters to observable linguistic cues (emotion, readability) and virality signals, the system provides explainable outputs suitable for non-technical stakeholders.
• Modesty on Claims: The authors explicitly note that they do not re-implement classical baselines (LDA, NMF) on their specific datasets for a head-to-head benchmark, citing the sensitivity of topic quality to corpus-specific preprocessing. Instead, they contextualise their results against literature values. They also acknowledge that for datasets without engagement metadata, the VP signal represents an estimated diffusion propensity rather than realised platform interaction, limiting cross-dataset magnitude comparisons to within-dataset ranks.

The study concludes that integrating semantic structure with virality-aware ranking enables scalable, interpretable comparative analysis of misinformation, supporting evidence-based strategies in digital health communication.