Click it or Leave it: Detecting and Spoiling Clickbait with Informativeness Measures and Large Language Models

1. Problem Statement

The paper addresses the degradation of online information quality caused by clickbait—headlines that use exaggerated, misleading, or emotionally manipulative phrasing to provoke curiosity and drive engagement, often without delivering on the promised content.

• Current Limitations: Existing research largely treats clickbait detection as a "black-box" binary classification problem. While deep learning models (e.g., BERT, RoBERTa) achieve high accuracy, they offer limited interpretability regarding why a headline is flagged. Furthermore, they often fail to explicitly model the specific linguistic cues (e.g., second-person pronouns, superlatives, specific punctuation) that define clickbait.
• Goal: To develop a detection system that not only achieves state-of-the-art accuracy but also provides interpretability through explicit, linguistically motivated features, bridging the gap between semantic understanding and stylistic manipulation.

2. Methodology

The authors propose a hybrid approach that combines the deep semantic understanding of Large Language Model (LLM) embeddings with explicit, handcrafted informativeness measures.

A. Dataset Construction

The study integrates and harmonizes four public datasets to create a unified, multi-dimensional training set:

1. Kaggle-1 & Kaggle-2: News articles with binary labels (BuzzFeed, Upworthy, etc.).
2. Clickbait Challenge 2017: Twitter posts with crowd-sourced graded labels (0.0–1.0), binarized at a threshold of 0.5.
3. SemEval Clickbait Spoiling: Social media posts dedicated to exposing clickbait, including extractive and abstractive spoilers.

• Preprocessing: Data was cleaned (English-only, deduplication), and labels were harmonized. The dataset was split into stratified 80/10/10 training/validation/test sets.

B. Feature Engineering: Informativeness Measures

Instead of relying solely on latent representations, the authors engineered 25 explicit features (using a subset of 15 for the final model) to quantify stylistic and linguistic properties. These include:

• Lexical & Structural: Word count, capital letters ratio, punctuation ratios (distinguishing "bait" punctuation like ! and ? from standard punctuation).
• Linguistic Cues: Frequency of second-person pronouns ("you"), superlatives, speculatives ("might", "possibly"), and bait phrases.
• Readability & Sentiment: Flesch Reading-Ease (FRES), sentiment polarity, and subjectivity scores.
• Semantic Similarity: Cosine similarity between the headline and the article body.

C. Model Architecture

The proposed pipeline follows an experimental workflow:

1. Embedding Generation: Text is converted into vectors using:
   • Traditional: TF-IDF, Word2Vec, GloVe.
   • Transformer-based: BERT, RoBERTa, and OpenAI text-embedding-3-large.
2. Hybrid Fusion: The 15 explicit informativeness features are concatenated with reduced-dimensional OpenAI embeddings (1000d).
3. Classification: The fused vector is fed into XGBoost, chosen for its superior performance and calibration compared to Random Forest and deep classifiers.

3. Key Contributions

1. Hybrid Detection Framework: Demonstrates that combining explicit linguistic features with dense semantic embeddings yields superior performance compared to using either method alone.
2. Interpretability: The model highlights specific "salient linguistic cues" (e.g., excessive use of "you," superlatives, and specific punctuation) that drive predictions, making the detection process transparent and explainable.
3. Comprehensive Benchmarking: The study rigorously compares traditional ML, static embeddings, fine-tuned transformers, and prompt-based LLMs (Zero-shot/Few-shot) on a unified dataset.
4. Spoiling Capability: The inclusion of the SemEval Spoiling dataset allows the framework to extend beyond binary detection to generating "spoilers" (abstractive and extractive) that reveal the actual content behind the clickbait.

4. Experimental Results

The experiments were conducted on a CPU-only workstation using Python, scikit-learn, XGBoost, and Hugging Face/OpenAI APIs.

• Performance Metrics: The proposed hybrid model achieved an F1-score of 90.9% (Accuracy: 91.0%, Precision: 92.2%, Recall: 89.6%, AUC: 0.969).
• Comparison with Baselines:
  • vs. Embeddings Only: The hybrid model outperformed the best embedding-only baseline (OpenAI 3072d + XGBoost) by +4.5 percentage points in F1-score (0.909 vs. 0.864).
  • vs. LLM Prompting: The hybrid model significantly outperformed prompt-based LLMs. GPT-4o-mini (Few-shot) achieved an F1 of 0.82, and Gemini-3 Flash (Few-shot) achieved 0.84.
  • vs. Traditional Models: Outperformed TF-IDF, Word2Vec, and GloVe baselines (which hovered around 0.83 F1).
• Ablation Study: Confirmed that while transformer embeddings provide a strong baseline, the handcrafted features add complementary information that is not redundant with semantic embeddings, specifically capturing pragmatic and stylistic manipulation cues.

5. Significance and Conclusion

• Beyond Black Boxes: The paper argues that high accuracy alone is insufficient for clickbait detection. The proposed method offers transparent decision-making, allowing users and developers to understand the specific linguistic triggers (e.g., "This headline is clickbait because it uses 3 superlatives and 2 second-person pronouns").
• Practical Application: The approach is computationally efficient (using XGBoost on reduced embeddings) compared to running full LLM inference for every headline, making it suitable for real-time deployment.
• Future Directions: The authors suggest extending this framework to low-resource languages and refining the "spoiling" mechanism to automatically generate accurate summaries of clickbait articles.

In summary, the paper establishes that explicit linguistic modeling combined with modern embeddings creates a more robust, accurate, and interpretable system for detecting clickbait than relying solely on the scale of Large Language Models.