💬 NLP

Towards Detecting Persuasion on Social Media: From Model Development to Insights on Persuasion Strategies

This paper presents a lightweight, state-of-the-art model for detecting persuasive political text that is fine-tuned on Australian election Facebook ads to reveal distinct persuasion strategies across funding, demographics, and timing, thereby enhancing transparency and voter awareness in digital campaigns.

Original authors: Elyas Meguellati, Stefano Civelli, Pietro Bernardelle, Shazia Sadiq, Irwin King, Gianluca Demartini

Published 2026-05-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Elyas Meguellati, Stefano Civelli, Pietro Bernardelle, Shazia Sadiq, Irwin King, Gianluca Demartini

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are walking through a giant, noisy digital marketplace (social media). In this marketplace, political campaigns are like vendors trying to sell you an idea (a candidate or a policy) rather than a physical product. Some vendors are honest and just list the facts, while others use "sales tricks" to tug at your heartstrings, scare you, or make you feel angry to get you to buy in.

This paper is about building a smart, lightweight "sales detector" to spot those tricks, and then using it to understand how Australian politicians ran their 2022 election campaign on Facebook.

Here is the breakdown of their work, using simple analogies:

1. The Problem: The "Sales Pitch" is Hard to Spot

Political ads often use subtle psychological tricks. The paper gives examples like:

Loaded Language: Using words that make you feel a strong emotion immediately (e.g., calling a refugee a "terrorist").
Name Calling: Insulting an opponent instead of arguing their point (e.g., calling someone "Bush the Lesser").
Appeal to Fear: Trying to scare you into agreeing with them.

Humans are bad at spotting these tricks when they are scrolling quickly through their phones. Experts can't read millions of ads manually. So, the researchers wanted to build a computer program that could read these ads and say, "Hey, this sentence is trying to manipulate you."

2. Study 1: Building a "Lightweight" Detector

The researchers first needed to teach their computer how to spot these tricks. They used a dataset of news articles (from a competition called SemEval-2023) where humans had already marked the tricky sentences.

The Challenge: Most other "smart detectors" are like giant, heavy trucks. They need massive amounts of fuel (computing power) and huge cargo loads (training data) to work.
The Innovation: The team built a "bicycle" instead. They created a model called PPAsy-XLNet.
- It is lightweight: It uses much less computing power.
- It is efficient: It learned to spot the tricks using a tiny fraction of the data other models needed (about 87% less data).
- The Result: Even though it's the "bicycle," it ran just as fast and won the race against the "giant trucks" (the state-of-the-art models). It proved you don't need a supercomputer to detect political manipulation.

3. Study 2: Testing the Detector on Real Ads

Now that they had their "bicycle," they wanted to see if it worked in the real world. They collected 56,958 political ads from the 2022 Australian Federal Election on Facebook.

The "Domain Gap" (The Translation Problem):
When they first tried to use their model (trained on news articles) on these Facebook ads, it stumbled. It was like taking a driver who is great at driving on a quiet country road and putting them in a chaotic city traffic jam; they didn't know how to react. The model wasn't accurate enough.
- The Fix: They gave the model a quick "refresher course" by showing it a small number of actual Facebook ads. Once it learned the specific style of social media ads, it became an expert again.
What They Discovered (The Insights):
Once the model was trained, they used it to analyze the whole election. Here is what they found:
- The "High-Octane" Ads: About 46% of the ads were "highly persuasive." These ads were packed with emotional language, calls to action, and fear-mongering.
- Money Talks: The "highly persuasive" ads got more attention (impressions) and cost more money to run. It seems campaigns realized that emotional, manipulative ads work better, so they spent more on them.
- The "Sprint" to the Finish: As Election Day got closer, the spending on these emotional, persuasive ads skyrocketed. It was like a sprinter saving all their energy for the final 100 meters. The day before the election, the spending was nearly 5 times higher than it was a month prior.
- Language Differences:
  - High Persuasion Ads: Used words like "future," "change," "vote," and "government." They were big-picture and urgent.
  - Low Persuasion Ads: Used words like "local," "community," and "support." They were more like dry fact sheets about local issues.

4. The Bottom Line

The paper concludes with two main takeaways:

You don't need a supercomputer to detect political manipulation; a smaller, smarter model can do the job just as well if you train it correctly.
Context matters. A model trained on news articles doesn't automatically work on Facebook ads. You have to "fine-tune" it for the specific environment.

Why this matters: By understanding these patterns, we can see that political campaigns are strategically using emotional manipulation, especially right before an election, and they are willing to spend huge amounts of money to make sure those messages reach us. This helps voters and watchdogs see the "sales tactics" behind the political ads.

Note: The authors mention that future work could involve looking at images in ads or using different prediction methods, but they did not test those things in this specific paper.

Technical Summary: Towards Detecting Persuasion on Social Media

Problem Statement

The paper addresses the critical challenge of detecting persuasive elements within political advertising on social media. While political propaganda shapes public opinion through subtle techniques (e.g., loaded language, name-calling, appeals to fear), manual detection is increasingly unfeasible due to the volume of content. Existing computational approaches often rely on heavy transformer ensembles or extensive data augmentation, creating high barriers to entry regarding computational resources and data requirements. Furthermore, there is a gap in understanding how these detection models perform when transferred from mainstream news domains to the specific context of social media political ads.

Methodology

The research is structured into two interconnected studies:

Study 1: Lightweight Model Development (SemEval-2023)

The authors developed PPAsy (Persuasion Prediction Asymmetric), a novel multi-label classification approach for detecting 23 distinct persuasion techniques.

Model Architecture: The study selected XLNet-base (110M parameters) as the backbone, prioritizing a balance between performance and resource efficiency over larger ensembles.
Loss Function: A custom Asymmetric Binary Cross-Entropy loss was introduced to handle severe label imbalance. This function applies instance-specific weighting ( $w_i$ ) modulated by a parameter $\beta$ , penalizing misclassifications variably based on class distribution.
Data Strategy: Instead of training on all available multilingual data, the authors translated only the French subset (which contained all 23 labels and the highest average of techniques per document) into English. This reduced the training data size to approximately 153K tokens, significantly smaller than the 1.16M tokens used by state-of-the-art (SOTA) competitors.
Preprocessing: Minimal preprocessing was applied (lemmatization, lowercasing), retaining punctuation as it was found to improve model interpretation.

Study 2: Domain Adaptation and Real-World Application (APA22)

The second study evaluated the model's utility on a newly curated dataset, APA22, comprising 56,958 political ads from the 2022 Australian Federal Election collected via the Meta Ad Library.

Domain Adaptation: The model trained on SemEval-23 (news articles) was tested on APA22. Recognizing a performance drop, the model was fine-tuned on a manually annotated subset of APA22 (658 samples, binarized to "persuasive" vs. "neutral").
Analysis: The fine-tuned model labeled the entire APA22 dataset to analyze patterns in funding, demographics, temporal shifts, and linguistic features (bi-grams and TF-IDF) distinguishing high-persuasion ads from low-persuasion ones.

Key Contributions

Efficient Model Development: Introduction of a lightweight, multi-label classification model that achieves SOTA performance on SemEval-2023 Task 3 Subtask 3 with significantly reduced computational costs.
Resource Efficiency: The proposed approach requires approximately 86.8% less data, up to 77.6% fewer parameters (compared to the APatt ensemble), and 86.2% less training time (compared to KInIT) while outperforming or matching existing SOTA methods.
New Dataset (APA22): Creation and partial annotation of a large-scale dataset of political ads from the 2022 Australian Federal Election, bridging the gap between academic benchmarks and real-world social media content.
Empirical Insights: Demonstration that domain-specific retraining is essential for social media ad analysis and the revelation of distinct persuasion strategies (e.g., higher spending and longer duration for high-persuasion ads) in political campaigns.

Results

Performance on SemEval-2023

PPAsy-XLNet achieved an F1-Micro of 40.62% and F1-Macro of 21.78% on the English test set.
This outperformed the baseline SVM (19.51% F1-Micro) and surpassed the APatt ensemble (37.56% F1-Micro) and the KInIT RoBERTa-large model (40.38% F1-Micro), despite using a fraction of the parameters and training data.
Ablation studies confirmed that the custom loss function and the specific use of the French-to-English translated dataset were critical drivers of performance.

Performance on APA22

Domain Shift: When applied directly to APA22 without retraining, the SemEval-trained model achieved only 59.39% accuracy, highlighting a significant domain gap between news articles and social media ads.
Fine-tuning: Retraining on the APA22 subset improved accuracy to 82.42%, demonstrating the necessity of domain-specific data.

Insights from APA22 Analysis

Correlation with Reach: Ads classified as "high persuasion" (containing >80% persuasive sentences) received 45.6% more impressions and 48.2% more spending than "low persuasion" ads.
Temporal Dynamics: High-persuasion ad spending and impressions surged significantly (up to 4.8x) in the days leading up to the election, whereas low-persuasion ads showed a steady, modest growth trajectory.
Linguistic Patterns: High-persuasion ads utilized action-oriented vocabulary ("vote," "future," "change") and emotive bi-grams ("climate change," "better future"). Low-persuasion ads focused on informational, local, and administrative details ("local," "support," "small business").

Significance and Claims

The paper claims that its work underscores the necessity of domain-specific modeling for analyzing persuasion on social media. It argues that while heavy, resource-intensive models are common, efficient, lightweight models can achieve comparable or superior results if paired with strategic data selection (e.g., selecting the most diverse language subset).

The authors posit that uncovering these persuasion strategies through automated detection can:

Enhance transparency in democratic processes.
Inform voters about the nature of political messaging.
Promote accountability in digital campaigns.

The study concludes that while the current models provide a foundation for understanding digital persuasion, future work should explore multimodal approaches (integrating images) and refine classification strategies to better capture the complex interplay of text and visual media in shaping public opinion. The authors remain modest regarding the limitations of their data (e.g., reliance on Meta's self-reported political flags and lack of precise micro-targeting data) and the challenges of generalizing findings across different political contexts.