💬 NLP

Spam and Sentiment Detection in Arabic Tweets Using MARBERT Model

This study proposes a deep learning approach using the MARBERT model to detect spam and analyze sentiment in 24,513 Arabic tweets regarding Saudi Telecom Company (STC), aiming to enhance customer service through improved sentiment classification metrics.

Original authors: Abrar Alotaibi, Atta-ur Rahman, Raheel Alhaza, Wala Alkhalifa, Narjes Alhajjaj, Atheer Alharthi, Dhai Abushoumi, Maryam Alqahtani, Dania Alkhulaifi

Published 2026-06-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Abrar Alotaibi, Atta-ur Rahman, Raheel Alhaza, Wala Alkhalifa, Narjes Alhajjaj, Atheer Alharthi, Dhai Abushoumi, Maryam Alqahtani, Dania Alkhulaifi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine STC (Saudi Telecom Company) as a giant, bustling city square. Every second, thousands of people are shouting their opinions, complaints, and praise into the air via Twitter. The company wants to listen to every single voice to make their service better, but there are simply too many voices for a team of human listeners to keep up with. It's like trying to hear a single whisper in a hurricane.

This paper is about building a super-smart robot ear (an AI model) that can listen to this chaotic crowd, understand what they are saying, and sort their shouts into neat categories.

Here is how the researchers built this robot, explained simply:

1. The Problem: The Language Barrier and the "Noise"

The researchers faced two main hurdles:

The Language: Arabic is tricky. It's not just one uniform language; it has a formal version (like the language in books) and many different "street" dialects (like slang used in chat rooms). It's like trying to teach a robot to understand English, but the robot has to handle formal Shakespearean English and heavy New York street slang simultaneously.
The Noise: The tweets aren't just clean sentences. They are full of "hashtags" (like #STC), links, usernames, and repeated messages. It's like trying to read a book where every page is covered in sticky notes and scribbles.

2. The Solution: The "MARBERT" Robot

Instead of building a robot from scratch, the team used a pre-trained brain called MARBERT.

The Analogy: Imagine a student who has already read millions of Arabic books and tweets. This student knows the grammar, the slang, and the context of the Arabic language perfectly. This is MARBERT.
The Twist: Most AI models are trained on formal text. MARBERT is special because it was trained specifically on tweets, meaning it understands the messy, informal, dialect-heavy way people actually talk on social media.

3. The Training: Teaching the Robot to Sort

The researchers took a massive pile of 24,513 real tweets from STC customers and taught the robot to sort them into five different bins:

Positive: "Great service!"
Negative: "My internet is down!"
Neutral: "I just checked my balance."
Sarcasm: "Oh, great, another outage. Just what I needed." (This is the hardest one to catch because the words say "great," but the meaning is "bad.")
Indeterminate: "I don't know what to say."

The "Imbalanced Class" Problem:
The researchers noticed a problem: The robot was great at spotting "Negative" and "Positive" tweets, but it kept getting confused by "Sarcasm" and "Indeterminate" tweets.

The Metaphor: Imagine a teacher grading a test where 90% of the answers are "Yes" and only 10% are "No." The student gets lazy and just guesses "Yes" every time because they are right most of the time. The robot was doing the same thing; it ignored the rare "Sarcasm" tweets because there weren't enough of them to learn from.
The Fix: The researchers went back to Twitter and collected more sarcastic tweets to balance the pile. They also tweaked the robot's "learning settings" (like how fast it learns and how many examples it looks at at once) to make it pay more attention to the difficult, rare cases.

4. The Results: A Smarter Ear

After tuning the robot and feeding it more data, the results were impressive:

Spam Detection: The robot became very good at spotting "spam" (junk messages), correctly identifying them 98% of the time.
Sentiment Analysis: It successfully sorted the customer tweets into the five categories with high accuracy.
The Secret Sauce: The researchers found that using a specific "batch size" (how many tweets the robot looks at before taking a break to think) and a slow "learning rate" (taking its time to learn) worked best.

5. The Goal

The ultimate goal isn't just to build a cool robot; it's to help STC. By automatically reading these tweets, STC can instantly know if customers are angry, happy, or being sarcastic. This allows them to fix problems faster and improve their customer service, turning that chaotic city square of shouting voices into a manageable conversation.

In summary: The paper describes taking a smart, pre-trained Arabic language brain (MARBERT), cleaning up a messy pile of tweets, teaching it to spot sarcasm and junk mail, and tuning it until it became a highly effective tool for understanding customer feelings.

Technical Summary: Spam and Sentiment Detection in Arabic Tweets Using MARBERT Model

Problem Statement
Saudi Telecom Company (STC), a major service provider in Saudi Arabia, faces significant challenges in managing customer satisfaction due to the sheer volume of feedback received via Twitter. While Twitter serves as a primary channel for customer complaints and sentiment expression, the rapid influx of tweets makes manual analysis impossible. Furthermore, Arabic presents unique linguistic challenges for Natural Language Processing (NLP), including complex morphology, the existence of multiple dialects (Dialectal Arabic vs. Modern Standard Arabic), and the prevalence of sarcasm. Existing literature often focuses on English or binary classification, leaving a gap in robust, multi-class sentiment analysis and spam detection specifically for unbalanced Arabic datasets.

Methodology
The study proposes a deep learning framework utilizing the MARBERT model, a pre-trained transformer-based architecture specifically optimized for Arabic text. The methodology encompasses the following stages:

Data Collection and Preprocessing:
- Two distinct datasets were utilized: a sentiment dataset containing 22,000+ manually labeled STC customer tweets and a spam dataset with 24,000+ tweets.
- The sentiment dataset includes five classes: Positive, Negative, Neutral, Sarcasm, and Indeterminate.
- Preprocessing involved removing noise (hashtags, URLs, usernames, extra spaces), eliminating duplicates, and normalizing Arabic characters (e.g., unifying variations of the letter 'A' and 'Y') to ensure uniformity.
Model Architecture:
- The core model is MARBERT, which differs from standard BERT by omitting the Next Sentence Prediction (NSP) objective, as it was trained specifically on short text like tweets.
- MARBERT was trained on a massive corpus of 15.6 billion tokens comprising both Dialectal Arabic (DA) and Modern Standard Arabic (MSA).
- The architecture employs a bidirectional transformer encoder to capture context effectively.
Handling Class Imbalance:
- The dataset exhibited significant class imbalance, particularly with underrepresented classes like "Sarcasm" (initially 200 samples) and "Indeterminate."
- Instead of using oversampling or under-sampling techniques (which the authors noted could introduce noise or data loss), the team addressed the imbalance by collecting additional diverse data from Twitter, increasing the sarcasm class to 1,400 samples.
Hyperparameter Tuning:
- Extensive experiments were conducted to optimize performance. The authors evaluated various batch sizes, learning rates, sequence lengths, and epoch counts.
- The optimal configuration identified was a training batch size of 96, an evaluation batch size of 32, a learning rate of $2 \times 10^{-5}$ , a maximum sequence length of 256, and 5 epochs.

Key Results
The study evaluated the model using Precision, Recall, and F1-score metrics.

Spam Detection: The MARBERT-based approach achieved an F1-score of 0.98 on the spam dataset, outperforming recent studies that utilized SVM, Naive Bayes, or other BERT-based models (which reported scores of 0.93 and 0.96).
Sentiment Analysis:
- The model successfully classified tweets into the five target categories.
- With the optimized hyperparameters (Batch size 96), the model achieved an overall F1-score of 0.75.
- Class-specific performance showed high accuracy for standard sentiments (Negative: 0.85, Positive: 0.92, Neutral: 0.89) but lower scores for the more complex classes (Sarcasm: 0.54, Indeterminate: 0.56), highlighting the difficulty of these specific categories even with data augmentation.

Significance and Contributions
The paper claims the following contributions to the field of Arabic NLP and customer service analytics:

Domain-Specific Application: The study demonstrates the practical application of MARBERT for improving customer service operations at STC by automating the analysis of unstructured Arabic feedback.
Multi-Class Capability: Unlike many prior works focused on binary classification, this research successfully addresses a five-class problem (including sarcasm and indeterminate sentiments) in an unbalanced Arabic environment.
Performance Benchmarking: The results indicate that MARBERT outperforms existing techniques in the literature for both spam detection and multi-label sentiment analysis in Arabic.
Data Strategy: The paper highlights the efficacy of expanding the dataset to address class imbalance rather than relying solely on algorithmic resampling techniques, which can degrade model performance.

The authors conclude that while the model shows promising accuracy, particularly for standard sentiments, the detection of sarcasm and indeterminate tweets remains a challenge. They suggest that future work could explore hybrid deep learning, fusion-based approaches, and federated learning to further enhance performance.