Multiclass Sentiment Analysis for Identifying Political Viewpoints

Technical Summary: Multiclass Sentiment Analysis for Identifying Political Viewpoints in Tamil

Problem Definition
The rapid expansion of social media, particularly platforms like X (formerly Twitter), has generated vast amounts of real-time political discourse. While sentiment analysis (SA) is a well-established task in Natural Language Processing (NLP) for extracting public opinion, applying it to political viewpoints remains challenging due to the complexity and context-dependency of the language. This paper addresses the specific problem of multiclass sentiment analysis for political discourse in the Tamil language, a Dravidian language with limited linguistic resources. The objective is to automatically discriminate between seven distinct sentiment classes regarding political issues and figures: Substantiated, Sarcastic, Opinionated, Positive, Negative, Neutral, and None of the above.

Methodology
The study utilizes a dataset provided for the DravidianLangTech@NAACL 2025 shared task, comprising 4,352 training samples, 544 development samples, and 544 unlabeled test samples. The authors employed a standardized preprocessing pipeline to ensure data consistency, which involved removing punctuation, emojis, and user mentions to minimize noise and author-specific bias.

Two distinct machine learning approaches were designed and evaluated:

1. XGBoost Approach:

   • Feature Extraction: Text data was converted into numerical representations using TF-IDF (Term Frequency-Inverse Document Frequency).
   • Model: An XGBClassifier was trained on the TF-IDF features.
   • Rationale: XGBoost was selected as a robust traditional machine learning algorithm known for strong performance in classification tasks.

2. BERT Approach:

   • Feature Extraction: The bert-base-uncased transformer model was utilized with its native BertTokenizer to handle tokenization and numerical conversion.
   • Model Configuration: The model was fine-tuned with a learning rate of 1e-5, a batch size of 32, and trained for 5 epochs using a softmax activation function at the output level.

Key Contributions
The paper makes three primary contributions to the field of low-resource NLP and political discourse analysis:

• Literature Review: A comprehensive examination of existing sentiment analysis literature in the political domain, highlighting its application in monitoring discourse and identifying biases.
• Feature Engineering: The investigation and selection of appropriate feature extraction methods (TF-IDF and BERT tokenization) tailored for the specific constraints of the Tamil political dataset.
• Empirical Evaluation: The selection, fine-tuning, and rigorous evaluation of state-of-the-art models (XGBoost and BERT) on a held-out test set using standard classification metrics, providing a baseline for future research in multiclass political sentiment analysis for Dravidian languages.

Experimental Results
The models were evaluated on the test set using Precision, Recall, F1-score, and Accuracy. The results indicate that classifying complex political discourse in Tamil is a difficult task, yielding relatively low macro F1-scores for both approaches:

• XGBoost Model: Achieved a Macro F1-score of 0.2835 and an accuracy of 0.3309.
• BERT Model: Achieved a Macro F1-score of 0.2806 and an accuracy of 0.3327.

Performance Analysis:

• Class-Specific Performance: Both models performed best on the "Neutral" and "Positive" classes. For instance, the XGBoost model achieved an F1-score of 0.7347 for "Neutral" and 0.4622 for "Positive."
• Challenges: The models struggled significantly with the "Sarcastic," "Opinionated," and "None of the above" classes, which yielded very low F1-scores (e.g., "None of the above" F1 was 0.0606 for XGBoost).
• Misclassification Patterns: Confusion matrix analysis revealed that "Negative," "Substantiated," and "Sarcastic" sentiments were frequently misclassified as "Positive." Conversely, the "Opinionated" class was over-predicted by the XGBoost model (305 predictions vs. 46 actual instances), likely due to class distribution biases in the training data.
• Comparison: Surprisingly, the traditional XGBoost model slightly outperformed the BERT-based transformer in this specific use case, suggesting that for this particular dataset size and language, the complexity of the transformer did not yield a performance advantage over the gradient-boosted tree approach.

Significance and Claims
The authors modestly frame the significance of this work as establishing a baseline for future research rather than claiming a solved problem. The low F1-scores demonstrate the inherent difficulty of classifying nuanced political sentiment in a low-resource language like Tamil. The paper asserts that these results provide a foundational reference point for the DravidianLangTech community and future researchers aiming to improve political opinion analysis.

The authors acknowledge limitations, noting that the small dataset size may hinder generalization to unseen data. They propose that future work should focus on expanding the dataset size and diversity, exploring additional languages, and testing other machine learning and deep learning algorithms to develop more robust, generalizable models for political discourse analysis.