A Unified BERT-CNN-BiLSTM Framework for Simultaneous Headline Classification and Sentiment Analysis of Bangla News

Technical Summary: A Unified BERT-CNN-BiLSTM Framework for Simultaneous Headline Classification and Sentiment Analysis of Bangla News

Problem Statement
The rapid expansion of online Bangla news platforms has created a challenge in efficiently navigating vast amounts of content. While automatic text analysis is well-established for high-resource languages like English, Bangla remains a low-resource language in Natural Language Processing (NLP). Existing approaches face three primary limitations: (1) reliance on static word embeddings (e.g., TF-IDF, Word2Vec) and shallow architectures that fail to capture deep contextual semantics; (2) a tendency to treat headline classification and sentiment analysis as separate tasks rather than a joint problem; and (3) insufficient handling of class imbalance in available datasets, which leads to overfitting and poor generalization. The specific dataset used in this study, BAN-ABSA, contains 9,014 headlines but suffers from severe class imbalance, exacerbating these issues.

Methodology
The authors propose a unified hybrid framework integrating BERT, Convolutional Neural Networks (CNN), and Bidirectional Long Short-Term Memory (BiLSTM) networks to perform simultaneous headline classification and sentiment analysis.

• Architecture: The model utilizes a pre-trained BERT layer to generate contextualized embeddings, capturing both local token-level patterns and global sequence dependencies. These embeddings are processed through two parallel branches:
  • CNN Branch: Employs 1D convolution with multiple kernel sizes (2, 3, 4) to extract local n-gram features and position-invariant semantic patterns.
  • BiLSTM + Attention Branch: Models long-range sequential dependencies in both forward and backward directions. An attention mechanism is applied to weight the most informative tokens for the classification task.
  • Fusion: Features from both branches are concatenated, passed through fully connected layers with batch normalization and dropout, and finally classified using a Softmax layer.
• Data Balancing Strategies: To address the imbalance in the BAN-ABSA dataset without introducing data leakage or evaluation bias, the study employs a "train-only" modification strategy. The validation and test sets remain untouched. Four training regimes are evaluated:
  1. Original imbalanced dataset.
  2. Train-undersampled dataset.
  3. Train-oversampled dataset.
  4. Train-augmented dataset (using back-translation from Bangla to English and back).
• Training Details: The model is fine-tuned using the AdamW optimizer with label smoothing to prevent overconfidence. The study also utilizes 5-fold cross-validation and LIME (Local Interpretable Model-agnostic Explanations) for interpretability analysis.

Key Contributions

1. Unified Framework: The introduction of a BERT-CNN-BiLSTM architecture capable of jointly learning topical categories (Religion, Politics, Sports, Other) and sentiment polarity (Positive, Negative, Neutral) within a single model.
2. Imbalance Mitigation: A systematic evaluation of data balancing strategies (undersampling, oversampling, and back-translation augmentation) applied strictly to the training set to ensure fair evaluation on the original distribution.
3. State-of-the-Art Performance: The framework achieves superior performance compared to traditional deep learning models and baseline CNN/BiLSTM approaches on the BAN-ABSA dataset.
4. Robustness Validation: The model's reliability is confirmed through external dataset validation (Potrika and Advanced Sentiment datasets) and interpretability analysis.

Experimental Results

• Headline Classification: The best performance was achieved using the original imbalanced dataset (without balancing), yielding a test accuracy of 81.37% and an F1-score of 81.54%. This suggests that preserving the natural semantic diversity of the data was more beneficial than artificial balancing for this specific task.
• Sentiment Analysis: The best performance was achieved using the train-augmented dataset (back-translation), yielding a test accuracy of 66.40% and an F1-score of 63.86%.
• Comparison: The proposed model outperformed frozen BERT models (which suffered from severe overfitting with training accuracies >99% but low test accuracy) and standalone deep learning models (CNN, LSTM, BiLSTM). For instance, the proposed model improved headline classification accuracy from the baseline 78.75% to 81.37%.
• Generalization: The model demonstrated strong generalization on external datasets, achieving 84.34% accuracy on the Potrika headline dataset and 73.05% on the Advanced Sentiment dataset.
• Interpretability: LIME analysis confirmed that the model relies on domain-specific terms for classification and contextual semantic information rather than isolated emotional words for sentiment prediction.

Significance and Claims
The paper claims that this work sets a high reference point for the BAN-ABSA dataset and offers an effective methodology for handling class imbalance in low-resource Bangla text classification without inducing evaluation bias. By combining contextual embeddings, local feature extraction, and sequential learning, the framework successfully addresses the limitations of static embeddings and single-task learning. The authors emphasize that their approach provides a robust, generalizable solution that balances training performance with test reliability, avoiding the overfitting pitfalls observed in frozen BERT models on this specific dataset. The study concludes that while the original imbalanced data is optimal for headline categorization, data augmentation via back-translation is the most effective strategy for sentiment analysis in this context.