A Comparative Analysis of Classical Machine Learning and Deep Learning Approaches for Sentiment Classification on IMDb Movie Reviews

Technical Summary: A Comparative Analysis of Classical Machine Learning and Deep Learning Approaches for Sentiment Classification on IMDb Movie Reviews

Problem Statement
Sentiment analysis is a critical task in Natural Language Processing (NLP) aimed at determining the polarity of textual data, such as movie reviews. While deep learning (DL) approaches have gained prominence for their ability to automatically learn feature representations and capture sequential dependencies, they often demand significant computational resources and training time. Conversely, classical machine learning (ML) methods are known for their efficiency but rely heavily on manual feature engineering. This study addresses the need to understand the trade-offs between these two paradigms in terms of performance, efficiency, and practicality when applied to the IMDb movie reviews dataset.

Methodology
The research employs a comparative framework evaluating classical ML and DL pipelines on a balanced subset of the IMDb dataset (50,000 samples, split evenly between positive and negative sentiments).

• Data Preprocessing:
  • ML Pipeline: Utilized an EDA-based pipeline involving text normalization and TF-IDF (Term Frequency–Inverse Document Frequency) feature extraction.
  • DL Pipeline: Implemented text cleaning (lowercasing, URL/HTML removal, non-alphabet filtering), stratified sampling of 10,000 instances, label encoding, and sequence padding.
• Machine Learning Approach:
  • Implemented using PyCaret AutoML with TF-IDF vectorization.
  • Evaluated models included Logistic Regression, Naïve Bayes, and Support Vector Machine (SVM).
• Deep Learning Approach:
  • Implemented using PyTorch on a subset of 10,000 samples.
  • Evaluated two architectures:
    1. Bidirectional LSTM (BiLSTM): To capture sequential dependencies in both forward and backward directions.
    2. BiLSTM with Attention Mechanism: To enhance contextual representation by focusing on significant words within the sequence.
  • Hyperparameters included a vocabulary size of 5,000, embedding dimension of 64, 1 hidden layer, dropout rate of 0.3, and training for 3 epochs with early stopping.

Key Contributions
The paper outlines four primary contributions:

1. Dual Preprocessing Pipelines: Distinct workflows for ML (Jupyter Notebook-based EDA and TF-IDF) and DL (Python script-based cleaning and stratified sampling).
2. ML Model Deployment: A machine learning model built with PyCaret and serialized as sentiment_model.pkl for efficient inference.
3. DL Framework: A comparative implementation of BiLSTM and BiLSTM with Attention architectures.
4. Interactive Deployment: Both models are made accessible via an interactive Hugging Face Space.

Results
Experimental results indicate a divergence in performance based on the dataset size and model complexity:

• Classical ML Performance: The SVM model achieved the highest accuracy of 0.8530, closely followed by Logistic Regression (0.8518). Naïve Bayes performed significantly lower (0.6106).
• Deep Learning Performance: The standard BiLSTM achieved an accuracy of 0.626. The BiLSTM with Attention mechanism improved performance to 0.706, demonstrating better contextual understanding than the standard BiLSTM.
• Comparison: Contrary to some literature suggesting DL superiority, the classical ML models outperformed the DL models in this specific experimental setup.

Significance and Claims
The study concludes that while deep learning models possess the theoretical capacity to capture complex sequential dependencies, classical machine learning approaches remain strong baselines for sentiment classification, particularly when combined with effective feature engineering like TF-IDF.

The authors claim that the superior performance of ML models in this study is likely due to:

1. Data Constraints: The DL models were trained on a reduced subset (10,000 samples) compared to the full dataset used for ML, which may have been insufficient for learning complex representations.
2. Training Limitations: The DL models were trained for a limited number of epochs (3), potentially preventing convergence to optimal performance.
3. Efficiency: ML models offer strong performance with lower computational costs and faster training times.

The paper emphasizes that model selection should be dictated by the specific constraints of the application, including dataset size, available computational resources, and the need for interpretability. It suggests that for scenarios with limited resources or smaller datasets, well-optimized classical ML models may be more practical than deep learning alternatives.