Benchmarking LightGBM and BiLSTM for Sentiment Analysis on Indonesian E-Commerce Reviews

Technical Summary: Benchmarking LightGBM and BiLSTM for Indonesian E-Commerce Sentiment Analysis

Problem Statement
The rapid expansion of e-commerce in Indonesia has generated vast amounts of unstructured textual data in the form of product reviews. Analyzing this data is critical for businesses to gauge customer satisfaction automatically. However, Indonesian e-commerce reviews present unique linguistic challenges, including heavy usage of informal language, code-mixing, slang, and implicit emotional expressions such as sarcasm (e.g., positive vocabulary masking negative experiences). The core problem addressed by this study is determining whether traditional Machine Learning (ML) approaches, specifically those optimized via automated frameworks, or Deep Learning (DL) architectures are more effective at capturing the sequential context and nuances required for accurate sentiment classification in this specific domain.

Methodology
The study employs a comparative experimental design utilizing a dataset of 15,000 Indonesian e-commerce reviews sourced from Hugging Face, balanced across Positive, Negative, and Neutral classes. The data underwent rigorous preprocessing, including case folding, noise removal, character filtering, and slang normalization to map abbreviated terms to standard vocabulary.

The research benchmarks two distinct pipelines:

1. Machine Learning (ML) Pipeline: Utilizing the PyCaret AutoML framework, the study automates the training and tuning of three algorithms: LightGBM, Logistic Regression, and Support Vector Machine (SVM) with a linear kernel. Feature extraction was performed using TF-IDF (Term Frequency-Inverse Document Frequency), capped at 5,000 features. The models were evaluated using stratified 10-fold cross-validation with automated hyperparameter tuning optimized for F1-Score.
2. Deep Learning (DL) Pipeline: A Bidirectional Long Short-Term Memory (BiLSTM) architecture was implemented using the PyTorch framework. The model consists of an embedding layer (dimension 128), a BiLSTM layer (hidden state size 64), spatial dropout layers (rate 0.3) to prevent overfitting, and fully connected layers with Softmax activation. The model was trained with the Adam optimizer (learning rate $1 \times 10^{-3}$) over 20 epochs with a batch size of 32.

Key Results
The experimental results demonstrate that while both approaches yield high performance, the Deep Learning model achieves superior accuracy:

• BiLSTM (DL): Achieved the highest performance with an accuracy of 98.87% and an F1-Score of 98.87%. The model showed consistent precision and recall (0.99) for Negative and Neutral classes, and 0.98/0.99 for the Positive class. The training time was approximately 3.7 minutes.
• LightGBM (ML): Emerged as the best-performing ML model with an accuracy of 98.23% and an F1-Score of 0.9823. It demonstrated exceptional computational efficiency, completing the training and cross-validation process in just 4.932 seconds.
• Other ML Models: Logistic Regression and SVM (Linear Kernel) achieved accuracies of 97.67% and 97.56%, respectively, establishing strong baselines but falling slightly short of LightGBM.

Key Contributions

• Empirical Benchmarking: The study provides direct empirical evidence comparing automated ML (PyCaret) against a custom DL architecture (BiLSTM) specifically for Indonesian e-commerce sentiment analysis.
• Handling Linguistic Nuance: The research validates that the BiLSTM architecture is particularly effective at capturing sequential dependencies and contextual nuances (such as sarcasm and negation) inherent in informal Indonesian text, outperforming the TF-IDF-based ML models which rely on bag-of-words assumptions.
• Deployment Accessibility: The authors provide the complete source code, dataset, and links to deployed Hugging Face models via a public GitHub repository, alongside an interactive web interface for model deployment.

Significance and Claims
The paper concludes that the BiLSTM architecture is the superior model for this specific classification task due to its ability to maintain a high balance between precision and recall across all sentiment classes. The authors assert that while ML models like LightGBM offer a highly scalable and rapid solution (ideal for environments requiring frequent retraining), the BiLSTM model is the optimal choice when maximum prediction quality and the ability to interpret complex, heterogeneous language are prioritized.

The study does not claim to solve all challenges in Indonesian NLP but specifically highlights that for e-commerce reviews characterized by slang and implicit sentiment, the bidirectional sequential learning mechanism of BiLSTM provides a measurable advantage over traditional gradient boosting methods. The research underscores the trade-off between computational efficiency and predictive nuance, suggesting that the choice of model should be dictated by the specific deployment context (speed vs. accuracy).