Sentiment and Emotion Classification of Indonesian E-Commerce Reviews via Multi-Task BiLSTM and AutoML Benchmarking

1. Problem Statement

The paper addresses the challenge of analyzing Indonesian e-commerce reviews, which are characterized by highly informal language. Unlike standardized text, these reviews contain:

• Linguistic Complexity: A mix of standard vocabulary, slang, regional loanwords, numeric shorthands (e.g., "50k"), and emojis.
• Domain Specificity: Frequent use of abbreviations (e.g., "ongkir" for shipping costs, "ori" for original) and spelling variations.
• Limitations of Existing Tools: Traditional lexicon-based sentiment analysis tools fail in this context because they cannot resolve slang or handle the noise inherent in user-generated content.
• Data Scarcity: There is a lack of large-scale, dual-labeled datasets for Indonesian text that cover both sentiment (Positive/Negative) and specific emotions (Happy, Sad, Fear, Love, Anger).

2. Methodology

The authors propose a two-track classification pipeline applied to the PRDECT-ID dataset (5,400 reviews across 29 product categories).

A. Data Preprocessing

A rigorous preprocessing module (src/preprocessing.py) was developed, featuring 14 sequential cleaning steps. Key components include:

• A custom 140-entry slang dictionary assembled from marketplace corpora to normalize informal terms.
• Handling of negation, intensifiers, transaction-specific vocabulary, and emojis.
• Tokenization and normalization to prepare text for both ML and DL models.

B. Track 1: Traditional Machine Learning (AutoML)

• Feature Extraction: TF-IDF vectorization.
• Modeling: Utilization of PyCaret AutoML to sweep across standard classifiers (Logistic Regression, Random Forest, LightGBM, SVM, etc.).
• Output: The best-performing models for sentiment and emotion are serialized for deployment.

C. Track 2: Deep Learning (Multi-Task Learning)

The authors implemented a Multi-Task Learning (MTL) framework using PyTorch, designed to predict sentiment and emotion simultaneously.

• Architecture: A shared encoder with two task-specific output heads.
• Configurations Benchmarked:
  1. BiLSTM Baseline: 1 layer, 128 hidden units (~1.4M params).
  2. BiLSTM Improved: 2 layers, 256 hidden units, includes BatchNorm1d (~3.1M params).
  3. BiLSTM Large: 2 layers, 256 embedding/hidden units, additional dense layer (~5.3M params).
  4. TextCNN: Parallel 1D convolutions with kernel sizes {2, 3, 4} and global max-pooling (~1.7M params).
• Training Strategy:
  • Loss: Class-weighted cross-entropy to handle label imbalance (especially between "Happy" and "Anger").
  • Optimization: Adam optimizer with ReduceLROnPlateau scheduling and Early Stopping.
  • Data Split: 80% training, 10% validation, 10% test (stratified by emotion).

3. Key Contributions

1. Preprocessing Module: A robust cleaning pipeline with a specialized 140-entry slang dictionary tailored for Indonesian e-commerce.
2. AutoML Benchmark: A comprehensive evaluation of classical ML models using TF-IDF and PyCaret, providing serialized baselines.
3. Multi-Task Deep Learning Framework: Implementation of four distinct neural architectures (3 BiLSTM variants and 1 TextCNN) sharing a single encoder for joint sentiment/emotion prediction.
4. Build-Factory Registry: A modular code structure (src/model.py) allowing easy switching between model configurations via a single string key.
5. Public Deployment: Two interactive Gradio applications deployed on Hugging Face Spaces for both ML and DL demos, with source code available on GitHub.

4. Results

Sentiment Classification (Binary: Positive/Negative)

• Winner: TF-IDF + Best AutoML (specifically a classical model from the PyCaret sweep).
• Performance: Achieved 0.9574 across Accuracy, Precision, Recall, and F1-score.
• Deep Learning Performance: The BiLSTM Baseline was the strongest DL model but lagged significantly (F1 ~0.86).
• Insight: For binary sentiment in this domain, frequency-based features (TF-IDF) combined with classical classifiers outperform deep sequential models, likely due to strong lexical polarity cues.

Emotion Classification (5-Class: Happy, Sad, Fear, Love, Anger)

• Winner: TextCNN.
• Performance: Achieved 0.5399 Accuracy and 0.5077 Macro-F1.
• Deep Learning vs. Classical: Deep learning models significantly outperformed the TF-IDF/AutoML baseline (which scored ~0.24 Macro-F1).
• Insight: Emotion classification is harder than sentiment. The success of TextCNN suggests that local n-gram features captured by convolutional filters are more effective than sequential dependencies for distinguishing subtle emotional nuances in short Indonesian reviews.
• Error Analysis: Confusion is highest between semantically adjacent classes (e.g., Fear vs. Sad, Love vs. Happy).

5. Significance and Conclusion

• Practical Utility: The study demonstrates that while deep learning is essential for complex emotion detection, classical ML remains highly competitive for binary sentiment analysis in noisy, informal Indonesian text.
• Methodological Rigor: The paper provides a reproducible benchmark for Indonesian NLP, highlighting the necessity of domain-specific preprocessing (slang normalization) before model training.
• Resource Availability: By releasing the code, models, and Hugging Face demos, the authors lower the barrier for future research in Indonesian e-commerce analytics.
• Future Direction: The results suggest that for emotion tasks, convolutional approaches may be more efficient than heavy recurrent architectures, and that hybrid approaches or transformer-based models (not tested here) could be the next step for further improvement.