Benchmarking Logistic Regression, SVM, Naive Bayes, and IndoBERT Fine-Tuning for Sentiment Analysis on Indonesian Product Reviews

Technical Summary: Benchmarking Logistic Regression, SVM, Naive Bayes, and IndoBERT for Indonesian Sentiment Analysis

Problem Statement
The rapid expansion of e-commerce in Indonesia has generated a massive volume of unstructured, user-generated product reviews. Analyzing the sentiment of these reviews is critical for assessing customer satisfaction and identifying product issues at scale. However, Indonesian text presents unique challenges, including informal spelling, slang, inconsistent punctuation, and contextual expressions. Furthermore, real-world marketplace data exhibits severe class imbalance, where positive reviews vastly outnumber neutral and negative feedback. This imbalance complicates model training, often leading to high accuracy metrics that mask poor performance on minority classes. The core problem addressed is determining whether traditional Machine Learning (ML) approaches, which are computationally efficient, can outperform or match modern Transformer-based Deep Learning models (specifically IndoBERT) in this noisy, imbalanced, and domain-specific context.

Methodology
The study implements a comparative benchmarking framework using the Tokopedia Product Reviews 2025 dataset, which contains tens of thousands of short, noisy reviews labeled into three classes: positif (positive), netral (neutral), and negatif (negative).

1. Preprocessing Pipeline: A robust pipeline was developed to clean Indonesian e-commerce vernacular. Steps included converting text to lowercase, removing URLs and non-alphanumeric characters (including emojis), normalizing whitespace, and mapping heterogeneous labels to standardized classes.
2. Baseline Models (Traditional ML): Three algorithms were implemented using TF-IDF feature extraction with unigrams and bigrams ($ngram\_range=(1,2)$). The vocabulary was capped at 50,000 features. The models included:
   • Logistic Regression
   • Linear Support Vector Machine (Linear SVC)
   • Multinomial Naive Bayes
   • Imbalance Handling: Discriminative linear models utilized class_weight="balanced" to penalize errors on minority classes.
3. Deep Learning Model (IndoBERT): The study fine-tuned the indobenchmark/indobert-base-p1 model.
   • Architecture: A Transformer encoder followed by a sequence classification head, processing inputs up to 256 tokens.
   • Imbalance Handling: A custom weighted cross-entropy loss function was engineered, where class weights ($W_c$) are inversely proportional to class frequency ($W_c = \frac{N}{C \times n_c}$).
   • Training: The model was trained for 3 epochs with a batch size of 16, learning rate of $2 \times 10^{-5}$, and a warmup ratio of 0.1.
4. Experimental Discrepancy: A critical methodological note is the difference in data regimes. Baseline models were trained on the full dataset (65,335 records), while the IndoBERT model was constrained to a stratified subset (~1,800 training records) due to computational limitations.

Key Contributions
The paper details five specific contributions:

1. A preprocessing pipeline tailored via regular expressions for noisy Indonesian e-commerce text.
2. A comprehensive benchmark of TF-IDF baselines (Logistic Regression, LinearSVC, Naive Bayes).
3. A detailed implementation of IndoBERT fine-tuning with a dynamically calculated weighted loss to address class imbalance.
4. An in-depth quantitative analysis comparing aggregate performance against class-sensitive metrics.
5. A reproducible deployment strategy using Gradio on Hugging Face Spaces.

Results
The evaluation utilized Accuracy, Macro F1-score, and Weighted F1-score. Contrary to the expectation that Deep Learning models would outperform traditional methods, the results showed that Linear SVC significantly outperformed the fine-tuned IndoBERT model in this specific experimental setup.

• Linear SVC: Achieved 97.60% Accuracy and a 0.5510 Macro F1-score.
• IndoBERT: Achieved 88.70% Accuracy and a 0.5088 Macro F1-score.
• Other Baselines: Logistic Regression (94.36% Acc, 0.5164 Macro F1) and Naive Bayes (97.50% Acc, 0.3290 Macro F1) also performed strongly, though Naive Bayes struggled significantly with minority class recognition.

The analysis revealed that while baseline models achieved high overall accuracy, they still struggled with the netral and negatif classes, often confusing them with the dominant positif class. However, the performance gap between the baselines and IndoBERT was attributed primarily to the data sampling regime: the baselines utilized the full corpus, whereas the Transformer was restricted to a small subset, preventing it from fully leveraging its contextual modeling capabilities.

Significance and Claims
The paper claims that for Indonesian sentiment analysis on e-commerce reviews, traditional Machine Learning approaches (specifically TF-IDF with Linear SVC) remain highly effective and robust, often outperforming state-of-the-art Transformers when computational constraints limit the training data for the latter.

The authors emphasize that the "Transformer advantage" is not automatic; it depends heavily on data volume, class balance, and fair evaluation setups. The study concludes that while IndoBERT offers superior theoretical contextual modeling, its empirical performance in this study was bottlenecked by the restricted training subset. Consequently, for applications prioritizing speed and robust baseline performance, TF-IDF with Linear SVC is presented as a highly viable solution. The paper advocates for future research to ensure fair comparisons by training all models on identical large-scale data splits and employing advanced imbalance-aware strategies to improve minority-class recognition.