Hybrid TF--IDF Logistic Regression and MLP Neural Baseline for Indonesian Three-Class Sentiment Analysis on Social Media Text

Technical Summary: Hybrid TF–IDF Logistic Regression and MLP Neural Baseline for Indonesian Three-Class Sentiment Analysis

Problem Statement
This study addresses the challenge of sentiment analysis for Indonesian social media text, a domain characterized by noisy, code-mixed, abbreviated, and semantically ambiguous user-generated content. While recent literature often prioritizes large transformer models, this paper argues that for small, imbalanced, and low-resource datasets, the immediate need is for a reproducible, interpretable, and deployable pipeline rather than increased architectural complexity. The specific task involves mapping a dataset of 191 fine-grained emotion labels into three operational classes: positive, negative, and neutral. The primary difficulties identified are severe class imbalance (dominated by positive instances), semantic ambiguity introduced by label compression (e.g., mapping "surprise" or "bittersweet" to a single class), and the limitations of small sample sizes for deep learning.

Methodology
The authors propose a production-first approach comparing a classical machine learning baseline against a lightweight neural baseline, both utilizing an identical hybrid feature space.

• Data Preprocessing: The raw dataset (732 rows) was cleaned to remove duplicates and empty texts, resulting in 707 samples. Preprocessing included lowercasing, URL/mention removal, leetspeak normalization, and Indonesian slang expansion (e.g., gak to tidak). Crucially, the 191 original emotion labels were remapped into the three target classes, a process noted to introduce semantic noise.
• Feature Engineering: The system employs a hybrid representation combining:
  1. Text Features: TF–IDF vectors using unigrams and bigrams (max 3,000 features, sublinear term frequency scaling).
  2. Numeric Metadata: Three standardized features derived from the text and engagement data: word count, total engagement (likes + retweets), and hashtag count.
• Models:
  • Production Model: A multinomial Logistic Regression classifier trained with class_weight=balanced to mitigate imbalance, using an 80/20 stratified split.
  • Neural Baseline: A Multilayer Perceptron (MLP) with two hidden layers (256, 64 neurons), ReLU activation, and Adam optimization. It was trained on the exact same hybrid feature vector to isolate the effect of the classifier architecture rather than feature representation.
  • Comparative Baselines: The study also preserved results for Linear SVM and Random Forest for benchmarking.

Key Contributions
The paper makes three specific contributions:

1. Reproducible Workflow: It formalizes an end-to-end pipeline for Indonesian three-class sentiment analysis, covering cleaning, label remapping, hybrid feature construction, and deployment-ready inference.
2. Controlled Comparison: It evaluates a classical linear model and a shallow neural network under the same input representation, providing a controlled comparison of classifier families in a low-resource setting.
3. Methodological Insight: It highlights the implications of label compression and data imbalance, demonstrating that careful preprocessing and interpretable feature engineering can remain competitive with neural approaches for small datasets.

Results
Experiments were conducted on a held-out test set (142 samples) using accuracy, weighted F1, and macro F1 as metrics.

• Logistic Regression (Production): Achieved 0.8028 accuracy, 0.8003 weighted F1, and 0.7276 macro F1. It performed best on the majority "positive" class but struggled with the minority "neutral" class due to semantic ambiguity and low sample count.
• MLP Baseline: Attained higher accuracy (0.8451) and weighted F1 (0.8304) but did not significantly outperform Logistic Regression in macro F1 (0.7263). This suggests that while the neural model improved performance on majority classes, it did not resolve the ambiguity of the minority classes.
• Linear SVM: Recorded the highest performance among all preserved models (0.8521 accuracy, 0.7553 macro F1), though it was not selected as the primary production model.
• Random Forest: Performed substantially worse, particularly in macro F1 (0.4821), indicating a bias toward the majority class.

Significance and Claims
The paper modestly claims that for compact Indonesian sentiment datasets, a carefully engineered classical pipeline remains a highly effective and preferable choice for production environments. The study argues that Logistic Regression offers a pragmatic trade-off: it is easier to serialize, interpret, and retrain than neural models, while delivering stable probabilistic outputs.

The authors conclude that while neural baselines (and Linear SVM) offer potential for higher accuracy, the gains are often driven by majority-class performance rather than solving the core challenge of minority-class ambiguity. Therefore, the neural baseline is best treated as a comparative experiment rather than the default deployment model. The study emphasizes that in low-resource settings, the quality of preprocessing, label remapping, and class balancing is often more critical than the choice of a complex model architecture. Future work is suggested to include cross-validation, qualitative error analysis, and testing stronger contextual models like IndoBERT, but the current work establishes a robust, interpretable baseline.