💻 computer science

Class-Aware Semantic Hybrid Data Augmentation for Imbalanced Sentiment Classification Using Multiple Transformer Models

This paper proposes a Class-Aware Semantic Hybrid Data Augmentation (CSHDA) framework that selectively applies diverse, semantically validated augmentation techniques to minority classes, effectively mitigating class imbalance and significantly improving the performance and robustness of multiple transformer-based models on imbalanced sentiment classification tasks.

Original authors: Prashant Upadhyaya, G.L. Saini, Atul Makrariya

Published 2026-08-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Prashant Upadhyaya, G.L. Saini, Atul Makrariya

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the vast digital landscape of the internet, people constantly share their opinions, from reviews of a local restaurant to complaints about a delayed flight. Computers have become remarkably good at reading these texts and figuring out whether the sentiment behind them is happy, sad, or somewhere in between. This ability, known as sentiment analysis, helps businesses understand their customers and organizations track public mood. However, for a computer to learn this skill, it needs to be trained on a massive amount of examples. A significant hurdle arises when these examples are not evenly distributed. Imagine a classroom where ninety students are learning to identify red objects, but only ten are learning to identify blue ones. The teacher, overwhelmed by the red examples, might start guessing that everything is red, simply because that is what they see most often. In the world of data, this is called class imbalance, and it causes computer models to ignore minority opinions, such as neutral or negative feelings, in favor of the majority.

To fix this, researchers often try to create more examples for the minority groups, a process known as data augmentation. They take existing sentences and make small changes, like swapping a word for a synonym or removing a few letters, hoping to generate new, valid training data. While this has helped in the past, a new study suggests that simply making more copies is not enough if the changes are made blindly. The researchers found that applying the same random changes to every type of sentence can actually confuse the computer, creating examples that sound nothing like the original feeling or, worse, just repeating the same sentence over and over.

A team of researchers led by Prashant Upadhyaya, G.L. Saini, and Atul Makrariya has proposed a smarter way to handle this problem. They developed a system they call Class-Aware Semantic Hybrid Data Augmentation. Instead of treating all opinions the same, this system acts like a careful editor who understands the specific needs of each group. When the system encounters a minority sentiment, such as a rare neutral opinion in a sea of positive reviews, it does not just randomly chop words or swap them. Instead, it chooses specific techniques based on what that particular type of sentence needs to stay true to its meaning. For some sentences, it might use a powerful language model to rewrite the sentence in a different way while keeping the exact same feeling. For others, it might simply remove a word or insert a new one. Crucially, the system checks every new sentence it creates against the original to ensure the meaning hasn't drifted. If a new sentence sounds too different or is just a duplicate of something it has already made, the system discards it.

The researchers tested this approach on two very different sets of real-world data: a collection of food court reviews from a university and a massive dataset of tweets about airlines. In both cases, the data was heavily skewed, with one sentiment dominating the others. They trained four different advanced computer models, known as transformers, which are the current state-of-the-art tools for understanding language. First, they trained these models on the original, unbalanced data to see how they performed. As expected, the models struggled to correctly identify the minority opinions, often mislabeling negative or neutral comments as positive. Then, they applied their new augmentation method to the training data, generating high-quality, diverse examples only for the minority groups.

The results were striking. When the models were retrained on this improved data, their ability to recognize the minority sentiments jumped significantly. On the food review dataset, the overall accuracy of the models improved by a large margin, with the ability to spot negative reviews rising from a very low level to a strong majority. On the airline tweet dataset, which is notoriously difficult due to its short and informal nature, the improvement was even more dramatic for some models. One model, which had previously failed to recognize neutral tweets almost entirely, began identifying them correctly more than half the time. The researchers confirmed that these improvements were not just lucky guesses but were statistically significant, meaning the results were reliable and repeatable.

What makes this work particularly valuable is that it does not require changing the computer models themselves. It works as a preparatory step that cleans and enriches the data before the learning begins. The study also showed that this method works well across different types of models, from very large, complex ones to smaller, faster ones. By carefully balancing the creation of new data with strict checks to ensure the meaning remains intact, the researchers demonstrated that quality is far more important than quantity. They proved that by giving the computer a few more, but much better, examples of the rare opinions, it can learn to listen to everyone, not just the loudest voices in the room. This approach offers a practical path forward for making artificial intelligence more fair and accurate in understanding the full spectrum of human emotion.

Technical Summary: Class-Aware Semantic Hybrid Data Augmentation for Imbalanced Sentiment Classification

Problem Statement
Sentiment classification using transformer-based models faces a critical challenge: class imbalance. In real-world datasets, one sentiment class (e.g., positive or negative) often significantly outnumbers others, causing models to become biased toward the majority class. This results in poor recognition of minority sentiments, particularly "neutral" and "negative" categories. While existing data augmentation techniques (such as Easy Data Augmentation or EDA) increase sample sizes, they typically apply identical transformations across all classes without considering class-specific characteristics. This uniform approach often leads to semantic drift, the generation of redundant or noisy samples, and limited improvements in minority-class learning.

Methodology
The paper proposes a Class-Aware Semantic Hybrid Data Augmentation (CSHDA) framework designed to mitigate class imbalance while preserving semantic fidelity. The methodology operates through the following steps:

Targeted Augmentation: Unlike standard methods that augment all data, CSHDA applies augmentation exclusively to minority sentiment classes. The goal is to balance the dataset by generating samples until the minority class count matches the majority class count ( $T = \max_{c} n_c$ ).
Class-Aware Probability Distribution: The framework assigns specific augmentation probabilities ( $P_c$ $P_{c}$ ) for each minority class, selecting from four complementary techniques:
- BERT-based paraphrasing (BP)
- Synonym replacement (SR)
- Random insertion (RI)
- Random deletion (RD)
- Note: Probabilities are empirically tuned based on dataset characteristics (e.g., higher BP probability for neutral classes in structured reviews, higher RD for negative classes).
Semantic Consistency Validation: To prevent semantic drift, every generated candidate sentence is validated using Sentence-BERT (SBERT). The cosine similarity between the original and augmented sentence is calculated. A sample is accepted only if it falls within a specific threshold range ( $0.65 \leq \text{sim} \leq 0.97$ ), ensuring the augmented text remains semantically similar to the original but distinct enough to add diversity.
Duplicate-Aware Filtering: A repository of accepted sentences is maintained to ensure no duplicate augmented instances are added to the training set.
Model Evaluation: The augmented datasets are used to fine-tune four transformer architectures: BERT, RoBERTa, DistilBERT, and ALBERT. The study utilizes two benchmark datasets with distinct imbalance profiles: the CUFC corpus (food reviews, positive majority) and the Twitter Airline dataset (social media, negative majority).

Key Contributions
The paper outlines three primary contributions:

A Novel Framework: The proposal of CSHDA, which dynamically assigns augmentation probabilities based on minority sentiment characteristics rather than applying uniform transformations.
Integrated Quality Control: Unlike standard EDA, CSHDA simultaneously adapts probabilities, validates semantic consistency via SBERT, removes duplicates, and restricts augmentation to minority classes only.
Semantic Consistency Mechanism: The introduction of SBERT similarity thresholds to balance sentence diversity with contextual preservation, effectively preventing semantic drift.

Experimental Results
The framework was evaluated on the CUFC and Twitter datasets across five random seeds. Key findings include:

CUFC Dataset:
- BERT: Macro F1-score improved from 60.80% to 81.10%.
- DistilBERT: Macro F1-score improved from 55.43% to 80.01%.
- RoBERTa: Improved from 70.79% to 81.22%.
- Significant gains were observed in the recall of minority classes (Negative and Neutral), reducing the bias toward the majority positive class.
Twitter Dataset:
- BERT: Macro F1-score improved from 60.65% to 69.82%.
- DistilBERT: Showed the most dramatic improvement, rising from 42.98% to 71.21%.
- RoBERTa: While the baseline was already strong, the framework maintained robust performance with only a marginal decrease in Macro F1 (approx. 3.1%), indicating stability even for high-performing models.
Statistical Significance: Paired t-tests confirmed that the improvements in Macro F1-scores are statistically significant ( $p < 0.05$ ) with large effect sizes (Cohen's $d$ ranging from 1.76 to 2.85).
Ablation Study: Comparisons with individual EDA techniques (SR, RI, RD, RS) demonstrated that the hybrid CSHDA approach provided more consistent and generalized performance across different transformer architectures than any single augmentation method alone.

Significance and Claims
The authors claim that the CSHDA framework effectively addresses the limitations of current augmentation strategies by generating semantically consistent and diverse minority-class samples. The significance of the work lies in its ability to:

Mitigate Class Imbalance: By focusing augmentation solely on minority classes, the framework reduces learning bias without distorting the original majority class distribution.
Ensure Semantic Fidelity: The use of SBERT filtering ensures that augmented data improves model robustness without introducing noise or altering the intended sentiment.
Generalize Across Domains: The framework proved effective on both structured review data (CUFC) and noisy, informal social media text (Twitter), suggesting its applicability to cross-domain sentiment classification.
Improve Minority Recall: The approach specifically enhances the detection of neutral and negative sentiments, which are typically the most difficult classes to identify in imbalanced scenarios.

The paper concludes that while the framework introduces some computational overhead due to SBERT similarity checks, the trade-off yields statistically reliable and practically meaningful improvements in transformer-based sentiment classification. Future work is suggested to explore adaptive thresholding and more efficient semantic filtering techniques.

Technical Summary: Class-Aware Semantic Hybrid Data Augmentation for Imbalanced Sentiment Classification

More like this