Sentiment Analysis for Education with R: packages, methods and practical applications

This paper reviews and compares various R packages and methods for conducting Sentiment Analysis in educational contexts, demonstrating their practical application in processing student feedback to monitor teaching effectiveness and enhance learning experiences.

The Gist

Imagine you are a teacher standing in front of a giant, noisy classroom. At the end of the semester, you ask your students for feedback. Instead of just giving you a simple "thumbs up" or "thumbs down" on a piece of paper, they write thousands of comments on a digital board. Some say, "This class was amazing!" while others grumble, "The lectures were boring and confusing."

Reading through thousands of handwritten notes is impossible for a human. You'd need a superpower to read them all instantly and understand the mood of the room. This is where Sentiment Analysis (SA) comes in. It's like a digital mood ring for text. It uses computers to read words and decide if they are happy (positive), sad (negative), or neutral.

This paper is essentially a review of five different "Mood-Reading Tools" (software packages) built for the R programming language, specifically designed to help educators understand student feedback better.

Here is a breakdown of the paper using simple analogies:

1. The Problem: Too Many Words, Not Enough Time

In the past, teachers only looked at numbers (like a 1-to-5 rating). But numbers don't tell the whole story. Students write long comments full of feelings.

• The Challenge: Computers are great at math, but they struggle with human language. A phrase like "The test was not easy" is tricky. Is it negative? Or is it a compliment that the test was hard?
• The Solution: The paper tests five different software tools (packages) to see which one is best at reading these student comments and figuring out if the student is happy or unhappy.

2. The Contestants: The Five "Mood Readers"

The authors tested five popular tools. Think of them as five different detectives trying to solve the mystery of "How do students feel?"

• syuzhet (The Storyteller):

  • How it works: It uses pre-made lists of words (dictionaries) to count happy vs. sad words. It's like a detective who has a big book of "good words" and "bad words."
  • Special Power: It can tell you not just if a text is happy, but which emotion it has (anger, joy, fear, etc.). It's great for seeing the "emotional journey" of a story.
  • Weakness: It sometimes misses the nuance. If you say, "I didn't not like it," it might get confused.

• RSentiment (The Grammar Police):

  • How it works: It looks at the grammar. It checks if a word is being used as a noun or a verb, and it pays close attention to words like "not" or "very."
  • Special Power: It's very strict about grammar rules.
  • Weakness: It is slow. It's like a detective who reads every single word very carefully but takes forever to finish the report.

• sentimentr (The Context Detective):

  • How it works: This is the star of the show. It doesn't just count words; it looks at the context. It understands "valence shifters"—words that flip the meaning.
  • The Magic: If a student says, "The class was not good," a simple tool might see "good" and think it's positive. sentimentr sees the "not" and knows the student is actually unhappy. It's like a detective who understands sarcasm and double negatives.
  • Result: It was the most accurate at guessing the true mood of the students.

• SentimentAnalysis (The Statistician):

  • How it works: It uses heavy math and statistics to compare words against huge databases. It's very flexible and can be customized.
  • Weakness: It can be a bit heavy and complex to set up, like a high-tech lab that requires a PhD to operate.

• meanr (The Speedster):

  • How it works: It is incredibly fast. It just counts the good words and subtracts the bad words.
  • Weakness: It's fast but "dumb." It doesn't understand context or grammar. It's like a robot that just counts "happy" faces and "sad" faces without reading the sentences. It was the fastest but made the most mistakes.

3. The Race: Speed vs. Accuracy

The authors put these tools to the test using real student comments from a university. They measured two things:

1. Speed: How long did it take to read the comments?
2. Accuracy: How often did the tool guess the student's mood correctly compared to the actual truth?

The Results:

• The Fastest: meanr finished the job in a blink of an eye. But, like a fast driver who doesn't look at the road, it made many mistakes.
• The Most Accurate: sentimentr took a little longer (but still very fast), but it got the answer right most of the time. Because it understood context (like the word "not"), it didn't get tricked by tricky sentences.
• The Middle Ground: syuzhet and RSentiment were okay, but they either lacked the context understanding of sentimentr or were too slow.

4. Why This Matters for Education

Imagine a school principal trying to improve a course.

• If they use the Speedster (meanr), they might think a class is great because the tool counted more "good" words, even though the students were actually complaining about the teacher.
• If they use the Context Detective (sentimentr), they will see that while students used the word "hard," they also said "challenging and fun," realizing the students actually loved the class.

The Bottom Line

This paper is a guide for teachers and researchers. It says: "Don't just grab the fastest tool. If you want to truly understand what your students are feeling, use sentimentr."

It's like choosing between a speed-reading machine and a thoughtful human reader. For a quick glance, the machine is fine. But if you want to understand the heart of the feedback, you need the tool that understands the nuance of human language.

Technical Summary

1. Problem Statement

The paper addresses the challenge of processing large volumes of unstructured textual data generated in educational contexts, specifically student feedback from post-course surveys, forums, and social media. While quantitative feedback (e.g., Likert scales) is easily analyzed, qualitative open-ended comments contain rich, nuanced insights regarding teaching effectiveness and student motivation that are difficult to process manually. The authors aim to bridge the gap between educational research and computational linguistics by providing a comparative review of Sentiment Analysis (SA) techniques available in the R programming language. The goal is to help researchers and practitioners select the most appropriate tools for detecting the semantic orientation (polarity) of student opinions.

2. Methodology

The study employs a comparative experimental design using a specific dataset and five prominent R packages.

• Dataset: The authors utilized a subset of the Student Course Evaluation Comments dataset (Welch & Mihalcea, 2017), specifically the course_evaluations subset from the sentimentr package. This dataset contains 566 student comments labeled with ground-truth polarity: 300 positive (+1), 217 negative (-1), and 49 neutral (0).
• Packages Analyzed: Five R packages were selected based on popularity and functionality:
  1. syuzhet: Uses various lexicons (syuzhet, AFINN, Bing, NRC) and calculates scores via algebraic summation. It supports emotional categorization (NRC lexicon) and trajectory plotting.
  2. RSentiment: Uses Parts-of-Speech (POS) tagging and the Hu & Liu lexicon. It accounts for negations and amplifiers but uses a simpler scoring logic.
  3. sentimentr: A sentence-level analyzer that incorporates valence shifters (negators, amplifiers, de-amplifiers, adversative conjunctions) to adjust polarity scores dynamically. It aggregates scores to the document level.
  4. SentimentAnalysis: Heavily relies on the tm package for preprocessing. It offers rule-based scoring, customizable dictionaries, and statistical methods (e.g., LASSO) to generate dictionaries based on response variables.
  5. meanr: A lightweight, fast package using the Hu & Liu lexicon. It calculates scores as the difference between positive and negative term counts without complex preprocessing or customization options.
• Evaluation Metrics:
  • Performance: Measured by running time (User, System, and Elapsed time) on a MacBook Pro.
  • Accuracy: Measured by converting continuous/discrete scores into categorical labels (Positive, Neutral, Negative) and comparing them against the ground truth. Metrics included Misclassification Rate, Overall Accuracy, Precision, Recall, and F1-Measure (macro-averaged).

3. Key Contributions

• Comprehensive Package Review: The paper provides a structured technical overview of the five most popular R packages for SA, detailing their underlying algorithms, default lexicons, and specific functions (e.g., get_sentiment, calculate_score, sentiment_by).
• Methodological Comparison: It highlights the fundamental differences in how packages handle linguistic complexity. Specifically, it contrasts dictionary lookup methods (simple summation) with valence shifter methods (context-aware adjustment), explaining why the latter is crucial for accurate sentiment detection in complex sentences.
• Educational Application Framework: The authors demonstrate how these tools can be applied to educational data, moving beyond simple quantitative analysis to extract semantic orientation from qualitative student feedback.
• Customization Capabilities: The review clarifies which packages allow for custom dictionaries and weighted scoring, a critical feature for domain-specific applications like education where general lexicons may not capture specific pedagogical terminology.

4. Results

The comparative study yielded distinct trade-offs between speed and accuracy:

• Performance (Speed):

  • meanr was the fastest (0.003s elapsed) due to its simple counting logic and parallel processing via OpenMP.
  • syuzhet was the second fastest (0.210s).
  • sentimentr and SentimentAnalysis had moderate runtimes (~0.25s and ~1.6s, respectively).
  • RSentiment was the slowest (67.3s), largely attributed to on-screen printing of processing steps in the default configuration.

• Accuracy:

  • sentimentr achieved the highest accuracy (75.8% Overall Accuracy) and the lowest misclassification rate (24.20%). It also had the highest F1-Measure (0.626).
  • RSentiment and syuzhet performed moderately well (68.9% and 67.1% accuracy, respectively).
  • meanr and SentimentAnalysis produced the least accurate results (62.7% and 55.7% accuracy, respectively).
  • Bias Observation: All packages tended to overestimate positive sentiment, misclassifying neutral or negative comments as positive.

• Key Finding: The superior performance of sentimentr is attributed to its ability to model valence shifters. By adjusting the polarity of words based on context (e.g., "not good" vs. "very good"), it captures semantic nuances that simple dictionary lookups miss.

5. Significance

• Practical Guidance for Researchers: The paper serves as a decision-making tool for educational researchers. It suggests that if speed is the primary constraint and the text is simple, meanr or syuzhet are suitable. However, for high-accuracy analysis of complex student feedback where context matters, sentimentr is the recommended tool despite a slightly higher computational cost.
• Advancement of Learning Analytics: By validating the use of SA in education, the paper encourages the shift from purely quantitative evaluation to mixed-method approaches that leverage the semantic depth of student comments.
• Reproducibility and Open Science: By providing code snippets and comparing open-source R packages, the authors promote reproducible research in the field of Educational Data Mining (EDM).
• Limitations and Future Work: The authors acknowledge that using default lexicons (which vary by package) is a limitation, as different dictionaries yield different results. They suggest that future work should focus on creating domain-specific lexicons for education to further improve accuracy.

In conclusion, the paper establishes that while no single "one-size-fits-all" solution exists, sentimentr currently offers the best balance of accuracy and functionality for analyzing educational sentiment in R, primarily due to its sophisticated handling of linguistic context.