What Makes a Review Read as Positive or Negative? Linguistic Signatures of Valence in Online Consumer and Cultural-Product Reviews

Technical Summary: Linguistic Signatures of Valence in Online Reviews

Problem Statement
Automated sentiment analysis tools increasingly govern the visibility, moderation, and prioritization of online consumer reviews. However, these systems often rely on sentiment lexicons and classifiers that treat valence as a property recoverable solely from sentiment scores. This study investigates a critical operational gap: whether the linguistic markers used to distinguish positive from negative reviews are stable across product categories or if they are category-specific. The core problem is that a feature predicting valence in one domain (e.g., film) may reverse its relationship or lose predictive power in another (e.g., consumer electronics), rendering "one-size-fits-all" classification rules unreliable for cross-category deployment.

Methodology
The study employs a comparative analysis of two independent, publicly available corpora to test the generalizability of six measurable text features:

1. Sentiment Intensity: VADER compound polarity scores.
2. Readability: Flesch Reading Ease and Flesch–Kincaid Grade Level.
3. Length: Total word count.
4. Lexical Diversity: Type–token ratio (TTR).
5. Affective Punctuation: Frequency of exclamation marks.
6. Average Word Length: Characters per word.

Corpora:

• Corpus 1 (Film): 2,000 balanced English-language film reviews (Pang & Lee, 2004), with valence derived from original star/letter grades.
• Corpus 2 (Consumer Electronics): 294 English-language product reviews (Hu & Liu, 2004) covering five electronics items. Valence was derived by summing feature-level sentiment scores, classifying reviews with positive net sums as positive and negative net sums as negative.

Analytic Strategy:
The authors utilized Welch's independent-samples t-tests and Mann-Whitney U tests to assess univariate group differences, reporting Cohen's d for effect sizes. Point-biserial correlations were computed between features and valence. To test the incremental predictive value of structural features beyond sentiment alone, multivariate binary logistic regression models were run for each corpus, with Variance Inflation Factors (VIFs) checked to ensure multicollinearity was not an issue.

Key Results
The analysis yielded three primary findings regarding the stability of linguistic signatures across domains:

1. Category-General Signals: Three features consistently distinguished valence in both domains with the same directional effect:

   • Sentiment Intensity: Positive reviews had significantly higher VADER scores in both corpora (strongest effect).
   • Readability: Contrary to intuitive "processing fluency" accounts, positive reviews were less readable (lower Flesch scores, higher grade levels) than negative reviews in both domains.
   • Lexical Diversity: Positive reviews exhibited lower lexical diversity (lower TTR) than negative reviews in both domains.

2. Category-Specific Reversals:

   • Affective Punctuation: The frequency of exclamation marks reversed direction entirely. In film reviews, exclamation marks were significantly more frequent in negative reviews (indicating indignant emphasis). In product reviews, they were significantly more frequent in positive reviews (indicating enthusiasm). This indicates punctuation rules cannot be deployed unvalidated across categories.

3. Multivariate Predictive Power:

   • Film Corpus: A full model including all six features significantly improved prediction over an intercept-only model (pseudo-R² = .12). Beyond sentiment, lexical diversity, average word length, review length, and exclamation mark frequency remained significant unique predictors.
   • Product Corpus: The full model also improved fit (pseudo-R² = .26), but once all predictors were entered simultaneously, only sentiment intensity remained a statistically significant independent predictor. Other features became statistically redundant, likely due to the smaller sample size and lower power of the product corpus.

Significance and Claims
The paper argues that review valence leaves a measurable linguistic signature beyond the presence of sentiment words, but this signature is only partially generalizable.

• Operational Implications for Triage: Relying solely on sentiment scores discards usable structural signal, particularly in longer-form review categories like film. However, because the directionality of features like punctuation can reverse, platforms must validate linguistic-valence rules within each specific category before deploying them broadly.
• Calibration of Heuristics: The study challenges the assumption that "clearer" or "more diverse" reviews are inherently more positive. In both tested domains, positive reviews were actually harder to read and less lexically diverse. Therefore, readability-based heuristics (e.g., "clearer reviews are more positive") would lead to misclassification.
• Modest Scope of Explanation: The authors emphasize that while these features are statistically significant, they explain only a modest-to-moderate share of the variance in valence (pseudo-R² values of .12 and .26). Consequently, these features should be treated as supplementary signals for triage and monitoring rather than a substitute for direct rating data.

The study concludes that while sentiment intensity, readability, and lexical diversity offer robust, cross-category signals, affective punctuation is highly context-dependent. Future operational tools must move away from universal rules toward category-aware calibration.