Semantic Gradients Interactions in SSD: A Case Study in Racial Identity and Hate Speech

Technical Summary: Semantic Gradients Interactions in SSD

Problem Statement
Supervised Semantic Differential (SSD) is a mixed quantitative-interpretive method that models how the meaning of texts or concepts varies with external outcomes by tracing a semantic gradient in an embedding space. However, the canonical SSD model estimates a single semantic gradient for the entire sample. This approach assumes a homogeneous meaning–outcome relationship, which can obscure critical moderation effects where semantic cues predict outcomes differently across groups, identities, or conditions. In social science and psychology, the same textual content may carry different evaluative implications depending on the judge's identity, yet standard SSD averages over these heterogeneous relationships, failing to capture how semantic structures diverge across theoretically meaningful subgroups.

Methodology: Interaction SSD
The paper introduces Interaction SSD, an extension of the canonical model designed to estimate how semantic gradients vary as a function of a moderator (e.g., group membership, traits, or experimental conditions).

1. Embedding and Dimensionality Reduction:

   • Documents are mapped to dense vectors using a fixed embedding model (e.g., GloVe) with SIF-weighted averaging and removal of the top principal component to reduce anisotropy.
   • Principal Component Analysis (PCA) is applied to reduce dimensionality. The number of retained components ($K$) is selected via a "PCA sweep," optimizing for retained variance, local semantic neighborhood coherence, and stability across nearby dimensionalities.

2. Regression Model:
   The method augments the standard SSD regression with a moderator main effect and a semantic interaction block. Let $y_i$ be the standardized outcome, $\tilde{x}_i$ the $K$-dimensional PCA-reduced document vector, and $m_i$ the standardized moderator. The model is specified as:
   $$y_i = \alpha + \beta^\top \tilde{x}_i + \gamma m_i + \delta^\top (\tilde{x}_i m_i) + \epsilon_i$$

   • $\beta$: Captures the baseline semantic association with the outcome (the main semantic gradient).
   • $\gamma$: Captures the non-semantic main effect of the moderator.
   • $\delta$: Tests whether the semantic association itself changes as a function of the moderator (the interaction gradient).

3. Statistical Testing and Interpretation:

   • Significance: The semantic block ($\beta$) and interaction block ($\delta$) are evaluated using Wald $F$ tests over their respective $K$ coefficients. The moderator main effect ($\gamma$) is evaluated as a single coefficient.
   • Back-Projection: Coefficient vectors are back-projected from the PCA space into the original embedding space to yield interpretable gradients:
     • Main Gradient: $\hat{g}_{main} = P\beta$
     • Interaction Gradient: $\hat{g}_{int} = P\delta$
     • Conditional Gradients: $\hat{g}(m^*) = \hat{g}_{main} + m^*\hat{g}_{int}$
   • Qualitative Analysis: Similar to standard SSD, the positive and negative poles of these gradients are interpreted by retrieving nearest neighbors, clustering them, and extracting representative text snippets.

Case Study and Results
The method was applied to the UC Berkeley Measuring Hate Speech corpus ($n=44,813$), specifically analyzing comments targeting people of color (POC). The moderator was annotator racial identity (binary: White vs. POC).

• Statistical Findings:

  • The model explained substantial variance in hate-speech scores ($R^2_{adj} = .662$).
  • The semantic block was highly significant, confirming a shared semantic structure.
  • The annotator race main effect was significant ($\hat{\gamma} = .028$), with White annotators assigning slightly higher hate-speech scores on average.
  • Crucially, the interaction block was significant ($F(62, 44687) = 13.76, p < 10^{-16}$), indicating that the semantic axis predictive of hate-speech judgments differs by annotator race. The partial $R^2$ for the interaction was .019, indicating a statistically robust but small effect size.

• Semantic Interpretation:

  • Main Gradient ($\hat{g}_{main}$): Revealed a shared structure contrasting overtly hostile discourse (slur-based contempt, eliminationist categorization, vermin metaphors) with community-affirming counter-speech (diasporic civic discourse, reflexive engagement with race, celebration of POC achievement).
  • Interaction Gradient ($\hat{g}_{int}$): Identified where the two groups diverged from the shared structure.
    • Positive Pole (White sensitivity): Organized around explicit, surface-marked hostility, including attack vocabulary, biomedical/contagion framings, and meta-discursive racism terms.
    • Negative Pole (POC sensitivity): Organized around context-dependent material, including affective racial commentary and Spanish-language intra-community conflict.
  • Conditional Gradients: Showed that while both groups shared high-score poles (hostility), the local organization of the negative poles (counter-speech) differed, with POC annotators exhibiting a more structurally elaborated counter-speech schema.

• Robustness: A reanalysis using the Common Crawl 840B GloVe model replicated the statistical moderation finding ($p < 10^{-16}$) with a similar partial $R^2$ (.017), though the specific semantic neighborhoods of the interaction gradient varied based on the embedding geometry.

Key Contributions and Significance
The paper's primary contribution is methodological: it extends SSD from estimating a single interpretable semantic gradient to modeling semantic moderation effects.

• Testability and Interpretability: Interaction SSD makes moderated meaning–outcome relationships statistically testable (via Wald tests on interaction blocks) and semantically interpretable (via back-projected gradients and nearest-neighbor analysis).
• Nuanced Understanding of Bias: The case study demonstrates that while a dominant shared semantic structure exists for hate speech judgments, group-linked differences emerge as reliable second-order deviations. This allows researchers to move beyond asking if groups differ to understanding how the semantic cues predicting judgments differ.
• Exploratory Utility: The authors position the method as a tool for theory-building and hypothesis generation. They note that because the interaction gradient is estimated from differences in conditional relationships, its local nearest-neighbor structure is less stable than the main gradient. Therefore, it is best used to identify where moderated semantic associations may occur rather than as a definitive classification system.

Limitations
The authors explicitly note several constraints:

• The analysis treats annotation records as independent observations, ignoring crossed random effects of annotators and comments.
• The binary White/POC moderator is a coarse proxy for social positioning and does not capture within-group heterogeneity.
• Results are specific to the dataset (English, US-based, MTurk annotators) and may not generalize to other languages or populations.
• The semantic interpretation of the interaction gradient is contingent on the choice of embedding model, meaning gradient neighborhoods should be viewed as exploratory maps of a specific model's geometry rather than stable empirical facts.