💬 NLP

Counterfactual Probing for the Influence of Affect and Specificity on Intergroup Bias

This paper investigates how pragmatic features like affect and specificity influence intergroup bias in NLP by analyzing their correlation with intergroup relationship labels and using counterfactual probing to reveal that neural models rely on affect but show inconclusive usage of specificity for classification.

Original authors: Venkata S Govindarajan, Kyle Mahowald, David I. Beaver, Junyi Jessy Li

Published 2026-04-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Venkata S Govindarajan, Kyle Mahowald, David I. Beaver, Junyi Jessy Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to figure out why people say the things they do when they talk about their friends versus their rivals.

Most previous research on "bias" in computers (AI) was like looking for a smoking gun: they only cared when people used mean words or insults. But this paper suggests that bias is actually everywhere, even in nice sentences. It's not just what you say, but how you say it depending on who you are talking to.

The authors wanted to test two specific "flavors" of language to see if they change based on whether the speaker is talking to a friend (In-Group) or a stranger/rival (Out-Group):

Affect (The Mood): Is the speaker happy, angry, or neutral toward the person they are talking about?
Specificity (The Detail): Is the speaker being vague and general ("You are a good person"), or are they being super specific with details ("You voted for bill X on Tuesday")?

The Big Question

The researchers asked: Do computers actually "think" using these two flavors when they decide if a tweet is friendly or hostile?

They used a dataset of tweets from US Congress members. In this world, if a Senator tweets at another Senator from their own party, it's an "In-Group" interaction. If they tweet at the opposing party, it's "Out-Group."

The Experiment: The "Magic Eraser" and "The Mood Ring"

To find out if the computer model was really using these clues, the researchers used a clever trick called Counterfactual Probing. Think of it like a science experiment with a magic wand:

The Mood Ring (Affect): They took a tweet and used a mathematical "magic wand" to force the computer's internal brain to feel more positive or more negative, even if the words in the tweet didn't change.
The Detail Dial (Specificity): They did the same thing for detail. They forced the computer to think the tweet was either very vague or very detailed.

Then, they asked the computer: "Now that we changed the mood or the detail, do you still think this tweet is from a friend or a rival?"

What They Found

1. The Mood Ring Works (Affect)
The computer was very sensitive to the "Mood."

The Result: When they forced the computer to feel positive about the target, the computer almost always decided, "Ah, this must be a friend!"
The Analogy: It's like a dog. If you walk in with a happy, wagging tail (positive affect), the dog assumes you are a friend, regardless of what you are wearing. The computer relies heavily on the "vibe" to make its decision.

2. The Detail Dial is Broken (Specificity)
The computer was confused by the "Detail."

The Result: When they tried to force the computer to think a tweet was "very specific," nothing happened. The computer didn't care. However, when they forced it to think a tweet was "very vague," the computer suddenly thought, "Oh, this must be a friend!"
The Analogy: Imagine trying to teach a child to recognize a car by its color. If you paint a red car blue, the child still knows it's a car. But if you try to teach them by the shape of the wheels, and the wheels are blurry, they get confused.
Why? The authors suspect the computer model got "damaged" during its training. It learned to recognize friends based on the mood, but it lost the ability to properly understand details. It's like a chef who lost their sense of taste but still has a great sense of smell; they can tell if food smells good, but they can't tell if the ingredients are fresh.

The Twist: The "General" Bias

The researchers had a theory (based on old psychology) that people are usually vague when praising friends and specific when criticizing enemies.

Theory: "You are a great guy!" (Vague praise for a friend) vs. "You voted wrong on Bill 402!" (Specific attack on an enemy).
Reality: The computer didn't follow this pattern. It only reacted to the mood. It didn't seem to care about the level of detail at all.

The Takeaway

This paper is a reality check for AI researchers.

Bias is subtle: It's not just about hate speech; it's about how we describe people differently based on our relationship with them.
AI is one-track minded: The computer model they tested is really good at detecting "good vibes" vs. "bad vibes," but it's terrible at understanding the nuance of "details."
We need better tools: The methods used to "poke" the AI (the magic wands) are powerful, but they can sometimes break the AI's brain, making it hard to see the full picture.

In short: The computer knows how to tell if you're being nice or mean, but it doesn't quite understand the difference between a vague compliment and a specific one. And that's a problem if we want AI to truly understand human relationships.

1. Problem Statement

Existing research on bias in Natural Language Processing (NLP) predominantly focuses on identifying negative or pejorative language. However, social psychology suggests that bias is not merely the presence of hate speech but a shift in language behavior reflecting social dynamics between groups.

The authors aim to investigate Intergroup Bias (IGB) through a new framing proposed by Govindarajan et al. (2023), which models relationships between "in-group" and "out-group" members. Specifically, the paper seeks to determine if two pragmatic features—Specificity (level of detail) and Affect (speaker's attitude toward the target)—systematically vary across different intergroup contexts.

The study addresses two main questions:

Do specificity and affect correlate with intergroup relationship labels in natural language data?
Do neural models trained to predict intergroup relationships causally rely on these features, or are they using spurious correlations?

2. Methodology

Data Source

The study utilizes a dataset of tweets from US Congress members (Govindarajan et al., 2023).

Labels: Tweets are labeled as In-group (target is a member of the same party) or Out-group (target is a member of a different party).
Annotations:
- Affect: Annotators on Amazon Mechanical Turk rated the writer's feeling toward the target (warm/cold) and the target's actions (approval/disapproval). These were aggregated into a binary label: Positive (+1) or Negative (-1).
- Specificity: Calculated using a pre-trained specificity predictor (Gao et al., 2019) that scores tweets on a scale of 1–5 based on lexical, syntactic, and semantic features.

Experimental Framework: Counterfactual Probing

To move beyond correlation and establish causality, the authors employ AlterRep (Ravfogel et al., 2021), a probing technique based on Iterative Nullspace Projection (INLP).

Base Model: BERTweet (pre-trained on 850M tweets), fine-tuned to predict In-group vs. Out-group labels.
Intervention Process:
1. Feature Extraction: An iterative linear classifier is trained on the model's internal representations (Layer 11) to identify the direction of a specific feature (e.g., Affect or Specificity).
2. Nullspace Projection: The classifier's parameters are used to project embeddings into a nullspace, effectively removing the feature's information.
3. Counterfactual Manipulation: The embeddings are then pushed in the orthogonal direction to artificially increase or decrease the feature (e.g., making a tweet "more positive" or "less specific").
4. Evaluation: The modified embeddings are fed back into the IGR classifier to observe changes in prediction probability.

Hypotheses

Based on the Linguistic Intergroup Bias (LIB) theory and the authors' generalization:

H1: Positive affect in-group and negative affect out-group utterances should be encoded with lower specificity (more abstract).
H2: Interventions increasing positive affect should increase the likelihood of a tweet being predicted as "In-group," particularly for low-specificity tweets.

3. Key Contributions

New Framing of Bias: Shifts the focus from explicit hate speech to subtle, context-dependent intergroup dynamics in natural language.
Causal Probing of Pragmatics: This is the first work to use counterfactual probing (AlterRep) to test if neural models causally utilize high-level pragmatic features like specificity and affect for social relationship classification.
Dataset Expansion: Extends the Govindarajan et al. (2023) dataset with new coarse-grained affect annotations and specificity scores.

4. Results & Analysis

Correlational Analysis

Affect: Found a modest positive correlation ( $r=0.2$ ) between positive affect and In-group labels.
Specificity: Found no significant correlation between specificity and IGR labels in the raw data ( $r=-0.07$ ). However, when filtering for extreme specificity scores (excluding the middle range), a small negative correlation emerged.

Counterfactual Probing Results

Affect Intervention:
- Pushing embeddings toward positive affect caused the model to classify nearly 100% of test tweets as "In-group" after 32 INLP iterations.
- Pushing toward negative affect had the inverse effect, though the drop was less steep (likely because 75% of the original data was already positive).
- Conclusion: The model causally relies on affect for its decision-making.
Specificity Intervention:
- Pushing embeddings toward high specificity had no effect on predictions (indistinguishable from random control).
- Pushing toward low specificity caused a significant shift, eventually classifying all data as "In-group" after 48 iterations.
- Conclusion: The model's usage of specificity is inconclusive and asymmetric. It reacts to low specificity but not high specificity.

Qualitative Error Analysis

The authors observed that fine-tuning BERTweet for IGR prediction degraded the underlying Language Model (LM) capabilities.

LM Degradation: The fine-tuned model began predicting nonsensical tokens (e.g., "opport__", "mug__") even before intervention.
Interpretation: The inconclusive results for specificity and the lack of interaction between affect and specificity may be artifacts of this LM degradation. The model may be "breaking" the semantic coherence required to process specificity changes effectively.

5. Significance and Limitations

Significance:

The study demonstrates that while neural models can learn to use affect as a causal signal for intergroup bias, they do not necessarily utilize specificity in the way human social psychology theories (like LIB) predict.
It highlights the utility of counterfactual probing to distinguish between features a model correlates with and features it causally uses.

Limitations:

LM Degradation: The fine-tuning process may have damaged the model's ability to handle semantic nuances, leading to the inconclusive specificity results.
Generalizability: The results are specific to US Congressional tweets; generalizability to other domains or languages is unknown.
Parameter Sensitivity: The AlterRep/INLP method is sensitive to hyperparameters (e.g., number of iterations, $\alpha$ ), and different settings might yield different outcomes.
Data Source: While tweets are treated as natural speech, they may be drafted by social media teams, potentially altering the "speaker-target" dynamic.

Future Work:
The authors suggest exploring more complex, non-linear probing methods (beyond linear INLP) and investigating whether the observed asymmetry in specificity is due to the specific dataset distribution or a fundamental limitation of the current model architecture.