Application of integrated gradients explainability to sociopsychological semantic markers

Technical Summary: Application of Integrated Gradients Explainability to Sociopsychological Semantic Markers

Problem Statement
While deep learning models, particularly Large Language Models (LLMs) based on the Transformer architecture (e.g., BERT, RoBERTa), have achieved state-of-the-art performance in classifying textual data for sentiment and sociopsychological markers, they operate as "black boxes." Current approaches typically provide classification outputs at the sentence or document level, obscuring the specific words that drive these predictions. This lack of granularity limits the interpretability required for researchers to understand the linguistic mechanisms behind sociopsychological constructs. Furthermore, while attribution methods exist, their suitability for word-level analysis of nuanced markers like "agency" (goal orientation and achievement) has not been systematically evaluated, particularly regarding the trade-off between computational complexity and attribution reliability. Additionally, a common challenge in sociopsychological research is the scarcity of large, labeled datasets for specific constructs, necessitating methods that can extract meaningful semantic patterns from limited data.

Methodology
The paper proposes a framework for word-level explainability using Integrated Gradients (IG) applied to sociopsychological markers. The methodology is divided into three main phases:

1. Attribution Method Selection and Optimization:

   • The authors evaluated several state-of-the-art attribution methods, including standard IG, Sequential IG, Discretized IG, DeepLIFT, and GradientSHAP.
   • Performance was assessed using faithfulness metrics: Comprehensiveness (how much the model output drops when top-scoring words are removed) and Sufficiency (how well the model output is preserved when only top-scoring words are kept), alongside Approximation Error.
   • Experiments were conducted on four diverse datasets (#covid19, IMDB, slurs, snippets) using the pre-trained BERTAgent model (based on RoBERTa) as the target classifier for "agency."
   • Key IG parameters were optimized: the number of integration steps ($N$) and the baseline input ($x_0$). The study tested baselines including zero, mask, padding, and mean embeddings.

2. Validation and Rendering:

   • The raw IG output was processed to address tokenization issues (e.g., subword splitting) and semantic coherence (e.g., handling negations and grouping syntactic dependencies) using SpaCy.
   • The reliability of the optimized IG approach was validated by three expert psychologists who evaluated the output for "hits" (correct identification of agentic words) and "false alarms" (hallucinations) on a "golden standard" dataset.

3. Application to Small Datasets (Overfitting Strategy):

   • To address scenarios with limited labeled data, the authors proposed a novel training procedure where the model is deliberately encouraged to overfit until it achieves 100% training accuracy.
   • The rationale is that in a small dataset scenario, overfitting forces the model to capture the specific, distinct statistical signatures of each class. IG is then applied to this overfitted model to identify the salient words that differentiate the classes, effectively generating a semantic dictionary for the target marker.

Key Contributions

• Identification of IG as the Optimal Method: Through rigorous comparison, the paper identifies Integrated Gradients as the most effective approach, offering a superior balance of performance (faithfulness) and computational complexity compared to Sequential IG, DeepLIFT, and GradientSHAP.
• Parameter Optimization: The study establishes that a zero baseline and 300 integration steps ($N=300$) provide the most reliable attributions for RoBERTa-based models, minimizing approximation error while maintaining high comprehensiveness and sufficiency.
• Validation of Word-Level Agency Detection: The paper demonstrates that IG can reliably identify words contributing to "agency" scores, with validation showing high agreement between the algorithm's highlights and expert human judgments, particularly for texts with high agency levels and shorter lengths.
• Small-Data Dictionary Construction: The paper introduces a method to utilize overfitting in conjunction with IG to extract class-specific keywords from small labeled datasets. This allows for the construction or expansion of dictionaries for complex constructs (e.g., objectification, shame, fear) where large pre-trained models or dictionaries do not yet exist.

Results

• Method Comparison: IG with a zero baseline and $N=300$ consistently outperformed or matched other methods in comprehensiveness and sufficiency across all datasets. Sequential IG offered marginal improvements only for very short texts but incurred a 10x computational cost. DeepLIFT and GradientSHAP showed the worst performance.
• Validation: Expert validation confirmed that IG correctly identifies agentic content in the majority of cases. The algorithm's reliability correlates positively with the magnitude of the agency score and negatively with text length. False alarms (hallucinations) occurred in approximately 20% of samples, a rate the authors attribute to the inherent complexity of the task and natural inter-coder disagreement among experts.
• Application 1 (Text Highlights): When applied to a dataset of texts written to mobilize collective action, IG revealed that expert-highlighted sections contained a significantly higher concentration of agentic words (capturing >40% of the maximum possible agentic content) compared to random selections. This empirically links the concept of agency to effective mobilization language.
• Application 2 (Dictionaries from Labels): Applied to the "slurs" dataset, the overfitting+IG approach successfully identified semantically coherent keywords for various labels. For example, it distinguished gender markers (e.g., "man," "boy" vs. "woman," "sister"), relationship types (e.g., "boss" vs. "husband"), and emotional states (e.g., "mortify," "insecurity" for high shame; "blowjob," "skirt" for high objectification). The results aligned with sociopsychological theories regarding objectification and emotional responses to aggression.

Significance
The paper claims that its approach provides a robust, explainable AI (XAI) tool for sociopsychological research. By moving from sentence-level to word-level attribution, it offers researchers a "readable" mechanism to understand how specific linguistic features drive classification outcomes. The significance lies in two areas:

1. Theoretical Insight: It validates the link between agentic language and collective action intentions, providing empirical evidence for sociopsychological theories regarding mobilization.
2. Practical Utility for Data-Scarce Domains: The proposed method for small datasets offers a pathway to build or refine dictionaries for complex, theoretically immature constructs (such as objectification or polarization) without requiring massive labeled corpora. This enables the identification of salient semantic content that characterizes specific classes, facilitating the creation of more targeted and theoretically grounded NLP tools.