A Systematic Comparison between Extractive Self-Explanations and Human Rationales in Text Classification

Technical Summary: A Systematic Comparison between Extractive Self-Explanations and Human Rationales in Text Classification

Problem Statement
Instruction-tuned Large Language Models (LLMs) can generate self-explanations (rationales) for their outputs without requiring complex external interpretability techniques. However, the reliability, plausibility, and faithfulness of these self-generated explanations remain under-explored. Existing research often focuses on short texts or keyword-based tasks, lacking systematic comparisons across diverse, realistic domains with varying text lengths and complexities. Furthermore, there is a need to understand how these self-explanations compare to human-annotated rationales and post-hoc attribution methods in terms of their ability to reflect the model's actual decision-making process.

Methodology
The authors conduct a controlled study evaluating four open-weight instruction-tuned LLMs (Gemma3, Llama3, Mistral, and Qwen3) across three distinct text classification tasks:

1. Sentiment Classification (SST/mSST): Binary classification of movie reviews in English, Danish, and Italian.
2. Forced Labour Detection (RaFoLa): Multi-label classification of news articles identifying specific risk indicators of forced labor.
3. Claim Verification (Climate-Fever): Verification of climate change claims against evidence snippets from Wikipedia.

For the first two tasks, the study utilizes existing human rationale annotations. For Climate-Fever, the authors collected new human rationale annotations for a subset of 104 claims (520 evidences) using three personally instructed annotators.

The evaluation framework assesses two primary dimensions:

• Plausibility: Measured by the agreement (Cohen's Kappa) between model-generated rationales and human annotations.
• Faithfulness: Measured via intervention-based probing. The authors mask tokens identified by human rationales, model self-explanations, and post-hoc attribution methods (Layer-wise Relevance Propagation [LRP] and Gradient×Input) to observe the resulting drop in the model's prediction probability.

The study also includes a statistical analysis of token-level strategies, comparing lexical diversity, stop-word fractions, and entity distributions across human, model, and post-hoc rationales.

Key Contributions

1. New Dataset Annotations: The collection and release of human rationale annotations for a subset of the Climate-Fever dataset, addressing a gap in available ground truth for claim verification tasks.
2. Controlled Comparative Study: A systematic comparison of human annotations, LLM self-explanations, and post-hoc attribution methods across three domains with varying text lengths and task complexities.
3. Faithfulness Analysis: An evaluation demonstrating that self-explanations yield faithful subsets of token-level rationales, whereas post-hoc attribution methods often prioritize structural and formatting tokens over semantic evidence.
4. Strategy Characterization: A detailed analysis revealing that human rationales emphasize narrative content and lived experiences, model rationales provide factual and analytical details, and post-hoc rationales focus on isolated evidence tokens and structural artifacts.

Key Results

• Plausibility: Agreement between human and model rationales varies significantly based on text length and task complexity.
  • SST: Moderate agreement (Kappa 0.4–0.6) for English subsets; lower for Danish and Italian.
  • RaFoLa: High variance across articles; agreement ranges from low (0.12) to moderate (0.48), correlating with task difficulty and keyword distinctiveness.
  • Climate-Fever: Generally low agreement (0.12–0.24), attributed to the intrinsic complexity and ambiguity of the claim verification task rather than surface-level text properties.
• Faithfulness:
  • Model self-explanations (specifically when ranked by greedy maximization of probability difference) demonstrate high faithfulness, often causing a steeper drop in prediction probability upon masking than human rationales, particularly in the first 10% of tokens.
  • Post-hoc methods (LRP, Gradient×Input) show lower faithfulness in identifying the most relevant initial tokens compared to self-explanations. They frequently assign high importance to structural tokens (e.g., <s>, <|begin_of_text|>) required for sequence delimitation rather than semantic content.
• Token Strategies:
  • Human Rationales: Higher lexical diversity and stop-word usage; focus on narrative and human context (e.g., "victims," "poverty").
  • Model Rationales: Similar lexical diversity to humans but denser, more technical phrasing focusing on mechanisms and quantitative evidence (e.g., "trafficking," "ILO").
  • Post-hoc Rationales: Lower lexical diversity and stop-word usage; higher concentration of named entities and formatting tokens (e.g., "Reuters," "https://").

Significance and Claims
The paper posits that self-explanations offer a promising avenue for interpretability, as they can produce faithful token-level rationales that effectively disrupt model predictions when masked. However, the study highlights a fundamental divergence in explanation strategies: while self-explanations communicate coherent evidence through natural language, post-hoc attribution methods often highlight structural cues.

The authors conclude that their work constitutes an "initial step" toward characterizing the reliability of self-explanations in realistic text classification settings. They emphasize that while self-explanations are faithful, they may not always be plausible to humans, and vice versa. The study supports the need for rigorous evaluation of self-explanations and suggests that future research should explore how these findings transfer to abstractive free-text explanations and process-oriented analyses of complex evidence integration. The authors explicitly avoid claiming that self-explanations are universally superior or that they fully solve the interpretability problem, instead framing their results as a foundation for further investigation into model trustworthiness and transparency.