ICE: Intervention-Consistent Explanation Evaluation with Statistical Grounding for LLMs

1. Problem Statement

The paper addresses the critical lack of rigorous tools to evaluate the faithfulness of explanations generated by Large Language Models (LLMs). Faithfulness refers to whether an explanation (e.g., highlighted tokens) genuinely reflects the model's internal reasoning process.

Current benchmarks, such as ERASER, suffer from three major limitations:

1. Lack of Statistical Testing: They report raw scores (e.g., sufficiency/comprehensiveness) without statistical significance testing, making it impossible to distinguish genuine faithfulness from chance-level performance.
2. Single Intervention Bias: They rely on a single intervention method (usually deletion of tokens). This creates Out-of-Distribution (OOD) inputs that degrade model performance regardless of the explanation's quality, potentially inflating or deflating faithfulness estimates.
3. Inability to Detect Anti-Faithfulness: Without random baselines, these frameworks cannot detect when an explanation is "anti-faithful" (performing worse than random selection), a phenomenon the authors argue is common.

2. Methodology: The ICE Framework

The authors introduce ICE (Intervention-Consistent Explanation), a framework designed to provide statistically grounded, operator-aware faithfulness evaluation.

Core Components

• Normalized Score Retention (NSR): The primary metric measuring how much of the original prediction score is preserved when only the rationale tokens (identified by the explanation method) remain.
  $$NSR(r) = \frac{s(x_r^o) - s(\emptyset)}{s(x) - s(\emptyset)}$$
  Where $s(x)$ is the original score, $s(x_r^o)$ is the score with only rationale tokens under operator $o$, and $s(\emptyset)$ is the baseline empty input score.
• Multi-Operator Intervention: ICE evaluates explanations using two distinct intervention operators to mitigate OOD artifacts:
  1. Deletion: Removes non-rationale tokens (fast but creates unnatural, truncated text).
  2. Retrieval Infill: Replaces non-rationale tokens with contiguous spans sampled from other corpus examples. This preserves surface form and natural language distribution while destroying task-relevant content.
• Randomization Testing: Instead of comparing raw scores, ICE compares the rationale's performance against matched random baselines.
  • For each input, $M$ random token sets of the same size as the rationale are generated.
  • A Win Rate is calculated: the percentage of times the rationale's NSR exceeds the random baseline's NSR.
  • Statistical Rigor: The framework uses one-sided permutation tests to calculate p-values, bootstrap confidence intervals (95%), and Cohen's $d$ effect sizes.
• Anti-Faithfulness Detection: If the Win Rate is significantly below 50%, the explanation is classified as "anti-faithful," meaning the model relies on the highlighted tokens less than random noise.

3. Key Contributions

1. Operator-Dependent Faithfulness: The paper demonstrates that faithfulness is not a single scalar value but an operator-dependent quantity. Different intervention operators (Deletion vs. Retrieval Infill) yield significantly different conclusions, with gaps reaching up to 44 percentage points.
2. Statistical Grounding: By introducing randomized baselines and win rates, ICE distinguishes genuine faithfulness from chance and reveals anti-faithfulness in nearly one-third of evaluated configurations, a phenomenon invisible to previous benchmarks.
3. Cross-Lingual and Cross-Architecture Generalization: The framework was applied to 7 LLMs (ranging from 1.5B to 8B parameters) across 4 English tasks and 6 non-English languages (French, German, Hindi, Chinese, Turkish, Arabic). It reveals complex model-language interactions that standard English-only benchmarks miss.
4. Orthogonality of Faithfulness and Plausibility: The study provides quantitative evidence that faithfulness (computational) and plausibility (human alignment) are orthogonal. The correlation between human rationale alignment (IoU) and ICE win rates was near zero ($|r| < 0.04$).

4. Key Results

• Text Length Sensitivity: The choice of operator significantly impacts results based on text length.
  • Short Text (e.g., SST-2): Deletion often yields higher faithfulness estimates than Retrieval Infill (gaps of 8–44 pp).
  • Long Text (e.g., IMDB): The pattern reverses; Retrieval Infill often outperforms Deletion because deleting most of a long review creates severely degraded inputs, whereas infill preserves natural context.
• Anti-Faithfulness: In ~33% of configurations (particularly gradient-based methods on short text), explanations were found to be anti-faithful. For example, gradient methods often highlighted function words (e.g., "a") instead of sentiment-bearing adjectives, performing worse than random selection.
• Multilingual Variance: Faithfulness varies drastically by language and model.
  • GPT-2 showed anti-faithfulness on French and Turkish but surprisingly strong performance on Hindi.
  • Tokenization alone does not predict faithfulness; model-language interactions and morphological structures play a larger role.
• Model Architecture: Instruction-tuned models generally showed higher faithfulness than base models. However, some efficient architectures (e.g., LFM2) showed task-specific faithfulness (high on NLI, random on sentiment).

5. Significance and Implications

• Redefining Evaluation Standards: The paper argues that faithfulness should be interpreted comparatively across intervention regimes rather than as a single benchmark score. Agreement between operators (e.g., both Deletion and Infill showing high win rates) is a strong signal of genuine faithfulness.
• Practical Guidelines: The authors provide specific recommendations for practitioners (e.g., using Attention for short-text sentiment and NLI, but Gradient for topic classification; specific model/language pairings like Qwen for German).
• Open Source: The authors released ICE (the framework) and ICEBench (the benchmark with pinned dataset versions) to ensure reproducibility.
• Ethical Warning: The paper cautions that "faithful to the model" does not equate to "correct reasoning." A model can faithfully rely on spurious correlations, necessitating domain validation in high-stakes applications.

In summary, ICE moves the field of explainable AI from static, single-metric reporting to a dynamic, statistically rigorous, and operator-aware evaluation paradigm, revealing that many current explanations are either unreliable or actively misleading.