🤖 machine learning

Why Self-Inconsistency Arises in GNN Explanations and How to Exploit It

This paper investigates the causes of self-inconsistency in Graph Neural Network explanations, proposing a latent signal assignment hypothesis to explain edge sensitivity and introducing a training-free Self-Denoising strategy that effectively calibrates explanations with minimal computational overhead.

Original authors: Wenxin Tai, Yaqian Liu, Ting Zhong, Fan Zhou

Published 2026-05-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Wenxin Tai, Yaqian Liu, Ting Zhong, Fan Zhou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Forgetful" Detective

Imagine you have a very smart detective (a Self-Interpretable Graph Neural Network, or SI-GNN) who solves crimes by looking at a map of clues (a graph). When the detective solves a case, they point to specific clues on the map and say, "These are the reasons I think the suspect is guilty."

Usually, we trust this detective. But the authors of this paper discovered a weird glitch: The detective is inconsistent with themselves.

If you take the detective's list of clues, erase everything except those specific clues, and ask the detective to solve the case again using only that smaller list, they often change their mind.

First time: "I need Clue A and Clue B to solve this."
Second time (looking only at A and B): "Actually, Clue A isn't important anymore. I need Clue C instead."

The paper calls this "Self-Inconsistency." It's like a witness who changes their story every time you ask them to repeat it, making us wonder if their original story was even true.

Why Does This Happen? The "Noise" in the Room

The authors investigated why this happens. They found two main reasons:

The Context Shift: When you remove all the other clues from the map, the "atmosphere" of the remaining clues changes. In the detective's brain (the neural network), clues are understood based on their neighbors. If you take away the neighbors, the meaning of the remaining clues shifts slightly. This shift causes the detective to re-evaluate their importance scores.
The "Signal" vs. "Noise" Theory: The authors propose that some clues have a strong, internal "signal" (they are genuinely important), while others are just "noise" (they look important only because of the surrounding context).
- Stable Clues: The truly important clues have a strong internal signal. They stay important even when the room changes.
- Unstable Clues: The unimportant clues are like chameleons; they only look important because of the company they keep. When the company changes (the graph is cut down), they lose their importance.

The paper suggests that the "budget" for explanations (how many clues the detective is allowed to pick) forces the detective to sometimes grab these unstable, noisy clues just to fill the quota.

The Solution: "Self-Denoising" (SD)

The authors created a simple, free tool called Self-Denoising (SD) to fix this. Think of it as a "second opinion" filter.

Here is how it works:

Ask the detective once: Get the first list of clues.
Ask the detective again: Feed that list back in and get a second list.
Compare the lists:
- If a clue appears as important in both lists, it's likely a real signal. Keep it.
- If a clue was important in the first list but disappeared or changed in the second list, it's likely noise caused by the context shift.
The Fix: The SD tool lowers the importance score of those unstable clues. It essentially says, "If you can't agree with yourself, you probably aren't that important."

The Results: Cleaner Explanations

The paper tested this method on several different types of AI detectives and various datasets (like synthetic graphs and real-world chemical molecules).

Better Explanations: After applying Self-Denoising, the explanations became much more accurate. They matched the "Ground Truth" (the actual correct reasons) much better.
More Concise: The explanations became shorter and less cluttered. The tool successfully removed the "noise" edges, leaving only the clear, stable ones.
Cheap and Fast: The best part is that this doesn't require retraining the AI. It just needs one extra quick look (forward pass) at the data. It adds only about 4–6% extra computing time, which is very small.
Works with Others: The authors also found that this method works well alongside other existing methods (like "Explanation Ensemble"), suggesting it catches a different type of error that other tools miss.

Summary

In short, the paper says: "AI explainers often lie to themselves because they get confused when the context changes."

The authors figured out that this confusion happens because some clues are only important due to their surroundings, not because they are truly significant. By asking the AI to explain itself twice and filtering out the parts where it changes its mind, they created a simple "denoising" tool that makes AI explanations more trustworthy, accurate, and concise without needing to rebuild the AI from scratch.

Technical Summary: Why Self-Inconsistency Arises in GNN Explanations and How to Exploit It

1. Problem Statement

Self-Interpretable Graph Neural Networks (SI-GNNs) integrate explanation generation directly into the prediction pipeline, theoretically offering more trustworthy explanations than post-hoc methods. However, recent observations indicate that SI-GNNs suffer from self-inconsistency: when a model is reapplied to its own generated explanatory graph subset, it often produces a different explanation, either removing previously selected edges or adding new ones. This behavior undermines the faithfulness and practical utility of SI-GNNs. While the existence of this phenomenon is documented, the underlying mechanism causing it remains poorly understood, and effective strategies to mitigate it without retraining are lacking.

2. Methodology and Theoretical Analysis

Root Cause: Context Perturbation
The authors identify re-explanation-induced context perturbation as the direct cause of score variation. In GNNs, node representations are updated via message passing. When an SI-GNN generates an explanation (a masked subgraph) and feeds it back into itself, the edge-wise masking alters the local message-passing context. Since edge importance scores are derived from these node representations, the perturbed context naturally leads to different scores in the second pass.

Latent Signal Assignment Hypothesis
To explain why only specific edges (typically unimportant ones) exhibit instability while ground-truth important edges remain stable, the paper proposes a Latent Signal Assignment Hypothesis. The authors partition edges into three disjoint sets:

$E_p$ (Positive-signal edges): Edges assigned high scores based on intrinsic, context-invariant features. These remain stable across passes.
$E_n$ (Negative-signal edges): Edges assigned low scores based on intrinsic features. These also remain stable.
$E_c$ (Context-driven edges): Edges whose scores are determined primarily by the local message-passing context. These are highly sensitive to the perturbations caused by re-explanation.

The authors argue that conciseness regularization (which limits the total "budget" of importance mass an explanation can have) shapes this allocation. Under tight budget constraints, the model is forced to reallocate importance, potentially pushing unimportant edges from a signal-driven state to a context-driven state, thereby increasing their sensitivity to perturbations.

Proposed Solution: Self-Denoising (SD)
Leveraging the observation that self-inconsistency acts as a signal for noise, the authors propose Self-Denoising (SD), a model-agnostic, training-free post-processing strategy.

Mechanism: SD performs a second forward pass on the explanatory graph to measure the instability ( $\Delta s_{ij}$ ) of each edge's importance score.
Calibration: It penalizes edges with high instability by scaling down their original scores: $\tilde{m}_{ij} = \max(0, (1 - \eta \Delta s_{ij})m^{(1)}_{ij})$ , where $\eta$ is a denoising strength parameter.
Hyperparameter Selection: To handle the distribution shift introduced by SD, the authors suggest a lightweight adaptation strategy: freezing the explainer and encoder, and fine-tuning only the classifier on the calibrated explanations. The denoising strength $\eta$ is then selected based on the validation accuracy of this adapted classifier.

3. Key Contributions

Mechanistic Insight: The paper provides the first theoretical explanation for self-inconsistency, attributing it to local context perturbations and formalizing the role of latent signal assignment under conciseness constraints.
Efficient Calibration: It introduces Self-Denoising (SD), a post-hoc strategy that improves explanation quality with minimal computational overhead (only one additional forward pass, adding ~4–6% cost).
Complementarity: The work demonstrates that SD captures a distinct source of noise (single-model self-inconsistency) that is complementary to existing cross-model inconsistency methods like Explanation Ensemble (EE).

4. Experimental Results

Experiments were conducted across four representative SI-GNN frameworks (GAT, CAL, SMGNN, GSAT), multiple backbone architectures (GIN, GraphSAGE, GatedGCN), and four benchmark datasets (BA-2MOTIFS, 3MR, BENZENE, MUTAGENICITY).

Explanation Quality: SD consistently improves the AUC (plausibility against ground truth) across all 16 tested configurations, with gains up to 3.9%. It also significantly reduces SPA (Soft Sparsity), indicating more concise and less redundant explanations.
Faithfulness: Results on FID (Faithfulness) are mixed, which the authors attribute to the distribution shift introduced by SD and the specific mechanics of certain models (e.g., CAL). However, after lightweight classifier adaptation, FID stability improves.
Predictive Performance: Applying SD alone results in minor accuracy changes. When combined with the lightweight classifier adaptation (SD*), downstream classification accuracy improves consistently, with gains up to 8% in some cases.
Efficiency vs. EE: Unlike Explanation Ensemble (EE), which requires training multiple models and incurs ~400% overhead, SD requires only a single model and adds ~4–6% overhead. Combining SD with EE yields further improvements, suggesting the two methods address different noise sources.

5. Significance and Claims

The paper claims that self-inconsistency is not merely a failure mode but an intrinsic signal that can be exploited to calibrate explanations. By understanding the mechanism of context perturbation and latent signal assignment, the authors demonstrate that a simple, training-free post-processing step can significantly enhance the reliability and conciseness of SI-GNN explanations.

The authors maintain a modest stance, noting that:

Their latent signal hypothesis is a mechanistic perspective inferred from behavior rather than a rigorously identifiable latent-variable model.
SD does not guarantee fully trustworthy explanations, as it cannot capture all noise sources (e.g., those not related to self-inconsistency).
The method is designed for post-hoc calibration of existing pretrained models, distinguishing it from approaches that require retraining with self-consistency objectives.

Ultimately, the work aims to improve the trustworthiness of AI systems in high-stakes domains (e.g., drug discovery, scientific discovery) by suppressing context-driven noise in graph explanations without requiring architectural changes or retraining.