SOFIA: Quantifying Sampling-Induced Explanation Instability in Graph Neural Networks
The paper introduces SOFIA, a framework demonstrating that training-time neighbor sampling in Graph Neural Networks causes significant, structure-dependent instability in post-hoc explanations, revealing that these outputs often reflect arbitrary hyperparameters rather than genuine structural node properties.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the modern digital world, vast networks of information connect everything from social media friends to chemical molecules. To make sense of these tangled webs, scientists use a type of artificial intelligence called a Graph Neural Network. These systems learn by looking at a specific item and the items directly connected to it, gathering clues from its immediate surroundings to make a prediction. However, as these networks grow to include millions or even billions of connections, it becomes impossible for a computer to look at every single connection at once. To solve this, engineers use a shortcut called neighbor sampling. Instead of seeing the entire neighborhood, the computer is trained to look at only a small, random handful of connections at a time. This makes the training process fast and manageable, but it raises a quiet, unsettling question: if the computer never saw the full picture while it was learning, can we trust the reasons it gives us later for why it made a decision?
For years, the field has operated on the assumption that these explanations are stable, revealing the true, unchanging importance of a node's neighbors. A new study challenges this belief, suggesting that the reasons an artificial intelligence gives for its choices might depend less on the structure of the data and more on the random luck of which neighbors happened to be visible during training. The researchers behind this work, led by Prabu N P and colleagues, set out to measure exactly how much this training shortcut distorts the explanations we rely on. They built a framework to compare two versions of the same system: one that learned by seeing every single connection, and another that learned using the standard shortcut of seeing only a random sample. By pitting these two against each other, they could see if the explanations changed simply because the training method changed.
The team tested their approach on three very different real-world networks to see how the results varied. One was a citation network where papers mostly link to other papers in the same field, a setup where connections are few and similar. Another was a massive social media graph where users have hundreds or thousands of connections, and a third was a network where connections often link different types of people, creating a more chaotic mix. In the massive social network, the shortcut was extremely aggressive, excluding nearly ninety percent of the true neighbors from the training view. In the smaller, more uniform network, the shortcut removed very few connections. The researchers then asked the trained systems to explain their decisions and compared the lists of important neighbors they produced.
The results revealed a stark divide. On the massive social network, where the training process had to ignore most of the available information, the explanations produced by the sampled system were almost completely different from those of the full-neighborhood system. The agreement between the two dropped to a level far lower than what would be expected from normal computer randomness. In fact, the study found that for this network, the specific neighbors that were left out of the training view were the strongest predictor of whether an explanation would be unstable. When the system was trained without seeing its most important neighbors, it confidently pointed to different neighbors instead, creating an explanation that looked complete but was built on a blind spot. This suggests that in large-scale systems, the "reason" for a decision is often an artifact of the training shortcut rather than a genuine reflection of the data's structure.
The story was different on the other networks. On the smaller citation network, where the shortcut removed very few neighbors, the explanations remained stable and consistent, showing that the instability is not a universal flaw but depends heavily on how much information is excluded. On the third network, which had a more complex and mixed structure, the instability appeared even when fewer neighbors were excluded. This happened because in such mixed networks, the few remaining neighbors carry a heavier load of information, so losing even a small number of them causes a larger shift in the explanation. The researchers confirmed that the systems were equally accurate in their predictions regardless of how they were trained, proving that the instability was not a sign of a broken model, but a specific side effect of the training method itself.
This work introduces a new way to measure this phenomenon, a diagnostic tool that can tell a user if an explanation for a specific decision has been contaminated by the training shortcut. It shows that the trustworthiness of an artificial intelligence's explanation is not just about the model's accuracy, but about the invisible choices made during its education. If a system is trained to ignore most of its surroundings, its explanations will inevitably reflect that limited view, potentially misleading anyone who relies on them to understand how the system works. The study concludes that as these networks grow larger and require more aggressive shortcuts, we must be far more careful about assuming that the reasons given by an algorithm are the whole truth, rather than a story shaped by the random neighbors it happened to see.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.