🤖 AI

The Confusion is Real: GRAPHIC -- A Network Science Approach to Confusion Matrices in Deep Learning

This paper introduces GRAPHIC, an architecture-agnostic method that applies network science to confusion matrices derived from intermediate neural network layers to systematically visualize and quantify class confusion dynamics, linear separability, and dataset issues throughout the training process.

Original authors: Johanna S. Fröhlich, Bastian Heinlein, Jan U. Claar, Hans Rosenberger, Vasileios Belagiannis, Ralf R. Müller

Published 2026-05-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Johanna S. Fröhlich, Bastian Heinlein, Jan U. Claar, Hans Rosenberger, Vasileios Belagiannis, Ralf R. Müller

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand how a student learns to identify different animals in a photo album. Usually, you just look at their final test score: "They got 90% right!" But that doesn't tell you why they got the 10% wrong, or if they are confusing a cat with a dog because they look similar, or because the student is just guessing.

This paper introduces a new tool called GRAPHIC (which stands for a long, fancy acronym about graphs and confusion). Think of GRAPHIC as a "Confusion Map" that lets you peek inside the student's brain while they are studying, not just at the end.

Here is how it works, using simple analogies:

1. The "Secret Test" (The Linear Classifier)

Deep learning models (the "students") have many hidden layers where they process information. We can't easily see what they are thinking in these middle layers.

The Trick: The researchers take the "notes" the model writes down in these middle layers and give them to a very simple, honest teacher (called a Linear Classifier).
The Test: This teacher asks, "If I only saw these notes, could I tell the difference between a 'flatfish' and a 'man'?"
The Result: This creates a Confusion Matrix. It's a grid that shows exactly how often the model mistakes one thing for another. For example, it might show that the model confuses "flatfish" with "man" 20% of the time.

2. Turning the Grid into a "Social Network"

Instead of just looking at a boring spreadsheet of numbers, GRAPHIC turns this grid into a social network map.

The Nodes (Dots): Every category (like "Apple," "Bear," or "Car") is a dot on the map.
The Edges (Lines): If the model often confuses two things, a line connects them.
- A thick line means they are very confused (e.g., the model thinks a "flatfish" is a "man" a lot).
- A thin line means they are rarely confused.
- Arrows show the direction of the mistake (e.g., "Man" is rarely called a "Flatfish," but "Flatfish" is often called a "Man").

3. What Did They Discover?

By watching this "social network" change as the model learns, the researchers found some surprising things:

The "Popular Kids" of the First Day: In the very first second of training, a few random classes (like "Computer Keyboard" or "Sea") became "hubs." The model guessed these names for almost everything. This wasn't because the model was smart; it was just because of the order the photos were shown in. It's like if a teacher showed you a picture of a keyboard first, you might guess "keyboard" for the next 10 pictures just because you're stuck on that idea.
Forming Cliques: As training went on, the dots started grouping into "cliques" based on real meaning. "Animals" started hanging out together, and "Trees" started hanging out together. The model was learning the concepts, not just memorizing pixels.
The "Flatfish" vs. "Man" Mystery: The map showed a strong line connecting "Flatfish" and "Man." Why? The researchers looked at the photos and realized the dataset was biased. Many photos of "flatfish" showed fishermen holding up their catch. The model learned: "If I see a human holding a fish, it's a flatfish." It wasn't a visual similarity; it was a contextual clue from a bad dataset.
The "Baby" vs. "Boy" vs. "Girl" Blur: The model got very confused between babies, boys, and girls. The researchers asked 31 humans to label these same photos, and the humans got confused too! The photos were just too blurry or the ages were too ambiguous. The model wasn't "stupid"; the labels in the dataset were just messy.
The "Leaf Color" Bias: The model confused "Maple trees" with "Oak trees." Why? The photos of Maples were mostly taken in autumn (red/yellow leaves), while Oaks were green. The model learned to identify trees by the season, not the tree type.

4. The "Transformer" Surprise

The researchers tested two types of models: a standard one (ResNet) and a newer, more complex one (a Vision Transformer).

The Standard Model: As it learned, it got better at separating things linearly (like sorting apples from oranges) all the way through.
The Transformer: It started out getting better at separating things, but then, in its early stages, it actually got worse at separating them before getting better again. It's like the Transformer had to "unlearn" some simple rules to learn more complex ones, whereas the standard model just kept stacking rules on top of each other.

Summary

GRAPHIC is like a microscope for the "mistakes" a neural network makes. Instead of just saying "the model is 90% accurate," it draws a map of who is confusing whom. This helps researchers see if the model is learning the right things, or if it's just picking up on bad habits (like confusing a fish with a man because of a fisherman in the background) or if the dataset itself is broken (like blurry photos of babies).

It turns the "black box" of AI into a visible social network of confusions, revealing that sometimes the confusion is real, and sometimes it's the data's fault.

Technical Summary: GRAPHIC – A Network Science Approach to Confusion Matrices in Deep Learning

Problem Definition

Explainable Artificial Intelligence (XAI) has become critical for addressing reliability concerns in deep learning, particularly in high-stakes domains like medical diagnostics and autonomous driving. However, existing post-hoc explainability methods face significant limitations:

Local methods (e.g., Grad-CAM, LIME) focus on individual samples, making them vulnerable to labeling noise and sampling bias, and failing to provide a global view of model behavior.
Concept-based methods (e.g., TCAV) rely on human-defined concepts, which may not align with the abstract representations actually learned by the network.
Global structural methods often focus on feature space organization but lack a systematic way to visualize class-level confusion dynamics over time.
Existing confusion matrix visualizations (e.g., ConfusionFlow) are often limited to the final layer and the converged model, missing the evolution of learning dynamics and intermediate layer behaviors.

A specific gap exists in constructing and analyzing confusion matrices (CMs) for hidden layers of neural networks (NNs) to understand how class separability and confusion evolve during training.

Methodology: GRAPHIC

The authors propose GRAPHIC (Graph-based Representation and Analysis of Predictions and Hidden-layer Interpretability via Confusion), an architecture-agnostic framework that treats class-level confusion as a network science problem.

1. Generating Confusion Matrices for Hidden Layers

Since hidden layers do not produce class predictions directly, GRAPHIC employs Linear Classifiers (LCs) trained on the feature vectors of intermediate layers.

Training Strategy: The LCs are trained using a custom cross-entropy loss function that integrates both true labels ( $y$ $y$ ) and the predictions of the NN under analysis ( $\tilde{y}$ $\tilde{y}$ ):
$L(z^{(k)}, y, \tilde{y}) = \lambda L_{CE}(z^{(k)}, y) + (1 - \lambda) L_{CE}(z^{(k)}, \tilde{y})$
- $\lambda = 1$ (True Labels): Measures the intrinsic linear separability of the features at that layer.
- $\lambda = 0$ (Predicted Labels): Measures the model's current "understanding" and training dynamics. The LC is trained to reproduce the NN's predictions, providing insights into class learning without relying on ground truth.
CM Construction: The trained LCs are applied to unseen feature vectors (from a held-out portion of the training set or validation set) to generate normalized confusion matrices $C$ .

2. Graph Representation

The confusion matrices are interpreted as adjacency matrices of directed graphs:

Nodes: Represent the classes in the dataset.
Edges: Represent the probability of a sample from class $i$ being predicted as class $j$ .
Weighting: The adjacency matrix is defined as $C_{ad} = C - C \odot I_N$ , isolating erroneous predictions (off-diagonal elements) to focus on confusion.

3. Network Science Analysis

The authors apply standard network science tools to these graphs:

Community Detection: Using Modularity ( $Q$ ) maximization (via the Directed Louvain algorithm) to identify "Confusion Communities" (CCs)—groups of classes that are frequently confused with each other. This reveals emergent semantic structures without predefined labels.
Assortativity: Computing the assortativity coefficient to measure how well the learned confusion structure aligns with predefined groupings (e.g., "natural" vs. "man-made" or dataset superclasses).
Temporal Evolution: Analyzing these graphs across training epochs and network layers to trace the evolution of learning dynamics.

Key Contributions

Novel Analysis Approach: GRAPHIC is the first architecture-agnostic method to generate and analyze confusion matrices for intermediate layers using LCs trained on both true and predicted labels. It interprets these matrices as directed graphs to leverage network science tools.
Dataset and Labeling Diagnostics: The method identifies hidden dataset issues, labeling ambiguities, and biases that are obscured by the sheer number of classes. These findings are validated through a human study.
Observation of Linear Separability Trends: The study provides empirical evidence on the linear separability of intermediate layers, notably observing a decline in separability in early decoders of visual transformers, contrasting with the monotonic increase seen in CNNs.

Experimental Results

Experiments were conducted on ResNet-50 (CNN) and EffVit (Vision Transformer) using CIFAR-100 and Tiny ImageNet.

1. Evolution of Confusion Communities

Early Training: A few dominant classes act as "confusion hubs" (e.g., "computer keyboard," "sea," "possum" in ResNet-50). The specific hubs depend heavily on the order of the training data, not just model initialization.
Convergence: As training progresses, semantically meaningful communities (e.g., animals, trees, humans) emerge. Intergroup confusion decreases, and modularity increases.
Layer Differences: In ResNet-50, deeper layers show clearer groupings and higher modularity. In EffVit, early decoders initially gain linear separability but then lose it as training progresses, suggesting a different learning dynamic compared to CNNs.

2. Dataset Issues and Biases

GRAPHIC successfully visualized specific dataset artifacts:

Leaf Color Bias: Maple trees and oak trees were frequently confused. Analysis revealed the model relied on seasonal leaf colors (red/yellow for maple, green for oak) rather than structural features. Modifying leaf colors in test images corrected the predictions.
Contextual Bias: A strong, non-reciprocal confusion was found between "flatfish" and "man." This was traced to dataset images where anglers hold flatfish, causing the model to associate the context (holding a fish) with the class "man."
Labeling Ambiguity: Classes like "boy," "girl," and "baby" formed a tight confusion community. A human study (31 participants) confirmed that these classes are inherently ambiguous even for humans due to image quality and age/gender definition boundaries.
Tiny ImageNet Issues: Confusions between "convertible" and "sports car," and "tabby" and "Egyptian cat" were identified, revealing label overlaps and definition inconsistencies in the dataset.

3. Linear Separability

ResNet-50: Linear separability (measured by LC accuracy on true labels) increased monotonically with depth and training epochs, consistent with prior literature.
EffVit: While later decoders showed increasing separability, early decoders exhibited a decline in linear separability after an initial rise. This suggests that visual transformers may learn locality and global context differently than CNNs, potentially unlearning linear separability in early stages to form more complex representations.

Significance and Claims

The paper claims that GRAPHIC offers a new perspective on how neural networks learn by uncovering "real confusions" through a data-driven, structural approach.

Actionable Insights: By visualizing class-level confusion, GRAPHIC helps identify dataset biases and labeling errors that standard accuracy metrics miss.
Architectural Understanding: It reveals fundamental differences in learning dynamics between CNNs and Transformers, specifically regarding the stability of linear separability in intermediate layers.
Trust and Reliability: The method provides interpretable visualizations that can guide dataset design and improve model reliability, which is crucial for high-stakes applications like medical imaging.
Limitations: The authors acknowledge the computational overhead of training LCs at multiple layers and epochs, though they suggest strategies like early stopping and reusing LCs to mitigate this. They also note that interpreting the semantic meaning of observed confusions still requires manual analysis.

In summary, GRAPHIC bridges the gap between local explainability and global structural analysis, using network science to provide a systematic, class-level view of the training process and dataset integrity.