Conceptual Views of Neural Networks: A Framework for Neuro-Symbolic Analysis

This paper introduces "conceptual views," a Formal Concept Analysis-based framework that enables faithful global explanation, architecture comparison, and abductive rule learning for neural networks, as demonstrated through experiments on ImageNet and Fruits-360 models.

The Gist

Imagine you have a super-smart robot chef (a Neural Network) that can perfectly identify thousands of different fruits. It's amazing at its job, but it's also a "black box." You ask it, "Why did you think this is an apple?" and it just stares back, unable to explain its reasoning in human words. It just knows the answer, but not how it got there.

This paper introduces a new way to peek inside that black box and translate the robot's secret language into something we can understand. The authors call this framework "Conceptual Views."

Here is the breakdown of their idea using simple analogies:

1. The Problem: The Robot's Secret Code

Inside the robot's brain, information is stored as numbers. When you show it a picture of an apple, it doesn't see "red" or "round." It sees a complex pattern of numbers (activations) flowing through its layers of neurons.

• The Issue: Humans can't read these raw numbers. We need a translator.

2. The Solution: Two Ways to Look at the Robot

The authors propose looking at the robot's brain in two different ways, like taking a photo of a building from two different angles.

Angle A: The "Many-Valued View" (The High-Res Map)

Imagine you take a snapshot of the robot's brain and write down exactly how active every single neuron is for every fruit.

• What it does: It creates a massive spreadsheet of numbers.
• Why it's cool: It is incredibly accurate. If you use this spreadsheet to guess the fruit, you get almost the exact same result as the original robot. It proves that we can capture the robot's entire knowledge in a structured format without losing any accuracy.
• The Analogy: It's like having a perfect, high-resolution map of a city. You can navigate it perfectly, but it's still just a map of coordinates, not a story.

Angle B: The "Symbolic View" (The Binary Switchboard)

This is the real magic. The authors take that messy spreadsheet of numbers and turn it into a simple On/Off switchboard.

• How it works: They set a rule: "If a neuron's number is above a certain line, it's ON (1). If it's below, it's OFF (0)."
• The Result: Suddenly, the complex math becomes a simple list of "Yes/No" facts.
  • Example: "Neuron 5 is ON" and "Neuron 12 is OFF."
• Why it's cool: This turns the robot's brain into a format that logic and human reasoning can understand. It's like translating a foreign language into simple English sentences.

3. The "Dictionary" (Connecting to Human Knowledge)

Once the robot's brain is translated into "On/Off" switches, the authors use a tool called Formal Concept Analysis (FCA). Think of FCA as a super-smart librarian.

• The Setup: You give the librarian the robot's "On/Off" switches and a dictionary of human knowledge (e.g., "Apples are red," "Bananas are curved").
• The Magic: The librarian looks for patterns. It might discover:
  • "Whenever Neuron 5 is ON and Neuron 12 is OFF, the robot is thinking about 'Red Fruits'."
  • "Whenever Neuron 3 is ON, it means the fruit is 'Round'."
• The Outcome: You can now ask the robot, "Why did you pick the apple?" and it can answer: "Because Neuron 5 was ON (Red) and Neuron 3 was ON (Round), which matches the pattern for Apples."

4. Comparing Different Robots

The paper also shows how to compare two different robots (e.g., a "VGG" robot vs. a "ResNet" robot).

• Instead of comparing their code line-by-line, they compare the shapes of their internal maps.
• Imagine two different maps of the same city. One might be drawn by a tourist, the other by a taxi driver. They look different, but if you measure the distance between landmarks, you can see how similar their understanding of the city is.
• The authors use a mathematical tool (Gromov–Wasserstein distance) to measure this "shape similarity," helping us understand which robots learn things in similar ways.

5. Why This Matters

• Trust: We can finally trust AI in high-stakes situations (like medicine or law) because we can ask it to explain its logic in plain English.
• Debugging: If the robot makes a mistake, we can look at the "switchboard" to see exactly which "On/Off" pattern went wrong.
• No Black Boxes: It turns the mysterious "black box" into a transparent "glass box" where we can see the gears turning.

Summary

The authors built a translator that turns the robot's secret math language into a simple On/Off switchboard. By connecting these switches to human concepts (like "red" or "round"), they allow us to read the robot's mind, compare different robots, and understand exactly why they make the decisions they do. It's like giving a super-intelligent alien a dictionary so they can finally tell us what they are thinking.

Technical Summary

1. Problem Statement

Neural networks (NNs) achieve state-of-the-art performance but suffer from a lack of human explainability, creating a tension between accuracy and interpretability. Existing approaches generally fall into two categories:

• Local Explanations: Methods like saliency maps explain individual predictions but fail to characterize the model globally or handle high-dimensional data effectively.
• Global Explanations: These are less explored but essential for understanding the model as a whole. Current global methods often rely on pre-defined concepts or require architectural changes (e.g., Concept Bottleneck Models).

The authors identify a gap in global, post-hoc analysis that can faithfully represent the internal logic of a neural network, compare different architectures, and derive human-comprehensible rules without altering the original model's architecture.

2. Methodology: Conceptual Views

The authors propose a formal framework called Conceptual Views, grounded in Formal Concept Analysis (FCA). The approach operates on the embedding space of the network, specifically the outputs of the last hidden layer. It consists of two complementary representations:

A. Many-Valued Conceptual View (MV View)

This representation captures the continuous, real-valued structure of the network.

• Object View ($O$): A matrix where rows represent input objects and columns represent neurons in the last hidden layer. Values are the activation levels $n_j(g_i)$.
• Class View ($W$): A matrix where rows represent output classes and columns represent neurons. Values are the weights $w_{i,j}$ connecting hidden neurons to output classes.
• Functionality: These views induce a pseudo-metric space on objects and classes. The classification logic is interpreted as the cosine similarity or Euclidean distance between an object's activation vector and a class's weight vector.
• Application: This view serves as a high-fidelity surrogate for the original NN, allowing for the comparison of different architectures using the Gromov–Wasserstein (GW) distance.

B. Symbolic Conceptual View (SV View)

This representation discretizes the MV view to enable symbolic reasoning.

• Conceptual Scaling: The continuous values in $O$ and $W$ are converted into a binary formal context using dichotomic scaling.
• Thresholding: A threshold $\delta$ is applied to activations and weights. For a neuron $n$ and threshold $\delta$, two symbolic attributes are created: $n_{\leq \delta}$ (activation $\leq \delta$) and $n_{\geq \delta}$ (activation $> \delta$).
• Formal Context: The result is a binary relation $K = (G, M, I)$ where objects/classes are related to binary attributes derived from neurons.
• Abductive Learning: This binary structure allows for the integration of background knowledge (e.g., ontologies, taxonomies) to derive human-readable rules linking neurons to semantic concepts.

3. Key Contributions

1. Formal Framework: Introduction of "Conceptual Views" as a principled, algebraic method to translate neural representations into formal contexts without modifying the network architecture.
2. Architecture Comparison: Demonstration that the Gromov–Wasserstein distance applied to Conceptual Views provides a robust, permutation-invariant metric for comparing the structural similarity of different neural network architectures.
3. Neuro-Symbolic Bridge: The ability to generate Concept Lattices from the symbolic view, enabling the extraction of propositional implications and hierarchical dependencies between learned features.
4. Abductive Rule Extraction: A method to use subgroup discovery on the symbolic view to derive human-comprehensible rules (e.g., "If neurons X, Y, and Z are active, then the object is likely an Orange") by integrating external background knowledge.

4. Experimental Results

The framework was evaluated on 24 ImageNet models and the Fruits-360 dataset.

• Fidelity of Many-Valued Views:

  • Using a 1-Nearest Neighbor (1-NN) classifier on the pseudo-metric space derived from the MV view, the authors achieved high fidelity (up to 99.9%) compared to the original models for architectures like ResNet and EfficientNet.
  • Euclidean distance generally outperformed cosine similarity, particularly for ResNet models.
  • MobileNet showed lower fidelity due to aggressive dimensionality reduction.

• Architecture Similarity:

  • GW distance successfully clustered models with similar architectures (e.g., VGG16/19, ResNet variants) together, revealing structural patterns that pairwise fidelity metrics missed.

• Symbolic View Performance:

  • Activation Function Sensitivity: The choice of activation function is critical. Tanh yielded the best results for dichotomic scaling (threshold $\delta=0$) because it produces symmetric positive/negative distributions. ReLU performed poorly in symbolic views because its non-negative range makes defining "negative" attributes difficult.
  • Classification: While 1-NN on symbolic views struggled with ReLU models, Decision Trees trained on the symbolic view of Tanh-based models achieved competitive accuracy (e.g., ~98% on Fruits-360), proving the symbolic view is a faithful surrogate for interpretable classifiers.
  • Class Separation: Symbolic views achieved perfect class separation (Class Sep = 1.0) in many cases, indicating the binary representation retains sufficient information for classification.

• Reasoning and Interpretability:

  • The framework successfully identified indistinguishable classes (e.g., "Cherry" vs. "Plum" in certain models) and visualized their structural relationships in concept lattices.
  • Subgroup discovery generated rules linking specific neurons to semantic features (e.g., color, shape, taxonomy), validating the neuro-symbolic integration.

5. Significance and Limitations

Significance:

• Global Explainability: Unlike local methods, this framework explains the entire model's logic and knowledge structure.
• Model-Agnostic: It does not require retraining or architectural constraints (unlike Concept Bottleneck Models).
• Theoretical Grounding: It bridges the gap between deep learning and formal logic (FCA), enabling the use of established symbolic reasoning tools (description logics, implication mining) on neural networks.

Limitations:

• Architecture Constraints: Currently limited to feed-forward networks with a distinct last hidden layer; not yet adapted for recurrent networks or Transformers.
• Activation Sensitivity: The symbolic view relies heavily on symmetric activation functions (like Tanh); ReLU requires more complex scaling strategies.
• Scalability: Concept lattices can grow exponentially with the number of attributes, making direct visualization difficult for large networks (though computational analysis remains feasible).
• Background Knowledge Dependency: The quality of human-readable rules depends on the availability and accuracy of external background knowledge (ontologies).

In conclusion, the paper presents a rigorous mathematical framework that transforms the "black box" of neural networks into a structured, analyzable, and explainable system, offering a new pathway for neuro-symbolic AI.