🧬 biology

A Quotient Homology Theory of Representation in Neural Networks

This paper introduces a metric-free "overlap homology" framework that leverages the piecewise linear structure of ReLU neural networks to define a quotient space, enabling the intrinsic calculation of Betti numbers that track purely topological features of neural representations rather than geometric ones.

Original authors: Kosio Beshkov

Published 2026-05-29

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Kosio Beshkov

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

The Big Picture: How Neural Networks "Glue" Things Together

Imagine a neural network as a giant, complex machine that takes a messy pile of raw data (like photos of cats and dogs) and tries to sort them out. To do this, the machine squishes, stretches, and folds the data until the cats are in one pile and the dogs are in another.

Mathematicians have long known that these networks act like origami artists. They take a flat sheet of paper (the input data) and fold it into different shapes. The paper is divided into flat, geometric pieces called polyhedra (think of them as flat tiles). On each tile, the machine performs a simple, straight-line movement (an "affine map").

The problem is: How do we know if the machine is actually understanding the shape of the data, or if it's just getting lucky with the geometry?

The Problem with Current Tools: The "Ruler" Trap

Currently, scientists use a tool called Persistent Homology to study these shapes. Think of this tool like a ruler or a tape measure. It looks at the data and asks, "How far apart are these points?"

The Flaw: If you use a ruler, you are measuring geometry (distance, curves, angles), not just topology (the actual shape, like whether something has a hole or is connected).
The Analogy: Imagine you have a rubber band with a hole in the middle (a donut shape). If you stretch it out, it's still a donut. But if you use a ruler to measure the distance between points, the stretching might make it look like the hole has disappeared or changed size. The ruler is too sensitive to how the object is stretched, rather than what the object is.

The authors argue that because neural networks stretch data so wildly, standard tools are getting confused by the stretching (geometry) and missing the actual structure (topology).

The New Solution: The "Glue" Map

The authors propose a new way to look at the network that ignores the ruler entirely. Instead of measuring distances, they ask a simpler question: "Which points does the network glue together?"

If the network takes two different points from the input and sends them to the exact same spot in the output, it has "glued" them together. This is the only way the network can change the fundamental shape of the data (like turning a donut into a solid ball by gluing the hole shut).

They call this new method Quotient Homology.

The Two Ways Gluing Happens

The paper identifies two ways the network glues points together:

The Rank Source (The Flattener): Imagine pressing a 3D ball of clay flat onto a table. It becomes a 2D pancake. The network does this by collapsing a whole region of data into a lower dimension.
The Overlap Source (The Glue): Imagine two different pieces of paper (polyhedra) that the network folds. Even though they started in different places, the network folds them so that they land on top of each other. The points on these two different pieces are now "glued" together in the output.

The authors prove a surprising fact: If the data sits nicely on the flat tiles (which is usually true), you only need to worry about the "Overlap Source." You can ignore the "flattening" part and just look at where the different tiles land on top of each other.

How They Did It (The Algorithm)

To find these "glued" points without using a ruler, the authors built a computer algorithm that acts like a detective:

Divide and Conquer: They split the input data into the flat tiles (polyhedra) the network creates.
The "Can They Meet?" Test: For every pair of tiles, they use a math technique called Linear Programming to ask: "Is there any point in Tile A and any point in Tile B that the network sends to the exact same destination?"
The Union-Find: If the answer is yes, they mark those points as "glued." They use a simple grouping method (like sorting socks into pairs) to see which big groups of points are now stuck together.

What They Found

They tested this new method on simple, made-up datasets (like circles, rings, and spheres) and compared it to the old "ruler" method.

Pure Topology: Their new method successfully ignored the stretching and only counted the actual holes and connections. It didn't get confused when the data was squished or curved.
Slower Changes: When they watched the network learn (train), they found that the "topology" (the holes and shapes) changes much more gradually than previous studies suggested. The old ruler-based methods made it look like the network was instantly destroying the shape of the data. The new method shows it happens slowly, step-by-step.
Training Effect: As the network learns to classify things (like sorting cats from dogs), the "glued" regions (overlaps) get smaller in volume, but there are more of them. The network becomes more precise in how it glues specific groups of data together.

The Limitations (The "Gotchas")

The authors are honest about where their method might stumble:

The "Unseen" Glue: Their method only looks at the data points they actually gave the network. If the network has a hidden "glue" spot that no training data ever touched, the method might miss it (a Type 2 error).
The "Fake" Glue: Conversely, the method might think two points are glued because they land on the same spot in the mathematical space, even if those points don't actually exist in the real world data (a Type 1 error).
Computational Cost: Checking every possible pair of tiles to see if they glue together is very hard for massive networks. It's like trying to check every possible handshake in a stadium of millions of people. It works for small to medium networks but gets expensive for huge ones.

Summary

This paper introduces a new "glue detector" for neural networks. Instead of measuring how far apart data points are (which gets messy when the network stretches them), it simply asks: "Which points did the network decide to stick together?"

By focusing on this "gluing," they can see the true shape of the data as the network processes it, revealing that topological changes happen more slowly and subtly than we previously thought. It's a way to see the structure of the network's thinking without getting distracted by the stretching of the data.

Technical Summary: A Quotient Homology Theory of Representation in Neural Networks

Problem Statement
Deep neural networks with ReLU activation functions are known to implement continuous piecewise-linear (CPWL) functions, partitioning the input domain into convex polyhedra (linear regions) over which the network operates affinely. While Topological Data Analysis (TDA), specifically persistent homology, has been used to study the topology of neural representations, standard methods rely on external metrics (e.g., Euclidean distance) to construct simplicial complexes. This reliance introduces geometric biases, such as curvature and convexity, which can conflate geometric features with purely topological ones. Furthermore, persistent homology is sensitive to outliers and sampling density. The paper addresses the need for a metric-free approach to quantify the topological changes induced by neural networks, specifically distinguishing between geometric distortions and genuine topological simplifications (e.g., gluing distinct parts of the manifold).

Methodology
The authors propose a framework based on quotient homology that leverages the intrinsic structure of ReLU networks. The core methodology involves three steps:

Polyhedral Decomposition: The input space is decomposed into convex polyhedra ( $G_J$ ) based on the activation patterns (codewords) of the network. Within each polyhedron, the network acts as an affine map $\Phi_J$ .
Overlap Decomposition ( $O_\Phi$ ): The authors define an equivalence relation $\sim_\Phi$ where $x \sim_\Phi x'$ if $\Phi(x) = \Phi(x')$ . They identify two sources of non-injectivity (topological change):
- Rank Source: The local affine map $\Phi_J$ has a rank lower than the dimension of the manifold, projecting the region to a lower-dimensional subspace.
- Overlap Source: Distinct polyhedra $G_{J_i}$ and $G_{J_j}$ map to intersecting images in the output space ( $\bigcap \text{Im}\Phi_{J_i} \neq \emptyset$ ).
The paper focuses on the overlap decomposition, which collects equivalence classes arising from the non-trivial intersections of images of different polyhedra.
Numerical Computation:
- The overlap decomposition is computed algorithmically using Linear Programming (LP). For pairs of populated polyhedra, the method checks for feasibility of points $p \in G_{J_1}$ and $p' \in G_{J_2}$ such that $\Phi(p) = \Phi(p')$ under the constraints defined by the H-representation of the polyhedra.
- A Union-Find algorithm is used to group these overlapping points into equivalence classes.
- The Betti numbers of the neural representation are then calculated as the homology of the quotient space $H_k(M/O_\Phi)$ , where $M$ is the input manifold.

Key Theoretical Results
The paper establishes a fundamental theorem (Theorem 3.2) stating that if the intersection between the data manifold $M$ and any polyhedron $G_J$ is convex, the homology groups of the neural representation are isomorphic to the quotient homology groups of the overlap decomposition:
$H_k(\Phi(M)) \simeq H_k(M/O_\Phi)$
Under this condition, the rank source of non-injectivity does not alter the homology groups, allowing the overlap decomposition alone to suffice for calculating topological invariants without external metrics.

Experimental Results
The authors validate their approach through several experiments:

Toy Datasets (Non-linear Curves): Comparing quotient homology against standard persistent homology on datasets with known topology. The results show that persistent homology often identifies geometric features (e.g., false circular structures due to proximity in output space) that quotient homology correctly ignores, as it only detects points explicitly glued by the network map.
Manifold Propagation: Revisiting the evolution of Betti numbers through network layers on datasets from Naitzat et al. (2020). While persistent homology suggested rapid topological simplification in early layers, the quotient homology approach reveals that topological changes occur much more gradually. This suggests that early layers primarily induce geometric twisting rather than topological gluing.
Training Dynamics: Analyzing random vs. trained networks on sphere classification tasks. The study finds that while the volume of overlapping regions decreases after training, the number of overlap regions tends to increase, particularly in higher dimensions.

Significance and Claims
The paper claims to provide a novel, metric-free perspective on neural representations by shifting focus from the output space geometry to the mapping structure itself. Key contributions include:

Intrinsic Topological Analysis: A method to calculate Betti numbers that is robust to geometric distortions and does not require the selection of an external metric.
Refined Understanding of Topological Change: Evidence that topological simplification in deep networks is a gradual process, contrasting with previous findings based on persistent homology.
New Measure of Expressivity: The overlap decomposition offers a new way to quantify network expressivity by measuring the capacity to implement non-injective maps, distinct from simply counting linear regions.
Theoretical Rigor: The work connects polyhedral decompositions of CPWL functions with quotient homology, providing conditions under which the overlap decomposition fully characterizes the topology of the representation.

The authors acknowledge limitations, including the computational cost of determining H-representations for high-dimensional spaces and potential "homological type 1 and 2 errors" arising from unpopulated polyhedra or limited dataset sampling. They conclude that this framework opens avenues for rigorously distinguishing between topological and geometric transformations in deep learning.