🔢 mathematics

The Geometry of Polynomial Group Convolutional Neural Networks

This paper introduces a graded group algebra framework for polynomial group convolutional neural networks (PGCNNs) on arbitrary finite groups, establishing two parametrizations via Hadamard and Kronecker products, computing the dimension of the associated neuromanifold, and characterizing the general fiber of the Kronecker parametrization while conjecturing a similar description for the Hadamard case.

Original authors: Yacoub Hendi, Daniel Persson, Magdalena Larfors

Published 2026-04-01

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yacoub Hendi, Daniel Persson, Magdalena Larfors

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to recognize patterns, like spotting a specific type of flower in a garden. Usually, we teach robots by showing them thousands of pictures. But what if the garden has a special rule: no matter how you rotate the picture or shift it left or right, the flower is still the same flower?

This is the world of Geometric Deep Learning. Instead of forcing the robot to memorize every single angle, we build the robot with "symmetry" baked into its brain. It knows that a rotated flower is the same flower.

This paper is about a specific, highly mathematical way of building these symmetry-aware robots. The authors, Yacoub Hendi, Daniel Persson, and Magdalena Larfors, are looking at a special kind of robot brain called a Polynomial Group Convolutional Neural Network (PGCNN).

Here is the breakdown of their discovery, translated into everyday language.

1. The Robot's Brain: A "Group" of Rules

In normal AI, the robot looks at a grid of pixels. In this paper, the robot looks at a "Group." Think of a group not as a social gathering, but as a set of rules for moving things around.

Example: Imagine a clock face. The "group" is the set of all possible moves you can make: rotate 1 hour, rotate 2 hours, etc.
The robot's job is to process information that lives on this clock face. Because the clock is symmetrical, the robot's brain is designed to respect that symmetry.

2. The "Polynomial" Twist

Most AI uses a "switch" (like a light switch) to decide if a signal is strong enough to pass on. This paper replaces that switch with polynomials (mathematical curves like $x^2$ or $x^3$ ).

Why? Math is easier to study when things are smooth curves rather than sharp switches. It's like trying to measure the volume of a smooth ball versus a jagged rock. The smooth ball (polynomial) is much easier to calculate the volume of.
The authors prove that even though real-world AI uses "jagged" switches, studying the "smooth" polynomial version tells us almost everything we need to know about the real thing.

3. The Two Ways to Build the Brain (The Analogy)

The authors discovered that you can describe this robot's brain in two different ways, like describing a recipe in two different languages.

Method A: The "Hadamard" Way (The Ingredient Mixer)
Imagine you have a bowl of ingredients. You take a pinch of salt and mix it with a pinch of pepper. Then you take that mix and add a drop of oil. You are mixing things together in the same bowl. This is the Hadamard product. It's intuitive but messy to calculate the size of the final result.
Method B: The "Kronecker" Way (The Lego Tower)
Imagine you build a tower of Legos. You take a block, and you stack another block on top of it, and another. You are building a structure where every layer is distinct but connected. This is the Kronecker product. It's like building a giant, multi-dimensional Lego castle.

The Big Discovery: The authors found a "translator" (a linear map) that converts the messy "Ingredient Mixer" recipe into the clean "Lego Tower" recipe. They proved that both methods build the exact same robot brain, just described differently.

4. The "Neuromanifold": The Shape of Possibility

This is the most abstract part, so let's use a Lego analogy.

Imagine you have a box of Lego bricks. You can build millions of different castles.

The Neuromanifold is the "shape" of all the possible castles you can build with that specific box of bricks.
If you have 100 bricks, the "shape" of your possibilities is huge and complex.
The authors wanted to know: How big is this shape?

In math, "size" is measured by dimension.

A line is 1-dimensional.
A flat sheet is 2-dimensional.
A cube is 3-dimensional.

The authors calculated the dimension of this "shape" for their robot brains.

The Surprise: They found that the size of the shape does not depend on the complexity of the rules (the group structure) or how "curvy" the polynomial is.
The Only Things That Matter:
1. How many layers of Lego you stack (the depth of the network).
2. How many "slots" are in your group (the size of the group, e.g., a 12-hour clock vs. a 24-hour clock).

It's like saying: "No matter how fancy the Lego instructions are, if you have 5 layers of bricks and 10 types of bricks, the total number of unique castles you can build is always the same."

5. The "Fiber": Are There Multiple Ways to Build the Same Castle?

Finally, they asked: "If I give you a finished castle, can you figure out exactly which bricks I used to build it?"

Sometimes, you can build the exact same castle using slightly different arrangements of bricks. This is called a fiber.
The authors proved that for these robot brains, the "fiber" is very small. It basically means: "If you build a castle, there is only one way to build it, up to a simple rotation or a scaling factor (making it slightly bigger or smaller)."
This is great news for AI! It means the robot's brain is identifiable. We can look at the final result and know exactly what the internal settings were. There aren't hidden, confusing ways to get the same result.

Summary: Why Does This Matter?

This paper is like a blueprint for a new kind of factory.

Simplicity: They showed that even though the math looks scary (using words like "graded group algebras"), the underlying structure is surprisingly simple.
Efficiency: They proved that the "capacity" of these networks (how much they can learn) depends only on the number of layers and the size of the symmetry group, not on the specific details of the group.
Reliability: They showed that these networks are stable and predictable. You won't get lost in a maze of different settings that all produce the same result.

In short, the authors took a very complex, abstract mathematical object (a polynomial group convolutional network) and mapped out its geometry, proving that it is well-behaved, predictable, and easier to understand than we thought. They essentially drew a map of the "universe of possibilities" for these specific AI models.

1. Problem Statement

The paper addresses the lack of a rigorous geometric understanding of Polynomial Group Convolutional Neural Networks (PGCNNs). While Geometric Deep Learning has established that equivariant architectures (like CNNs) reduce sample complexity by enforcing symmetry, the internal geometric structure of the function spaces these networks generate remains poorly understood.

Specifically, the authors aim to analyze the neuromanifold $\mathcal{M}_\Phi$ associated with PGCNNs. A neuromanifold is the set of all realizable functions by a network architecture. Understanding its geometry (dimension, singularities, and fiber structure) is crucial for determining:

Expressivity: How complex a function the network can represent.
Identifiability: Whether different network parameters map to the same function (i.e., are the parameters uniquely determined?).
Training Dynamics: How the geometry influences optimization (e.g., the presence of singularities).

The challenge lies in the compositional complexity of these networks and the non-analytic nature of standard activation functions (like ReLU). The authors restrict their study to polynomial activation functions to leverage tools from algebraic geometry, justifying this via the Weierstrass Approximation Theorem and the fact that polynomial networks have shown efficacy in various fields.

2. Methodology

The authors develop a novel mathematical framework based on graded group algebras to formalize PGCNNs.

Algebraic Formulation: They define the network filters as elements of a graded group algebra $S[G]$ , where $G$ is an arbitrary finite group and $S$ is a commutative ring (e.g., polynomials).
Two Parametrizations: The core methodological innovation is describing the polynomial activation function (degree $r$ $r$ ) in two equivalent but structurally distinct ways:
1. Hadamard Parametrization ( $\Phi$ ): Uses the component-wise (Hadamard) product. This maps parameters $\theta \in (K[G])^L$ to the output function space directly.
2. Kronecker Parametrization ( $\varphi$ ): Uses the Kronecker product and convolution. This maps parameters to a higher-dimensional space $K[G^{r^{L-1}}]$ .
- These two maps are related by a linear map $\Lambda$ such that $\Phi = \Lambda \circ \varphi$ .
Symbolic Rank Analysis: To compute the dimension of the neuromanifold, the authors treat the Jacobian matrices of the parametrization maps as symbolic matrices (matrices with polynomial entries). They analyze the rank of these matrices over a dense set (general condition) rather than specific numerical values.
Fiber Analysis: They investigate the "general fiber" of the parametrization maps (the set of parameters mapping to a single function) to determine identifiability. This involves induction on the number of layers and the use of symmetric tensor rank lemmas.

3. Key Contributions

Algebraic Framework: Introduced a formal language using graded group algebras to describe PGCNNs, unifying the treatment of arbitrary finite groups and polynomial activations.
Dual Parametrization: Established the relationship between Hadamard and Kronecker product parametrizations, showing they are related by a linear map, allowing the authors to switch between the "function space" view and the "filter space" view.
Dimension Computation: Proved that the dimension of the neuromanifold depends only on the size of the group $|G|$ and the number of layers $L$ , and is independent of the activation degree $r$ or the specific group structure.
Fiber Classification: Provided a precise description of the general fiber for the Kronecker parametrization and conjectured the same for the Hadamard parametrization, characterizing the non-uniqueness of parameters as a combination of group actions and rescaling.
Software Tool: Developed PGCNNGeometry, a package using Sage and Macaulay2 to compute dimensions and fiber sizes for small groups, validating theoretical results.

4. Key Results

A. Dimension of the Neuromanifold

For a PGCNN with depth $L \ge 1$ , activation degree $r \ge 2$ , and finite group $G$ of size $n = |G|$ :

The dimension of the neuromanifold $\mathcal{M}_\Phi$ (Hadamard) and its image $\mathcal{M}_\varphi$ (Kronecker) is:
$\dim(\mathcal{M}) = L(n - 1) + 1$
Significance: This result is counter-intuitive to general polynomial neural networks (PNNs), where dimension often depends on width and degree. Here, the weight-sharing constraints imposed by group equivariance reduce the effective degrees of freedom. The dimension is invariant under the group's internal structure (e.g., cyclic vs. non-abelian) and the polynomial degree $r$ .

B. General Fiber and Identifiability

The authors characterize the "general fiber" (the set of parameters producing the same function, up to scaling):

Kronecker Parametrization ( $\varphi$ ): The general fiber is finite. Specifically, two parameter tuples $\theta$ and $\psi$ map to the same function if and only if they are related by a sequence of group translations and rescalings:
$\psi = (\theta_1 * g_1, g_1^{-1} * \theta_2 * g_2, \dots, g_{L-1}^{-1} * \theta_L)$
where $g_i \in G$ and $*$ denotes convolution.
Hadamard Parametrization ( $\Phi$ ): The authors conjecture that the same fiber structure holds. This is supported by explicit computations for small groups (e.g., $Z_2, Z_3, S_3$ ) and shallow networks.
Implication: The network is "generically identifiable" up to these symmetries (group actions and scaling).

C. Singularities

The paper notes that singularities in the neuromanifold (where the Jacobian rank drops) occur when the matrix representation of a convolution layer is not full rank. This suggests a link between local layer singularities and global network singularities, which may impact training dynamics (e.g., convergence to extrema).

5. Significance and Outlook

Theoretical Advancement: This work generalizes previous results on polynomial CNNs (which were limited to cyclic groups $\mathbb{Z}_n$ ) to arbitrary finite groups. It demonstrates that the geometric properties of equivariant networks are robust across different symmetry groups.
Practical Implications: By quantifying the dimension and fiber structure, the paper provides bounds on the sample complexity required to train these networks and clarifies the identifiability of learned features.
Future Directions:
- Proving the conjecture regarding the Hadamard fiber.
- Extending the framework to compact groups (infinite groups) using vector bundles or discretization.
- Investigating the singular locus to understand how singularities affect training convergence.
- Generalizing to sums of Hadamard products to approximate non-polynomial activations more broadly.

In summary, the paper establishes that the geometry of polynomial group convolutional networks is governed by the size of the symmetry group and the network depth, providing a rigorous algebraic foundation for understanding the expressivity and identifiability of equivariant deep learning models.