🤖 machine learning

bispectrum: Selective $G$ -Bispectra Made Practical

This paper introduces **bispectrum**, an open-source PyTorch library that implements efficient, selective $G$ -bispectra for seven group actions to achieve near-exact invariance with reduced computational costs, demonstrating consistent performance improvements over existing pooling methods in low-data, moderate-capacity deep learning regimes.

Original authors: Johan Mathe, Adele Myers, Simon Mataigne, Nina Miolane

Published 2026-05-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Johan Mathe, Adele Myers, Simon Mataigne, Nina Miolane

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to recognize a friend in a crowded room. If they turn around, walk to the other side, or tilt their head, they are still the same person. In the world of machine learning, computers often struggle with this. If you show a computer a picture of a cat, and then show it the same cat rotated 90 degrees, a standard computer might think it's a completely different object.

To fix this, scientists use "symmetry." They teach the computer that a rotation is just a transformation, not a new object. But there's a catch: when you try to summarize a complex image into a simple "fingerprint" that ignores these rotations, you often lose important details. It's like trying to describe a song by only listing the volume of each note, but forgetting the order in which they play. You lose the melody.

This paper introduces a new tool called bispectrum (specifically, a "selective G-bispectrum") that solves this problem. Here is how it works, using simple analogies:

1. The Problem: The "Incomplete Fingerprint"

Imagine you have a unique, intricate snowflake. You want to create a description of it that stays the same no matter how you spin it.

Old Methods (Norm Pooling): This is like taking a photo of the snowflake, measuring how much "white" is in the picture, and ignoring the shape. If you have two different snowflakes with the same amount of white, the computer thinks they are identical. It throws away the "shape" details (the phase).
The New Method (Bispectrum): This is like taking a 3D scan that captures every tiny ridge and valley. Crucially, it records the relationships between the ridges. Even if you spin the snowflake, the relationships between the ridges stay the same. This creates a "complete fingerprint" that preserves every detail of the original shape, just rotated.

2. The Innovation: "Selective" and "Practical"

The math behind this "complete fingerprint" (the G-bispectrum) has existed for decades, but it was too heavy to use.

The Old Way: Calculating the full fingerprint was like trying to count every single grain of sand on a beach to describe the beach. It was too slow and expensive for computers.
The "Selective" Breakthrough: The authors realized you don't need to count every grain of sand. You only need a specific, cleverly chosen handful of grains to reconstruct the whole beach perfectly. They call this selectivity.
- For simple groups (like rotating a 2D image), this reduces the work from a massive square number to a manageable linear number.
- For complex 3D rotations (like spinning a globe), they invented a new way to pick the "handful of grains" that cuts the work down from a cubic explosion to a manageable square.

3. The Tool: A "Plug-and-Play" Library

The authors didn't just do the math; they built a software library called bispectrum.

Think of this library as a universal adapter. In the past, if you wanted to use this math for 2D images, you needed one code; for 3D medical scans, you needed another; for spherical data, another. It was a "patchwork" of different tools.
This library is a single, clean "plug-in" that works for seven different types of symmetry (like 2D rotations, 3D rotations, and translations). You can drop it into any modern AI system, and it instantly starts working.
It is incredibly fast. On a modern computer chip (GPU), it calculates these complex fingerprints in less than a millisecond—faster than you can blink.

4. The Results: Better with Less Data

The authors tested this new tool on three different challenges:

Medical Pathology (2D): Identifying cancer in tissue samples.
3D Organ Classification: Identifying organs in CT scans.
Spherical Digits: Recognizing numbers drawn on a sphere (like a globe).

The Finding:
When the computer has lots of data, the new tool performs about as well as the old, simpler methods.
However, when the computer has very little data (the "low-data regime"), the new tool shines. Because it preserves all the information (it's "complete"), it learns much faster and more accurately than the old methods that throw away details. It's like a student who memorizes the whole textbook versus a student who only memorizes the chapter summaries; when the exam is tricky, the one with the full knowledge wins.

Summary

The paper presents a practical, fast, and open-source software library that allows AI to understand shapes and patterns perfectly, regardless of how they are rotated or moved. By using a "selective" mathematical trick, they made a previously too-complex calculation fast enough to use in real-world deep learning, proving that it helps AI learn better when data is scarce.

Where to find it: The code is free and open-source, available on GitHub under the name bispectrum.

Technical Summary: bispectrum: Selective G-Bispectra Made Practical

Problem Statement

Many machine learning tasks require invariance under specific group actions (e.g., translations, 2D/3D rotations). While equivariant neural networks enforce this by maintaining equivariant feature maps, the final step requires mapping these features to invariant representations for prediction. Existing invariant pooling methods (norm pooling, gated pooling, max pooling) are computationally efficient but incomplete: they discard phase information, meaning distinct signals that share the same power spectrum but differ in relative phases are mapped to identical descriptors.

The G-bispectrum offers a principled, complete invariant that retains all signal information up to the group action. However, its adoption in deep learning has been hindered by:

High computational cost: Full bispectra scale quadratically ( $O(|G|^2)$ ) for finite groups and cubically ( $O(L^3)$ ) for spherical rotations.
Fragmented implementations: Existing constructions are scattered across theoretical papers with no unified, differentiable, GPU-ready library.
Lack of empirical evaluation: There is no unified benchmark comparing bispectral pooling against standard alternatives in modern deep learning pipelines.

Methodology

The authors present bispectrum, an open-source PyTorch library that implements selective G-bispectra as differentiable nn.Module layers. The core methodology involves three technical pillars:

1. Selective Bispectrum Construction

Instead of computing the full bispectrum (which contains redundant information), the library computes a selective subset of coefficients that remains complete for generic signals.

Finite Groups: Leveraging a breadth-first search over the Kronecker product table of irreducible representations (irreps), the library selects $O(|G|)$ coefficients instead of $O(|G|^2)$ .
Planar Rotations (Disk): Implements the selective disk bispectrum, reducing coefficients from $O(m^3)$ to $O(m)$ while maintaining completeness under genericity conditions.
Spherical Rotations (SO(3) on $S^2$ ): The authors introduce a novel augmented selective bispectrum.
- They construct an invariant $\Phi_{sel}$ of size $\Theta(L^2)$ (reducing from the full $\Theta(L^3)$ ).
- The construction uses a "seed" block for low degrees and "bootstrap" blocks for higher degrees.
- To ensure completeness on real signals (where the standard scalar bispectrum may be rank-deficient due to parity constraints), they augment the bispectral entries with CG-power scalars (squared norms of projected tensor products).
- This results in an invariant of size $\Theta(L^2)$ that is empirically complete.

2. Library Design and Implementation

The library provides a uniform API for seven group/domain pairs (e.g., $C_n$ on $C_n$ , $SO(3)$ on $S^2$ , Octahedral group).

Differentiability: All modules are fully differentiable, allowing end-to-end training.
Efficiency: Pre-computation of Clebsch-Gordan matrices and kernels ensures sub-millisecond forward passes on GPUs.
Inversion: The library supports signal reconstruction (up to group action) from the bispectrum, serving as a proof of completeness.

3. Experimental Evaluation

The authors evaluate the library on three benchmarks, comparing bispectral pooling against norm pooling, gated pooling, Fourier-ELU pooling, max pooling, and data-augmented non-equivariant baselines:

PatchCamelyon (PCam): 2D histopathology classification under $C_8$ symmetry.
OrganMNIST3D: 3D organ classification under the chiral octahedral group $O$ .
Spherical MNIST: Digit classification under $SO(3)$ rotations on the sphere.

Key Results

Computational Performance

Speed: Selective forward passes are sub-millisecond on an NVIDIA H100 GPU (e.g., 0.48 ms for $SO(3)$ at band-limit $L=16$ ).
Reduction: Selectivity reduces coefficient counts by up to $512\times$ (e.g., for $|G|=1024$ ) and reduces spherical complexity from $O(L^3)$ to $\Theta(L^2)$ .

Empirical Completeness (Spherical MNIST)

Reconstruction: The augmented selective invariant $\Phi_{sel}$ successfully reconstructs signals up to rotation with low residuals ( $\sim 10^{-3}$ ), providing empirical evidence of completeness.
Invariance: The model achieves near-exact $SO(3)$ invariance ( $\sigma_{rot} \sim 10^{-4}$ ), whereas standard CNNs collapse under unseen rotations.
Completeness vs. Capacity: The power spectrum (incomplete invariant) plateaus at $\sim 78\%$ accuracy even when scaled to match the parameter count of the bispectrum model ( $95\%$ ), confirming that the performance gap is due to incompleteness, not model capacity.

Data Efficiency

Low-Data Regime: In all three benchmarks, the bispectral pooling layer consistently outperforms alternatives when training data is scarce (e.g., 1% of the dataset).
- On PCam (1% data), bispectrum achieved 0.912 AUC vs. 0.873 for NormReLU.
- On OrganMNIST3D (5% data), bispectrum reached 33.2% accuracy vs. 18.9% for max pooling.
High-Capacity Regime: As model capacity and data size increase, the advantage of the complete invariant narrows, and incomplete methods (like max pooling) can match or surpass bispectral performance.

Significance and Claims

The paper claims to close the gap between the theoretical appeal of G-bispectra and their practical utility in modern deep learning.

Practical Deployment: By providing a unified, differentiable, and efficient library, the authors make selective G-bispectra accessible as "drop-in" layers for any equivariant architecture.
Theoretical Contribution: The introduction of the augmented selective SO(3) invariant on the sphere is a significant theoretical step, reducing the complexity of spherical bispectra to $\Theta(L^2)$ while empirically demonstrating completeness.
Inductive Bias: The results suggest that complete invariants provide a stronger inductive bias than incomplete ones, particularly in low-data, moderate-capacity regimes, effectively substituting for missing training examples.

The authors remain modest regarding the theoretical status of the spherical construction, noting that while empirical reconstruction suggests completeness, a formal proof remains open. They also acknowledge that at high capacities, the computational overhead of complete invariants may not be justified compared to simpler, incomplete pooling strategies.