Technical Summary: Group-Equivariant Poincaré Convolutional Networks

Problem Statement

Deep learning in hyperbolic space has demonstrated superior capabilities for embedding hierarchical visual data with minimal distortion. However, current hyperbolic visual networks, such as the Poincaré ResNet, face two critical inefficiencies:

1. Computational Overhead: Optimizing deep networks on Riemannian manifolds requires expensive operations (e.g., exponential mappings, Fréchet mean estimations) and is constrained by the manifold's rigid boundaries.
2. Lack of Structural Inductive Bias: Standard hyperbolic networks treat spatial transformations (e.g., a 90-degree rotation) of the same object as entirely distinct hierarchical concepts. This forces the optimizer to learn redundant representations for different orientations, leading to inefficient parameter usage, vanishing signals, and poor sample efficiency.

While Group-Equivariant Convolutional Networks (G-CNNs) in Euclidean space solve this by enforcing symmetry priors (e.g., $C_4$ and $D_4$ groups), directly translating these mechanisms to hyperbolic space is non-trivial. Naive application of Euclidean tensor reshaping and concatenation artificially inflates vector magnitudes, pushing features outside the valid Poincaré ball boundary and breaking geometric constraints.

Methodology

The authors propose Equivariant Poincaré ResNets, a framework that integrates discrete symmetry groups ($C_4$ rotations and $D_4$ rotations/reflections) directly into the Poincaré ResNet architecture. To achieve this, they reconstruct three core components to satisfy both group algebra and hyperbolic geometry:

1. Geometrically Safe Tensor Unflattening ($\beta$-Scaling)

In Euclidean networks, feature maps are often flattened and reshaped to separate group channels. In hyperbolic space, naive concatenation violates manifold norms.

• Mechanism: The authors introduce a geometrically safe inverse $\beta$-split.
• Process:
  1. Map the concatenated Poincaré output vector to the tangent space at the origin using the logarithmic map ($\log_c^0$).
  2. Decouple the dimensions in the Euclidean tangent space.
  3. Restore the magnitude of the decoupled vectors using a scaling factor derived from the Beta function ($\beta_n$) to correct for the distortion introduced during the initial concatenation.
  4. Project the restored vectors back to the Poincaré ball using the exponential map ($\exp_c^0$).
• Goal: This ensures that separating spatial and group channels does not violate the expected norms of the manifold or push features beyond the boundary.

2. Poincaré Lifting and Group Convolutions

Standard hyperbolic convolutions cannot handle the routing required for equivariance.

• Lifting Convolution: Instead of applying complex Möbius transformations directly to gyro-vectors, the authors construct symmetric filter banks entirely within the tangent space. A single base filter $w$ is learned, and the required $|G|$ filters are generated via deterministic spatial actions (rotations and flips) applied to $w$.
• Group Convolution: To map features from $G \to G$, the network must route activations to shifted orientation channels when the input transforms.
  • Left-Regular Permutation: The authors encode this routing by pre-permuting the orientation axis of the tangent-space weights according to the inverse left-regular representation of the group.
  • Safe Flattening: Before computing the hyperbolic inner product, the input tensor is flattened using the native $\beta$-concatenation algorithm to prevent numerical instability and NaN propagation.
• Result: A single gradient update to the base filter updates the representation for all spatial orientations simultaneously.

3. Joint-Orientation Midpoint Batch Normalisation

Standard batch normalization computes statistics independently per channel. In an equivariant network, this is catastrophic: if one orientation detects a strong feature and another does not, independent normalization would force them to have identical means and variances, erasing the network's knowledge of the object's true orientation.

• Mechanism: The authors propose Joint-Orientation Poincaré Batch Normalisation.
• Process: The input tensor is reshaped to merge the group dimension with the spatial dimensions. A unified Poincaré midpoint and variance are computed over this expanded volume.
• Goal: All orientation channels are shifted and scaled by the exact same manifold scalar, preserving the relative magnitude differences between distinct orientations and maintaining strict equivariance.

Key Results

The proposed architecture was evaluated on the CIFAR-10 dataset using a Poincaré ResNet-20 backbone, with a strict protocol omitting spatial data augmentation to isolate the architectural inductive bias.

• Classification Performance: The D4-Equivariant Poincaré ResNet achieved 88.77% Top-1 accuracy. This represents an 11.9% absolute improvement over the standard Poincaré baseline (76.87%) and outperforms the standard Euclidean ResNet-20 (78.26%).
• Sample Efficiency: In low-data regimes (1% and 5% of training data), standard Poincaré networks degraded severely (26.92% and 33.79%). The D4-Equivariant model maintained 63.77% accuracy with only 10% of the data, demonstrating vastly superior generalization.
• Convergence: Equivariant models converged to peak accuracy in fewer epochs than standard Poincaré networks, mitigating computational overhead.
• Out-of-Distribution (OOD) Detection: The equivariant models preserved and slightly improved the OOD robustness inherent to hyperbolic geometry (e.g., AUROC of 83.17 on SVHN vs. 80.72 for standard Poincaré).
• Equivariance Verification: The authors measured the Relative Equivariance Error ($\Delta_{eq}$). Standard hyperbolic convolutions failed the test ($\Delta_{eq} \approx 1.0$), while the proposed Lifting and Group convolutions achieved errors bounded by machine precision ($\Delta_{eq} \approx 10^{-7}$), mathematically confirming the preservation of symmetries.

Significance and Claims

The paper claims to present the first framework to embed strict discrete group equivariance ($C_4$ and $D_4$) directly into hyperbolic convolutional architectures.

• Foundational Step: The work establishes a foundational step toward highly efficient, geometrically grounded hyperbolic visual models by successfully decoupling group orientations without violating manifold constraints.
• Optimization Space Reduction: By embedding equivariance, the optimization space is drastically reduced, accelerating convergence and improving sample efficiency.
• Geometric Integrity: The proposed mechanisms (safe $\beta$-scaling, left-regular permutations, and joint normalization) collectively ensure that discrete spatial symmetries are explicitly maintained while respecting the non-Euclidean geometric constraints of the Poincaré ball.

The authors note that while the approach offers significant structural advantages, it inherits the computational overhead of deep hyperbolic networks and is currently limited to discrete symmetry groups ($C_4$, $D_4$), not continuous transformations like $SO(2)$. Future work is suggested to explore continuous groups and scaling to larger datasets.