🤖 machine learning

Equivariant Neural Networks for General Linear Symmetries on Lie Algebras

This paper introduces Reductive Lie Neurons (ReLNs), an exactly GL(n)-equivariant architecture that resolves stability issues for reductive Lie algebras to enable efficient, parameter-light learning on matrix-valued and Lie-algebraic features across diverse scientific domains, outperforming existing baselines in accuracy and computational efficiency.

Original authors: Chankyo Kim, Sicheng Zhao, Minghan Zhu, Tzu-Yuan Lin, Maani Ghaffari

Published 2026-02-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chankyo Kim, Sicheng Zhao, Minghan Zhu, Tzu-Yuan Lin, Maani Ghaffari

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to understand the world. The world is full of shapes, movements, and uncertainties. If you rotate a cube, it's still the same cube. If you speed up a drone, its "uncertainty" about where it is changes in a very specific, mathematical way.

Most current AI models are like students who memorize answers for specific test conditions. If you rotate the test image, they get confused. To fix this, scientists have built "Equivariant Neural Networks"—models that are built to understand rotation and symmetry from the ground up.

However, there's a problem. Most of these smart models are great at handling simple rotations (like spinning a toy), but they struggle when the data gets complicated. They break down when faced with matrices (grids of numbers) that represent things like uncertainty, inertia, or 3D shapes. They also struggle with "non-compact" symmetries, which are like stretching or squeezing space, not just rotating it.

This paper introduces a new solution called Reductive Lie Neurons (ReLNs). Here is how it works, using simple analogies:

1. The Problem: The "Broken Ruler"

Imagine you have a ruler to measure things. For simple rotations, the ruler works perfectly. But for complex matrix data (like a drone's uncertainty map), the standard ruler (mathematically called the "Killing form") is broken or "degenerate." It can't measure certain things accurately, causing the AI to become unstable or linear (dumb).

The authors say: "Let's fix the ruler." They invented a new, non-degenerate bilinear form. Think of this as a custom-made, super-accurate measuring tape that works for both simple rotations and complex matrix stretching. It allows the AI to measure relationships between data points without breaking.

2. The Solution: The "Universal Adapter"

Before this paper, if you had a drone, you needed one AI to understand its speed (a vector) and a totally different AI to understand its uncertainty (a matrix). You had to glue them together clumsily, often losing important geometric information.

ReLNs act as a Universal Adapter.

The Analogy: Imagine you have a screwdriver (speed) and a wrench (uncertainty). Usually, you need two different tools to use them. ReLNs are like a "Swiss Army Knife" that can hold both the screwdriver and the wrench in the same slot, treating them as part of the same family.
How it works: It takes the speed (a vector) and the uncertainty (a matrix) and translates them both into a single, shared language (a Lie Algebra). Once they speak the same language, the AI can process them together naturally, respecting how they transform when the drone turns or speeds up.

3. What They Tested It On

The authors didn't just build it; they put it through the wringer in several real-world scenarios:

The Shape Sorter (Platonic Solids): They tested if the AI could recognize 3D shapes (like dice) even when the camera was rotated wildly. ReLNs got it right almost 100% of the time, while standard models failed miserably.
The Drone Pilot: They simulated a drone flying aggressively. The AI had to guess where the drone was based on noisy speed data and a changing "uncertainty map."
- Result: ReLNs were the best at this. They used the uncertainty data to correct the path, whereas other models got confused when the uncertainty data was added.
The 3D Painter (Gaussian Splatting): This is a new way of rendering 3D scenes using fuzzy, anisotropic "clouds" (Gaussians). These clouds have a center and a shape (uncertainty). ReLNs learned to understand these 3D clouds much better than previous methods, especially when the objects were rotated.
The Double Pendulum: A classic physics problem with a swinging arm. ReLNs learned the physics rules just as well as the current state-of-the-art models but used significantly less computing power (about 11 times fewer calculations per step).

4. The Big Win: Efficiency and Stability

The paper claims that ReLNs are not just accurate; they are efficient.

The Metaphor: Imagine two cars driving the same distance. One is a heavy, gas-guzzling truck (previous models) that needs complex calculations to stay on the road. The other is a sleek electric sports car (ReLNs). It gets the same (or better) mileage but uses a fraction of the energy.
Because ReLNs use a "closed-form" mathematical approach (a direct formula) rather than trying to solve complex puzzles for every single group, they are faster and lighter.

Summary

In short, this paper presents ReLNs, a new type of AI brain that can natively understand complex, matrix-based data (like uncertainty and 3D shapes) just as easily as it understands simple points. It fixes a mathematical "broken ruler" that plagued previous models, allowing it to handle a wider variety of symmetries (rotations, stretches, and squeezes) with higher accuracy and much less computing power. It's a "one-size-fits-all" backbone for geometric deep learning.

Technical Summary: Equivariant Neural Networks for General Linear Symmetries on Lie Algebras

1. Problem Statement

Many scientific and geometric problems exhibit general linear symmetries, yet current equivariant neural network architectures are predominantly designed for compact groups (e.g., $SO(3)$, $E(n)$ ) or simple vector features. This limitation restricts their applicability to matrix-valued data, such as covariances, inertias, or shape tensors, which transform under the non-compact general linear group $GL(n)$.

A central challenge in $GL(n)$-equivariant learning is the non-compactness of the group. Unlike compact groups, $GL(n)$ lacks a canonical positive-definite Ad-invariant inner product. Specifically, the Killing form on the Lie algebra $\mathfrak{gl}(n)$ is degenerate because $\mathfrak{gl}(n)$ is reductive but not semisimple (it contains a non-trivial center). This degeneracy prevents the construction of stable invariant scalars (essential for gating and normalization) and complicates the design of stable nonlinear equivariant operations. Furthermore, real-world geometric inputs are often heterogeneous, combining vectors (transforming via left action) and matrices (transforming via congruence, e.g., $\Sigma \mapsto A\Sigma A^\top$ ). Naively concatenating these features discards tensor structure, while spectral parameterizations can be fragile under differentiation.

2. Methodology: Reductive Lie Neurons (ReLNs)

The authors introduce Reductive Lie Neurons (ReLNs), a general-purpose architecture designed to be exactly equivariant to the adjoint action of $GL(n)$ on its Lie algebra $\mathfrak{gl}(n)$ .

Core Innovation: Non-Degenerate Ad-Invariant Bilinear Form

To resolve the degeneracy of the Killing form on reductive algebras, the authors propose a modified, non-degenerate Ad-invariant bilinear form $\tilde{B}$ . For a reductive Lie algebra $\mathfrak{g} = \mathfrak{z}(\mathfrak{g}) \oplus [\mathfrak{g}, \mathfrak{g}]$ (center plus semisimple ideal), $\tilde{B}$ is constructed by combining the Killing form on the semisimple part with an arbitrary Ad-invariant inner product on the center.
For the specific case of $\mathfrak{gl}(n)$ , this form is given in closed form as:
$\tilde{B}(X, Y) = 2n \text{Tr}(XY) - \text{Tr}(X)\text{Tr}(Y)$
This form is symmetric, Ad-invariant, and non-degenerate, enabling principled nonlinear interactions and invariant feature construction.

Architecture Components

ReLNs operate directly on $n \times n$ matrix representations, unifying vectors and matrices into a shared feature space via Lie-algebraic liftings (e.g., mapping velocity vectors to $\mathfrak{so}(3) \subset \mathfrak{gl}(3)$ and covariances to symmetric matrices). The architecture includes:

ReLN-Linear: A linear map acting on the channel dimension that commutes with the group action on the geometric dimension.
ReLN-ReLU: An equivariant nonlinearity that uses $\tilde{B}$ to compute an invariant scalar gate, applied along an equivariant direction.
ReLN-Bracket: A layer utilizing the Lie bracket (matrix commutator) to create nonlinear interactions between channels, preserving Ad-equivariance.
Max-Killing Pooling & Invariant Layers: Mechanisms for feature aggregation and producing group-invariant outputs using $\tilde{B}$ .

Unified Representation for Heterogeneous Inputs

ReLNs employ a "unified lifting" strategy. Heterogeneous inputs, such as velocity vectors (left action) and covariance matrices (congruence action), are embedded into a single $\mathfrak{gl}(n)$ feature space. Under the isomorphism between congruence and adjoint actions (specifically for orthogonal frames), both types of data transform under the same adjoint action $Ad_g(X) = gXg^{-1}$ . This allows a single backbone to process coupled geometric data without separate specialized architectures.

3. Key Contributions

ReLNs Architecture: A novel, general-purpose architecture for exact $GL(n)$ adjoint equivariance, built upon a non-degenerate Ad-invariant bilinear form that resolves the degeneracy issues of the standard Killing form on reductive algebras.
Unified Lifting: A method to embed heterogeneous geometric inputs (vectors and congruence-transforming matrices) into a shared $\mathfrak{gl}(n)$ feature space, enabling a single adjoint-equivariant backbone.
Empirical Validation: Demonstration of practical utility and efficiency across diverse regimes, including algebraic benchmarks, uncertainty-aware state estimation, and 3D Gaussian Splatting.

4. Experimental Results

The authors evaluated ReLNs on five distinct tasks:

Algebraic Benchmarks ( $\mathfrak{sl}(3)$ and $\mathfrak{sp}(4)$ ):
- Platonic Solid Classification: ReLNs achieved near-perfect accuracy (99.78%) on rotated camera views, matching Lie Neurons while demonstrating robustness where non-equivariant MLPs failed (36.54%).
- Invariant Function Regression: On $\mathfrak{sp}(4)$ , ReLNs achieved near-zero invariance error and low test MSE, comparable to specialized Lie Neurons but with a unified construction.
Drone State Estimation:
- In a task reconstructing 3D trajectories from noisy velocity and covariance data, ReLNs significantly outperformed non-equivariant ResNets and other equivariant baselines (Vector Neurons, TFN, SE(3)-Transformer).
- The best configuration (using $(v, \log C)$ ) achieved an Absolute Trajectory Error (ATE) of 13.92m, compared to 191.78m for Vector Neurons with covariance inputs.
- Ablation studies confirmed that modeling the full reductive structure (center + semisimple ideal) is superior to using only the semisimple part.
3D Gaussian Splatting (3DGS):
- Integrated into a Gaussian-MAE framework, ReLNs maintained 95.15% classification accuracy on ModelNet10 under random rotations, whereas the baseline dropped to 18.28%.
- ReLNs converged faster during pre-training and achieved lower reconstruction errors across rotation, scale, and density attributes.
EMLP Double-Pendulum Benchmark:
- ReLNs matched or improved upon EMLP rollout errors across multiple symmetry groups ( $O(2)$ , $SO(2)$, $D_6$ ).
- Efficiency: ReLNs reduced per-step FLOPs by approximately 11.18 $\times$ and inference latency by 27.81 $\times$ compared to EMLP, due to the use of closed-form matrix operations rather than group-specific basis construction.
Lorentz-Equivariant Top Tagging:
- ReLNs matched the performance of parameter-matched LorentzNet baselines in jet tagging tasks, demonstrating applicability to non-compact Lorentz groups via specific embeddings.

5. Significance and Claims

The paper claims that ReLNs provide a practical, reusable backbone for learning with broad linear symmetries. By resolving the stability issues associated with the Killing form on reductive algebras, ReLNs enable the principled processing of matrix-valued data (like uncertainties and inertias) that was previously difficult to handle in a symmetry-consistent manner.

The authors emphasize that ReLNs improve the accuracy–efficiency trade-off, consistently matching or outperforming strong baselines while using substantially fewer parameters and compute. The framework is presented as a solution for applications requiring the joint processing of heterogeneous geometric attributes (e.g., velocity and covariance in robotics, or mean and covariance in 3D Gaussian Splatting) under general linear symmetries. The work suggests that exact equivariance on reductive Lie algebras is a viable and efficient approach for structured 3D scene models and physical system modeling.