🔢 mathematics

1-Lipschitz Neural Networks on Hadamard Manifolds

This paper introduces a class of 1-Lipschitz neural networks on Hadamard manifolds built upon Busemann functions and gradient flows, demonstrating their effectiveness in robust hyperbolic classification and masked-Wishart covariance reconstruction through geometry-preserving layers.

Original authors: Davide Murari, Marta Ghirardelli, Ben Adcock, Elena Celledoni, Brynjulf Owren, Carola-Bibiane Schönlieb

Published 2026-07-22

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Davide Murari, Marta Ghirardelli, Ben Adcock, Elena Celledoni, Brynjulf Owren, Carola-Bibiane Schönlieb

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to understand the world, but the world isn't flat like a sheet of paper. Instead, it's curved, like the surface of a sphere or a saddle. In the world of artificial intelligence, most robots are trained on flat, Euclidean ground. They are great at drawing straight lines and measuring distances on a grid. But many real-world problems—like understanding how data grows in a tree-like structure or how materials stretch and twist—live on these curved surfaces. This is the realm of "geometric deep learning," where scientists try to build AI that respects the natural curves of the data.

However, there is a catch. When you teach a robot to navigate a curved world, you have to make sure it doesn't go haywire. If you nudge the robot's input just a tiny bit, you don't want its output to jump wildly to a completely different place. In math, this stability is called being "1-Lipschitz." Think of it like a speed limit for the robot's brain: no matter how much you push it, it can't move faster than a certain speed. While scientists have figured out how to put speed limits on robots in flat worlds, doing the same on curved surfaces has been a tricky puzzle. This paper is about solving that puzzle, creating a new kind of AI layer that stays calm and stable even when the ground beneath it is curving.

The Curved Road and the Speed Limit

Imagine you are hiking on a mountain trail. In a flat city, if you walk one step forward, you move exactly one step forward. But on a mountain, the ground curves. If you take a step, the path might bend away from you, or two hikers starting side-by-side might end up walking in different directions without ever getting closer. This is what happens in "Hadamard manifolds," a fancy math term for spaces that are curved but don't loop back on themselves like a sphere.

The authors of this paper wanted to build a neural network (a type of AI brain) that lives on these curved trails. But they had a specific goal: they wanted the network to be "1-Lipschitz." In everyday language, this means the network is a "good neighbor." If you change the input slightly, the output changes only slightly. It never overreacts. This is crucial for safety. If you are using AI to diagnose a disease or control a robot, you don't want a tiny error in the data to cause a massive, dangerous mistake.

The problem is that most AI layers are designed for flat ground. When you try to drag them onto a curved mountain, they often lose their "good neighbor" behavior. They might start overreacting, making the whole system unstable. The authors asked: Can we build AI layers that are naturally stable on curved surfaces, just like they are on flat ones?

The Magic of the "Busemann" Compass

To solve this, the authors turned to a very specific type of mathematical tool called a Busemann function. If you imagine the curved mountain stretching out forever toward the horizon, a Busemann function is like a compass that points toward a specific point on that distant horizon. It measures how "far" you are from that horizon in a very special way.

The authors discovered that if you build their AI layers using these Busemann compasses, the layers naturally behave like a "gradient descent." Imagine a ball rolling down a hill. In math, "gradient descent" is just the process of rolling down the steepest slope to find the bottom. The authors showed that if you roll a ball down a hill defined by a Busemann function, the ball moves in a very predictable, controlled way. It never speeds up too much or jumps around.

They proved that by using these Busemann-based steps, they could create AI layers that are 1-Lipschitz. This means the layers have a built-in speed limit. No matter how the data curves, the network's response stays within a safe, predictable range. They also showed that these layers are "quasi-α-firmly nonexpansive," which is a fancy way of saying that if you keep applying these layers over and over, the system is guaranteed to settle down to a stable answer rather than spinning out of control.

Putting the Theory to the Test

The authors didn't just stop at the math; they built these networks and tested them in two very different scenarios to see if they actually worked.

1. The Hyperbolic Classification Game
First, they tested their network on the "Poincaré disk," which is a way of representing a hyperbolic (saddle-shaped) space. They created a game where the AI had to sort points into different groups. They pitted their new "Busemann network" against two other types:

A standard, unconstrained network (the "wild" one).
A network that only moved points around using perfect rotations (the "rigid" one).

The results were fascinating. The "wild" network was very good at sorting the points correctly when everything was perfect. But as soon as the authors added "noise" (tiny, tricky perturbations meant to confuse the AI), the wild network fell apart. It became unstable. The rigid network was stable but couldn't sort the points well enough. The Busemann network, however, was the Goldilocks solution. It was stable enough to handle the noise without panicking, but flexible enough to sort the points accurately. It proved that adding this "speed limit" makes the AI more robust against attacks.

2. The Mystery Covariance Puzzle
The second test was more like a detective story. The goal was to reconstruct a missing piece of a complex puzzle: a "covariance matrix" (a table of numbers that describes how different things are related). In this case, the data came from a "masked-Wishart" problem, meaning they only saw parts of the picture and had to guess the rest.

They used their Busemann network as a "denoiser"—a tool to clean up the messy, incomplete data. They compared it to a standard method called "Log-Euclidean" denoising. The results showed that their Busemann network was better at reconstructing the missing parts of the matrix. It didn't just fit the data it saw; it used its stable, geometric understanding to guess the hidden parts more accurately. The authors found that their method reduced the error significantly compared to the other methods, suggesting that the "speed limit" helps the AI make smarter guesses about the unseen world.

What This Means

The paper doesn't claim to have solved every problem in AI. It doesn't say this is the only way to build stable networks, nor does it claim these networks are perfect in every single situation. Instead, it offers a new, reliable tool. It shows that by using Busemann functions, we can build neural networks that respect the geometry of curved spaces while staying calm and stable.

The authors suggest that this approach opens the door for more robust AI in fields where data is naturally curved, like medical imaging (where brain scans are often analyzed as curved surfaces) or material science. By proving that these layers are mathematically stable and showing they work in simulations, they have laid a foundation for building AI that is not just smart, but also safe and reliable, even when the ground beneath it is curving.

Title: 1-Lipschitz Neural Networks on Hadamard Manifolds

Problem Statement
Controlling the Lipschitz constant of neural networks is a standard mechanism for ensuring robustness and stability, particularly in adversarial classification, Wasserstein-based generative modeling, and learned iterative methods for inverse problems. While extensive literature exists for constructing 1-Lipschitz networks in Euclidean spaces (using spectral normalization, orthogonality constraints, or gradient-type residual blocks), many learning problems are naturally posed on curved manifolds. Data in fields such as directional statistics, rotation-valued imaging, and covariance-based vision (e.g., diffusion tensor imaging) often reside on Riemannian manifolds, specifically Hadamard manifolds (complete, simply connected, nonpositively curved spaces). Existing manifold-valued architectures focus on geometric expressivity but lack a systematic theory for stability and convergence. There is a gap between the geometric deep learning literature, which builds intrinsic layers, and the Euclidean stability literature, which relies on Lipschitz and averaged operator theory. This paper addresses the design and analysis of stable, 1-Lipschitz neural networks on Hadamard manifolds.

Methodology
The authors propose a class of neural network layers based on Riemannian gradient descent steps driven by specific potentials. The core theoretical framework relies on the properties of Busemann functions on Hadamard manifolds.

Theoretical Foundation:
- The authors utilize the fact that Busemann functions are geodesically convex, 1-Lipschitz, and $C^2$ on Hadamard manifolds.
- They define Busemann gradient-descent layers of the form $T_\tau(x) = \exp_x(-\tau \nabla V(x))$ , where the potential $V$ is constructed from a Busemann function $b$ via a scalar nonlinearity: $V(x) = \phi(\lambda b(x) + \beta)$ .
- The paper establishes that if the step size $\tau$ and the function $\phi$ satisfy specific conditions (specifically, $\phi$ is convex, non-decreasing, and its derivative is Lipschitz with a bound related to $\tau$ ), the resulting map is nonexpansive (1-Lipschitz).
- Furthermore, under these conditions, the layers are shown to be quasi- $\alpha$ -firmly nonexpansive. This property is crucial as it guarantees the convergence of fixed-point iterations (e.g., $x_{k+1} = T(x_k)$ ) in Hadamard spaces, a requirement for learned solvers in inverse problems.
- The theory is extended to product manifolds ( $M^m$ ), allowing the application of these layers to vector-valued data on manifolds (e.g., images or graphs where each pixel/node lies on a manifold).
Implementation:
- Explicit constructions are provided for two specific Hadamard manifolds:
  - Hyperbolic Space (Poincaré Ball): Using the explicit formula for Busemann functions on the Poincaré disk.
  - Symmetric Positive Definite (SPD) Matrices: Utilizing Cholesky-based expressions for Busemann functions on the manifold of SPD matrices equipped with the affine-invariant metric.
- Admissible activation functions $\phi$ include Softplus and Squared ReLU, with derived step-size constraints to ensure nonexpansiveness.

Key Contributions

Theoretical Bridge: The paper establishes a rigorous bridge between Euclidean 1-Lipschitz gradient layers and intrinsic gradient steps on Hadamard manifolds.
Busemann Layers: It proposes and proves the nonexpansiveness of "Busemann gradient-descent layers" under explicit step-size restrictions. These layers are shown to be quasi- $\alpha$ -firmly nonexpansive, linking them to convergent fixed-point dynamics.
Constructive Examples: The authors provide implementable, closed-form expressions for these layers on hyperbolic manifolds and the SPD manifold, making the theory practically applicable.
Product Manifold Extension: The framework is generalized to product manifolds, enabling applications to structured data like images or graphs on manifolds.

Experimental Results
The proposed architecture is validated through two numerical experiments:

Robust Classification on the Poincaré Disk ( $D^2$ ):
- Task: Classifying synthetic data on the Poincaré disk under adversarial perturbations.
- Comparison: The constrained Busemann model is compared against an unconstrained hyperbolic ResNet and a constrained isometric model.
- Findings: The Busemann model achieves clean accuracy comparable to the unconstrained ResNet but demonstrates significantly higher robustness against hyperbolic perturbations. While the isometric model was insufficient for complex boundaries, the Busemann layers provided the necessary geometric expressivity while maintaining the nonexpansive structure.
Masked-Wishart Covariance Reconstruction on SPD(10):
- Task: Reconstructing an unknown covariance matrix from noisy, masked principal-block observations (an inverse problem) using a Plug-and-Play (PnP) scheme.
- Comparison: A constrained "split Busemann" denoiser is compared against a Log-Euclidean residual denoiser and a data-only gradient descent baseline.
- Findings: The split Busemann denoiser, when used as a PnP prior, significantly outperformed both the data-only baseline and the Log-Euclidean denoiser in terms of affine-invariant distance to the ground truth. The results suggest that the nonexpansive constraint stabilizes the reconstruction process, allowing the learned prior to effectively recover global covariance structure even from underdetermined observations. Empirical tests on the denoiser-only iteration confirmed the expected convergence behavior of averaged nonexpansive maps.

Significance and Claims
The paper claims to fill a critical gap in geometric deep learning by providing a systematic stability theory for manifold-valued networks. By transporting Euclidean stability constructions (specifically gradient-flow-based nonexpansive operators) to the Hadamard setting, the authors demonstrate that:

Stability can be encoded structurally in manifold-valued layers, not just checked a posteriori.
Busemann functions serve as a natural and computationally efficient building block for these stable layers.
The resulting networks are not only theoretically grounded in fixed-point convergence theory (via quasi- $\alpha$ -firm nonexpansiveness) but also empirically effective in both robust classification and inverse problems.

The authors note that while the current work focuses on Hadamard manifolds where Busemann functions are tractable, the methodology opens the door for future extensions to broader classes of Riemannian manifolds and applications in fields like diffusion tensor imaging. The paper does not claim to solve all stability issues in non-Hadamard spaces but establishes a principled framework for the nonpositively curved case.

The Curved Road and the Speed Limit

The Magic of the "Busemann" Compass

Putting the Theory to the Test

What This Means

More like this