Intrinsic Lorentz Neural Network

The paper proposes the Intrinsic Lorentz Neural Network (ILNN), a fully intrinsic hyperbolic architecture operating entirely within the Lorentz model that introduces novel components like point-to-hyperplane layers and GyroLBN to achieve state-of-the-art performance on image and genomic benchmarks while surpassing both existing hyperbolic and Euclidean baselines.

The Gist

Imagine you are trying to organize a massive library.

In the old way of doing things (using Euclidean geometry, which is like a flat, grid-based city map), you try to fit everything into a giant, flat room. If you have a simple list of books, this works fine. But what if your library has a complex family tree? You have "Animals," then "Mammals," then "Dogs," then "Golden Retrievers," and then specific individual dogs.

On a flat map, to show that "Golden Retrievers" are a tiny subset of "Animals," you have to stretch the map out infinitely. The further down the family tree you go, the more space you need. Eventually, your flat map becomes a mess of stretched-out, distorted shapes. It's like trying to flatten a globe onto a piece of paper; the edges get warped and the distances don't make sense anymore.

Hyperbolic geometry is like a giant, expanding tree or a giant coral reef. In this shape, space naturally expands as you go deeper. You can fit an infinite number of "Golden Retrievers" under the "Dog" branch without stretching the map. It's the perfect shape for hierarchical data (like family trees, word meanings, or DNA).

The Problem with Current "Tree" Computers

Scientists have been building computers that use this "tree" shape (called Hyperbolic Neural Networks) to understand complex data. But there's a catch: most of these computers are hybrids.

Imagine you are trying to navigate a coral reef, but every time you take a step, you are forced to step onto a flat concrete sidewalk for a second, measure your distance, and then jump back into the water.

• The "Flat" Step: This is where current models take data out of the "tree" shape, do some math on a flat surface (Euclidean), and then try to shove it back into the tree.
• The Result: This causes "leakage." The data gets distorted, the math gets messy, and the computer gets confused about the true shape of the world it's trying to understand.

The Solution: ILNN (The "Pure Reef" Navigator)

The paper introduces a new system called ILNN (Intrinsic Lorentz Neural Network). Think of ILNN as a computer that never leaves the coral reef. It does all its thinking, measuring, and decision-making entirely within the curved, tree-like shape of the data.

Here are the three main "tools" ILNN uses to make this work, explained simply:

1. The "Point-to-Hyperplane" Layer (The Compass)

• Old Way: Imagine trying to decide if a book belongs in the "Fiction" section by drawing a straight line on a flat map.
• ILNN Way: Instead of drawing a straight line, ILNN measures the curved distance from a book to a "curved wall" (a hyperplane) inside the tree.
• The Analogy: It's like a sailor measuring the distance to a curved horizon rather than a straight line on a map. Because the measurement respects the curve of the ocean, the decision is much more accurate. This is called the PLFC layer.

2. GyroLBN (The "Stabilizer")

• The Problem: When you have a huge group of data points (a "batch"), they can get scattered. In a flat world, you just average them out. In a curved tree, averaging is tricky because "up" and "down" change depending on where you are.
• The Old Way: Some computers try to find the "average" by taking a million tiny steps (very slow) or by ignoring the curvature (inaccurate).
• ILNN Way: ILNN uses a special math trick called GyroLBN. Imagine a group of people walking on a curved hill. Instead of trying to find a single center point that doesn't exist, ILNN uses a "gyroscopic" method to gently pull everyone toward the center while keeping their relative distances correct.
• The Benefit: It's faster, more stable, and keeps the data organized without breaking the shape of the tree.

3. Log-Radius Concatenation (The "Stitching" Tool)

• The Problem: When you combine two different chunks of data (like stitching two pieces of fabric), the combined piece often gets too big or too small, throwing off the balance of the whole system.
• ILNN Way: ILNN uses a special "stitching" technique that scales the pieces perfectly before joining them. It's like tailoring a suit; it adjusts the size of each sleeve so that when you sew them together, the shoulders don't end up too wide or too narrow. This ensures the "tree" doesn't get distorted as it grows deeper.

Why Does This Matter?

The authors tested ILNN on two very different types of problems:

1. Pictures (CIFAR-10/100): Recognizing cats, dogs, and cars.
2. Genomics (DNA): Understanding the complex family trees of genes and viruses.

The Result: ILNN beat every other model, including the best "flat" computers and the previous "tree" computers.

• It was more accurate (better at recognizing patterns).
• It was faster (didn't waste time jumping back and forth between flat and curved math).
• It was more stable (didn't crash or get confused).

The Bottom Line

The Intrinsic Lorentz Neural Network is like a master navigator who finally stopped trying to navigate a coral reef using a flat paper map. By staying entirely within the natural, curved shape of the data, it can see connections and hierarchies that other computers miss. It's a cleaner, faster, and smarter way to teach AI how to understand the complex, tree-like structures of our world.

Technical Summary

1. Problem Statement

Real-world data often exhibits latent hierarchical structures (e.g., taxonomies in vision, evolutionary trees in genomics) that are naturally represented by hyperbolic geometry due to its exponential volume growth. While Hyperbolic Neural Networks (HNNs) have shown promise, existing architectures suffer from two critical limitations:

• Partial Intrinsic Nature: Many current models (e.g., HCNN, LFC) mix Euclidean operations (like standard matrix multiplication in ambient Minkowski space) with hyperbolic operations. This breaks geometric consistency and fails to fully leverage the manifold's curvature.
• Numerical Instability and Efficiency: Models based on the Poincaré ball suffer from boundary saturation and numerical instability. While the Lorentz model offers better stability, existing Lorentz-based layers (like Lorentz Batch Normalization, LBN) often ignore gyro-variance, or rely on computationally expensive Fréchet means (as in GyroBN) for normalization, leading to slow training.

The paper argues that to fully harness hyperbolic geometry, all network components (parameters, operations, and updates) must be defined intrinsically within the Lorentz model, avoiding any reliance on extrinsic Euclidean parameterizations.

2. Methodology: Intrinsic Lorentz Neural Network (ILNN)

The authors propose ILNN, a fully intrinsic architecture where every operation is defined strictly within the Lorentz hyperboloid model ($L^n_K$). The core components are:

A. Point-to-Hyperplane Lorentz Fully Connected Layer (PLFC)

This is the central innovation replacing traditional affine transformations.

• Concept: Instead of applying a Euclidean linear map $Wx + b$ to Lorentz vectors, PLFC computes the signed hyperbolic distance from an input point to a set of learned Lorentz hyperplanes.
• Mechanism:
  1. It defines a Lorentz hyperplane $H_{z,a}$ parameterized by a distance $a$ and orientation $z$.
  2. It calculates the closed-form Lorentzian distance $d_L(x, H)$ from the input $x$ to these hyperplanes.
  3. The output logits are derived directly from these distances. The output vector $y$ is reconstructed on the hyperboloid such that its signed distances to coordinate hyperplanes match the computed logits.
• Advantage: This ensures the decision function respects the manifold's curvature. Theoretically, it preserves the margin of the classification boundary, whereas previous extrinsic methods (LFC) contract the margin, leading to information loss.

B. GyroGroup Lorentz Batch Normalization (GyroLBN)

To address the trade-off between efficiency and geometric consistency in normalization:

• Problem: Standard LBN re-centers features but ignores variance control under gyro-operations. GyroBN controls variance but relies on iterative Fréchet mean calculations, which are slow.
• Solution: GyroLBN combines the Lorentzian centroid (a closed-form, efficient mean) with gyro-scaling.
  • It performs gyro-centering (using the closed-form centroid) and gyro-scaling (controlling dispersion).
  • It avoids iterative Fréchet solvers, significantly reducing training time while maintaining the geometric shape of the distribution under gyro-operations.

C. Supporting Intrinsic Modules

• Log-Radius Concatenation: When concatenating feature blocks (e.g., in CNNs), the expected radius of the feature vector grows with dimension, skewing subsequent layers. The authors introduce a scaling factor based on the digamma function to align the expected log-radius across concatenated blocks, ensuring stability without learnable parameters.
• Gyro-Additive Bias: A learnable offset added via gyro-addition ($y \leftarrow y \oplus b$) to the PLFC output.
• Lorentz Dropout: A dropout operator that applies a mask to coordinates and immediately projects the result back onto the hyperboloid to satisfy the constraint $\langle x, x \rangle_L = 1/K$, rather than applying dropout in the tangent space (which can perturb the entire point upon re-projection).

3. Key Contributions

1. Fully Intrinsic Architecture: ILNN is the first HNN that eliminates reliance on extrinsic Euclidean operations, ensuring all computations remain within the Lorentz manifold.
2. Novel PLFC Layer: Introduces a point-to-hyperplane formulation that replaces affine logits with closed-form hyperbolic distances, enhancing representational fidelity and margin preservation.
3. Efficient GyroLBN: Proposes a batch normalization layer that unifies gyro-group theory with efficient Lorentz statistics, outperforming both LBN and GyroBN in accuracy and training speed.
4. Comprehensive Validation: Demonstrates state-of-the-art (SOTA) performance across vision (CIFAR-10/100), genomics (TEB, GUE), and graph learning benchmarks.

4. Experimental Results

The authors evaluated ILNN against strong Euclidean baselines and existing hyperbolic models (HCNN, Hybrid Poincaré/Lorentz) on three fronts:

• Image Classification (CIFAR-10/100):

  • ILNN achieved 95.36% on CIFAR-10 and 78.41% on CIFAR-100.
  • It outperformed the Euclidean ResNet-18 baseline and the previous SOTA hyperbolic model (HCNN-Lorentz).
  • Visualizations showed ILNN embeddings form tighter, more compact clusters with clearer decision margins compared to HCNN.

• Genomic Classification (TEB & GUE):

  • On challenging genomic tasks (e.g., pseudogene detection, promoter detection, COVID variant classification), ILNN significantly outperformed both Euclidean CNNs and HCNN.
  • Notably, on the "Covid" task, HCNN collapsed (MCC ~14.8), while ILNN matched/surpassed the Euclidean baseline (MCC ~64.8), demonstrating robustness where other hyperbolic models failed.
  • Improvements of up to +13.0 percentage points over Euclidean baselines were observed on specific TEB tasks.

• Graph Learning:

  • When applied to graph datasets (AIRPORT, CORA, PUBMED) by replacing the linear layer in the Hypformer, ILNN (Hypformer+PLFC) achieved new SOTA results, confirming the efficacy of intrinsic hyperbolic layers for hierarchical graph structures.

• Efficiency:

  • Ablation studies confirmed that GyroLBN reduces wall-clock training time significantly compared to GyroBN (due to avoiding Fréchet iterations) while improving accuracy over LBN.

5. Significance

The paper establishes a new paradigm for Intrinsic Hyperbolic Deep Learning. By rigorously defining all operations within the Lorentz model, ILNN proves that:

1. Geometric Consistency Matters: Mixing Euclidean and hyperbolic operations limits the representational power of HNNs. Fully intrinsic designs yield superior accuracy and stability.
2. Lorentz Model Superiority: The Lorentz model, when paired with intrinsic operations, offers a numerically stable and efficient alternative to the Poincaré ball for deep learning.
3. Practical Impact: The proposed modules (PLFC, GyroLBN) are drop-in replacements that improve performance on real-world hierarchical data (vision, genomics, graphs) without increasing computational cost, making hyperbolic learning more accessible and effective for practical applications.