🤖 machine learning

Steerable Neural ODEs on Homogeneous Spaces

This paper introduces steerable neural ordinary differential equations on homogeneous spaces, which model feature evolution as parallel transport of sections in associated vector bundles to achieve $G$ -equivariant continuous-time dynamics for general vector-valued features.

Original authors: Emma Andersdotter, Daniel Persson, Fredrik Ohlsson

Published 2026-05-13

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Emma Andersdotter, Daniel Persson, Fredrik Ohlsson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Moving Points and Steering Features

Imagine you have a machine learning model that moves data around. In a standard "Neural ODE" (Ordinary Differential Equation), the model acts like a river. It takes a drop of water (a data point) and flows it from a starting location to a destination. The path is determined by the current (a learnable vector field).

The Problem:
In many real-world problems, data isn't just a single drop of water; it's a drop of water carrying a flag or a compass.

If you move a drop of water from New York to London, the water stays water.
But if you move a compass from New York to London, the needle doesn't just move with the drop; it rotates because the direction of "North" changes relative to the ground.

Standard models struggle with this. They can move the point, but they don't know how to naturally "steer" or rotate the attached feature (the compass) as it moves across a curved surface or a complex shape.

The Solution:
The authors introduce Steerable Neural ODEs. Think of this as a river that not only carries the drop of water but also has a built-in mechanism to automatically rotate the compass attached to it, ensuring it always points the right way relative to the terrain.

The Setting: The "Homogeneous Space"

To understand where this happens, imagine a perfectly smooth, uniform ball (like a sphere).

The Ball (M): This is the surface where your data lives. It's a "homogeneous space," meaning every spot on the ball looks exactly the same as every other spot. You can roll the ball, and the pattern doesn't change.
The Group (G): This is the set of all possible ways you can roll or rotate the ball.
The Stabilizer (H): Imagine you pin one spot on the ball. The "stabilizer" is the set of rotations you can do that keep that specific pin in place (like spinning the ball around the pin).

The paper deals with data living on these uniform shapes (like spheres or rotation groups), where the rules of movement are dictated by symmetry.

The Mechanism: Parallel Transport (The "Steering" Wheel)

How does the model know how to rotate the compass? It uses a concept from geometry called Parallel Transport.

The Analogy: The Ant on a Globe
Imagine an ant walking on a globe holding a stick.

The Path: The ant walks along a specific curve (the flow of the Neural ODE).
The Rule: The ant is told to keep the stick pointing in the "same direction" relative to the ground as it walks.
The Result: If the ant walks in a triangle around the North Pole, when it returns to the start, the stick will be pointing in a different direction than when it started. The stick has been "steered" by the curvature of the globe.

In the paper's model:

The Vector Field is the path the ant walks (the flow).
The Connection is the rulebook that tells the ant how to keep the stick "parallel" as it moves.
Steering: The model learns this rulebook (the connection) just like it learns the path. It figures out exactly how to twist and turn the feature vector so it stays consistent with the geometry of the space.

The "Bundle" Concept: The Suitcase and the Clothes

The paper uses heavy math terms like "associated vector bundles." Here is a simple way to visualize it:

Imagine every point on the map (the homogeneous space) has a suitcase attached to it.

Inside the suitcase is a feature vector (like a compass or a molecular structure).
The suitcases are all different sizes and shapes depending on the local rules (the group $H$ ).
When you move from one point to another, you don't just move the suitcase; you have to unpack it, rotate the contents according to the local rules, and repack it to match the new location.

The "Steerable Neural ODE" is the system that learns:

Where to move the suitcase (the flow on the map).
How to rotate the contents inside the suitcase while it moves (the parallel transport).

Why is this "Equivariant"?

"Equivariance" is a fancy word for "consistency."

If you rotate the entire world (the input data) before feeding it to the model, the model should just rotate its output by the same amount.
The paper proves that if the "path" (vector field) and the "rotation rules" (connection) are designed symmetrically, the whole system respects the symmetry of the world. It's like a dance troupe: if the whole group turns left, the individual dancers' arm movements should also turn left in a coordinated way.

What Does the Model Actually Do?

The model is a coupled system of two equations running at the same time:

The Flow Equation: Moves the data point across the surface (like driving a car).
The Steering Equation: Rotates the feature vector attached to that point (like turning the steering wheel to keep the car on the road).

Both the "driving" and the "steering" are learned by the computer. The computer figures out the best path and the best way to rotate the features to solve the problem.

Summary of Claims

The paper claims to have:

Created a new model that moves points and their attached features simultaneously on curved, symmetric shapes.
Proven that if the movement rules and rotation rules are symmetric, the whole model stays symmetric (equivariant).
Shown that this new model includes older models (like standard Neural ODEs and Continuous Normalizing Flows) as special cases where the "steering" part is trivial or non-existent.
Provided a mathematical recipe (using Wang's theorem) to build these models so they automatically respect the symmetries of the shape they are working on.

In short, the paper gives us a way to teach AI how to move data and its "accessories" (features) across complex, curved worlds without breaking the rules of geometry or symmetry.

Technical Summary: Steerable Neural ODEs on Homogeneous Spaces

Problem Statement

Neural Ordinary Differential Equations (NODEs) provide a continuous-time framework for modeling data transformations via learnable vector fields, offering advantages such as reversibility and adaptive computation. While existing research has extended NODEs to Riemannian manifolds and established conditions for global equivariance under group actions, a significant gap remains in handling data that possesses local symmetries.

Specifically, many modern machine learning problems involve data defined on homogeneous spaces $M = G/H$ where features are not merely scalar values but vector- or tensor-valued fields transforming under the local symmetry group $H$ (the stabilizer subgroup). Existing models, including Continuous Normalizing Flows (CNFs) and equivariant NODEs, primarily address the transport of points or scalar densities. They lack a general, geometrically consistent formulation for transporting general feature fields that transform under representations of $H$ while respecting both the global symmetry $G$ and the local symmetry $H$ .

Methodology

The authors propose Steerable Neural ODEs, a geometric extension of NODEs defined on homogeneous spaces $M = G/H$ . The core methodology relies on differential geometry, specifically the theory of principal and associated vector bundles.

1. Geometric Formulation

Feature Fields as Sections: Instead of treating features as simple functions $f: M \to V$ , the model interprets them as sections of an associated vector bundle $E = G \times_\rho V$ , where $V$ is a vector space carrying a representation $\rho$ of the stabilizer subgroup $H$ .
Parallel Transport: The evolution of these feature fields along the trajectory of a point $p \in M$ is governed by parallel transport with respect to a principal connection $\omega$ on the principal bundle $G \to G/H$ .
Coupled System: The model is defined as a coupled system of two Ordinary Differential Equations:
1. Base Flow: A standard manifold NODE on $M$ driven by a learnable vector field $\phi: M \to TM$ .
  $\frac{d\Phi_p(t)}{dt} = \phi(\Phi_p(t)), \quad \Phi_p(0) = p$
2. Steering Equation: An ODE on the Lie group $H$ (or the principal bundle) that determines the rotation/steering of the feature vector. This is derived from the horizontal lift of the base flow using the connection $\omega$ .
  $\frac{dh_p(t)}{dt} = -(R_{h_p(t)})_* \omega(\sigma^* \phi(\Phi_p(t)))$
  where $h_p(t) \in H$ and $\sigma$ is a local section of the principal bundle.

The final output of the Steerable NODE is a diffeomorphism $\Psi: M \times V \to M \times V$ defined by:
$\Psi(p, v) = (\Phi_p(1), \rho(h_p(1))v)$
Here, the feature vector $v$ is transported along the flow $\Phi_p$ and simultaneously "steered" by the action of $\rho(h_p(1))$ .

2. Learnable Components

Both the vector field $\phi$ and the principal connection $\omega$ are treated as learnable quantities. During training, the model learns not only the trajectory of the data points but also the geometric structure (the connection) required to induce the observed transformations of the feature vectors.

Key Contributions and Results

1. Definition and Construction

The paper formally defines Steerable NODEs (Definition 3.8) as a system of non-linear ODEs that simultaneously transport points on a homogeneous space and associated feature vectors via parallel transport. This provides a rigorous geometric foundation for continuous-time equivariant dynamics of vector-valued features.

2. Equivariance Conditions

The authors prove that the Steerable NODE is $G$ -equivariant (respecting the global symmetry of the space) if and only if:

The generating vector field $\phi$ is $G$ -invariant.
The principal connection $\omega$ is $G$ -invariant.
This is established in Theorem 4.6. The proof demonstrates that the invariance of the connection ensures that the parallel transport flow commutes with the global group action.

3. Parametrization via Wang's Theorem

To make these models practical, the authors provide a method to parametrize the space of admissible invariant connections. Using Wang's Theorem (Theorem 4.11), they show that $G$ -invariant connections on $G \to G/H$ are in one-to-one correspondence with linear maps $\Lambda: \mathfrak{g} \to \mathfrak{h}$ satisfying specific equivariance and identity conditions. This allows for the explicit construction of invariant connections for specific groups (e.g., $SO(3)$ acting on $S^2$ ).

4. Relation to Existing Models

The paper situates Steerable NODEs within the broader landscape of geometric deep learning:

Generalization of CNFs: When the stabilizer subgroup $H$ is trivial, the Steerable NODE reduces to standard Continuous Normalizing Flows on Lie groups (as studied in lattice gauge theories).
Extension of Push-forwards: The framework generalizes the push-forward action on tangent vectors (used in previous equivariant NODE work) to general vector bundles.
Incorporation of Local Symmetry: Unlike discrete-time equivariant CNNs which handle local symmetry via induced representations in a discrete layer, Steerable NODEs probe the continuous geometric structure of the bundle via the connection $\omega$ .

Significance and Claims

The paper claims to provide the geometric foundation for learning continuous-time equivariant dynamics of general vector-valued features on homogeneous spaces.

Novel Perspective: The construction introduces a new perspective where learning involves not just the flow of data points, but also the learning of intrinsically geometric structures (connections) that dictate how features transform.
Unified Framework: It unifies the modeling of point transport and feature transport, bridging the gap between scalar density evolution (CNFs) and vector-valued feature dynamics.
Theoretical Rigor: By grounding the model in principal bundles and parallel transport, the authors ensure that the resulting dynamics are naturally consistent with the symmetries of the problem, avoiding ad-hoc constructions.

The authors note that while the global construction is well-defined, its local realization depends on the choice of a section (gauge), and future work may explore gauge-equivariance and extensions to non-homogeneous manifolds or discrete geometric settings like graphs. The paper does not present experimental results but focuses on the theoretical formulation and the derivation of conditions for equivariance.