🔬 condensed matter

Approximating velocity fields with planted attractors via Neural-ODEs for classification purposes

This paper proposes a classification method using Neural ODEs that leverages planted equilibrium points as class-specific attractors, where the learned velocity field guides input trajectories from initial conditions into the corresponding basins of attraction for accurate categorization.

Original authors: Feliciano Giuseppe Pacifico, Duccio Fanelli, Lorenzo Buffoni, Lorenzo Chicchi, Diego Febbe, Raffaele Marino

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Feliciano Giuseppe Pacifico, Duccio Fanelli, Lorenzo Buffoni, Lorenzo Chicchi, Diego Febbe, Raffaele Marino

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a giant, invisible landscape made of hills and valleys. In the world of machine learning, this landscape is where data lives. Usually, when a computer tries to sort things (like telling a cat from a dog), it draws a static line or a complex maze to separate them.

This paper proposes a different way to do sorting. Instead of drawing a static map, the authors build a dynamic river system.

Here is the simple breakdown of how it works, using the paper's own logic:

1. The Goal: Sorting by "Sinking"

Imagine you drop a pebble (your data, like a picture of a shoe) into a river. You want that pebble to naturally flow and settle into a specific pool at the bottom of the river.

The Pools (Attractors): The researchers decide ahead of time exactly where these pools will be. They "plant" a pool for every category they want to sort. If you have 10 types of clothing, they plant 10 specific pools.
The Flow (Velocity Field): The computer learns to shape the riverbed (the velocity field) so that no matter where you drop the pebble, the current pushes it toward the correct pool.

2. The Problem: "Rigid" vs. "Flexible"

In the past, scientists tried to build these river systems by strictly defining the rules of the water flow.

The Old Way: It was like building a canal with concrete walls. It was very clear how the water moved (interpretable), but if the water needed to take a weird turn to reach a pool, the concrete walls stopped it. The system wasn't flexible enough to learn complex patterns.
The Other Extreme: Modern AI (Neural ODEs) is like a magical, shape-shifting river. It can flow anywhere and learn almost any pattern (universal approximation), but it doesn't guarantee that the water will actually stop in the right pools. It might just keep flowing forever or get lost.

3. The Solution: "Planted Attractors"

The authors found a clever trick to get the best of both worlds. They used a mathematical technique (based on something called "QR decomposition") to force the river to have specific pools where the water stops moving, while keeping the rest of the river flexible enough to learn any shape.

Think of it like this:

You tell the river, "No matter what, the water must stop at these 10 specific spots."
The computer then figures out how to bend the riverbed to make that happen, without breaking the rule.
Because the "stopping points" are hard-coded into the math, the computer can't accidentally "forget" them while it's learning. It's like having a safety rail that keeps the river on track.

4. How It Works in Practice

The researchers tested this on two things:

Spiral Shapes: They took two spirals of dots mixed together. The system learned to create a flow that swept the dots from one spiral into one pool and the other spiral into a different pool. You can actually see the paths the dots take on a graph.
Real Images (Fashion MNIST & CIFAR-10): They tried to sort pictures of clothes and objects.
- The Result: The system worked just as well as a standard, top-of-the-line AI classifier. It didn't win by being faster or more accurate, but it proved that a "river system" can do the same job as a "static map."
- The Bonus: Because the system is a river, you can watch the process. You can see the data point moving through time until it settles. If it settles near the "Shoe" pool, the computer knows it's a shoe.

5. The "Two-Population" Secret (The Deep Dive)

The paper also mentions a more complex version of this idea involving two interacting groups of variables (like two schools of fish swimming together).

One group moves slowly (the main river).
The other group moves very fast (a quick adjustment mechanism).
When the fast group moves infinitely fast, the whole system behaves exactly like the simpler "planted pool" model.
Why this matters: This gives scientists a new way to look at the math, interpreting the connections between data points as interactions between two different "species" of variables. It makes the "black box" of AI a little more transparent.

The Bottom Line

The paper claims that by "planting" specific stopping points (attractors) into a flexible, learning river (Neural ODE), they created a classifier that is:

Universal: It can learn to sort almost anything.
Interpretable: You can watch the data flow and see exactly why it made a decision (it flowed to that pool).
Stable: The rules of the game (the pools) never change, even while the computer is learning.

They didn't claim this is faster than current methods (it's actually slightly slower due to the math of simulating the flow), but they proved that a "dynamic" approach can be just as powerful as the "static" approaches we use today, offering a new, clearer way to understand how AI makes decisions.

Technical Summary: Approximating Velocity Fields with Planted Attractors via Neural-ODEs for Classification

Problem Statement

The paper addresses a theoretical and practical gap in the field of dynamical classifiers. Existing approaches generally fall into two categories:

Dynamical Classifiers with Planted Attractors: These models encode decision rules directly into the phase portrait of a dynamical system, where classes correspond to stable equilibria (attractors). While highly interpretable, these frameworks (e.g., Hopfield networks, Wilson-Cowan models) often rely on specific closed-form interaction terms and lack guarantees of universal approximation. Their expressive power for arbitrary classification tasks remains unclear.
Neural Ordinary Differential Equations (Neural-ODEs): These models leverage the universal approximation capabilities of neural networks to parameterize vector fields. They are highly flexible but typically require an external readout head (e.g., a linear classifier or MLP) to perform classification because the attractors are not a priori enforced. Consequently, they lack the inherent interpretability of dynamical classifiers where the label is the destination equilibrium.

The core problem is to unify these approaches: to construct a Neural-ODE that possesses universal approximation properties while strictly enforcing a finite set of prescribed fixed points (attractors) that serve as class prototypes, thereby eliminating the need for an external readout head.

Methodology

The authors propose a framework termed ConsNODEs (Constrained Neural-ODEs), which integrates a curated collection of equilibrium points directly into the velocity field of a Neural-ODE.

1. Model Architecture:
The system is defined by the residual Neural-ODE:
$\dot{x}(t) = F_\theta(x(t)) = -x(t) + A_1 f(A_2 x(t) + b_2) + b_1$
where $x(t) \in \mathbb{R}^n$ is the state vector, $f$ is a component-wise nonlinearity (e.g., sigmoid), and the parameters include weight matrices $A_1, A_2$ and biases $b_1, b_2$ . The linear term $-x(t)$ provides stabilization, ensuring asymptotic attraction to equilibria.

2. Planted Attractor Constraints:
For $C$ target classes, the authors prescribe $C$ distinct equilibrium points $\{\bar{x}^{(\ell)}\}_{\ell=1}^C$ . The model must satisfy the hard constraint $F_\theta(\bar{x}^{(\ell)}) = 0$ for all $\ell$ .
To enforce this without restricting the model's expressivity, the authors utilize a QR-based projection method (adapted from [12]):

The constraint $F_\theta(\bar{x}^{(\ell)}) = 0$ is reduced to a linear system $A_1 S = Y$ , where $S$ and $Y$ are matrices derived from the target points and network parameters.
The weight matrix $A_1$ is parameterized as:
$A_1 = A_{part} + W(I - P)$
where $A_{part}$ is a particular solution ensuring the constraints are met, $P$ is an orthogonal projector onto the subspace spanned by the planted features, and $W$ is a matrix of free, trainable parameters.
This parameterization guarantees that for any value of $W$ , the fixed-point constraints hold exactly. Thus, gradient-based training is structurally confined to the hypothesis class of vector fields with the prescribed equilibria.

3. Classification Protocol:

Training: Inputs are mapped to initial conditions $x(0)$ . The system evolves to a terminal time $T$ . The loss function is a regression loss minimizing the distance between the terminal state $x(T)$ and the corresponding class equilibrium $\bar{x}^{(\ell)}$ .
Inference: Classification is performed by identifying the nearest planted equilibrium to the terminal state $x(T)$ . No external classifier head is used.

4. Theoretical Foundation:
The paper relies on Theorem 1, which establishes that the constrained network class is dense in the space of continuous vector fields that vanish at the prescribed points. This proves that the modified Neural-ODE retains universal approximation capabilities despite the hard constraints.

5. Multi-Population Interpretation:
The authors introduce a coupled two-population system (2PoP Model) involving variables $(x, u)$ with distinct time scales controlled by a parameter $\epsilon$ . In the singular limit $\epsilon \to 0$ , this system reduces exactly to the ConsNODEs formulation. This provides a dynamical systems interpretation where $A_1$ and $A_2$ represent weighted adjacency matrices governing interactions between visible and hidden populations.

Key Contributions

Universal Dynamical Classifiers: The paper demonstrates that it is possible to construct a Neural-ODE with proven universal approximation properties that simultaneously enforces a finite set of stable fixed points as class prototypes.
Constraint-Preserving Training: The proposed QR-based parameterization allows for gradient-based optimization where the existence of attractors is preserved by construction, removing the need for auxiliary loss penalties or post-hoc corrections.
Interpretability: The decision-making process is transparent: inputs are driven by a learned vector field toward specific equilibria. The basins of attraction can be analyzed using standard dynamical systems tools (e.g., phase portraits, Lyapunov exponents).
Two-Population Equivalence: The work establishes a link between the residual Neural-ODE and a multi-species dynamical system, offering a neuroscientific interpretation of the weights as interaction matrices between populations.

Experimental Results

The authors evaluated the framework on three datasets, comparing ConsNODEs against parameter-matched static Multi-Layer Perceptrons (MLPs):

Two Interleaving Spirals (Toy Dataset): In this 2D setting, the model successfully learned a vector field that routed trajectories from two non-linearly separable spirals into distinct basins of attraction corresponding to the planted equilibria. The decision boundary was visualized as the separatrix of the flow.
Fashion-MNIST: Using raw pixel inputs ( $n=784$ $n = 784$ ) without convolutional encoders, ConsNODEs achieved 88.96% test accuracy, comparable to the static MLP baseline (89.32%).
- Observation: Accuracy saturated rapidly during integration. The authors noted that inference time could be reduced by stopping integration early ( $t_{best} < T$ ) without significant accuracy loss, mitigating the computational overhead of ODE integration.
CIFAR-10: Using a shared CNN feature extractor, ConsNODEs achieved 84.39% accuracy versus 84.69% for the MLP.
- Computational Cost: The dynamical approach incurred higher computational costs due to ODE integration and QR operations (approx. 21% higher FLOPs and 1.8x longer training time per epoch). However, the accuracy difference was within one standard deviation.

Significance and Claims

The paper claims that its primary contribution is conceptual rather than purely computational. The significance lies in demonstrating that a classifier can be built as a structured dynamical system with hard-planted equilibria while retaining the universal approximation guarantees of Neural-ODEs.

Interpretability vs. Performance: The authors argue that the proposed method trades a moderate computational overhead for a qualitatively different, more interpretable classification mechanism. Unlike static feedforward maps, the decision process is a continuous-time dynamical evolution toward class-dependent equilibria.
Theoretical Bridge: The work bridges the gap between "transparent" dynamical classifiers (which lacked universality) and "universal" Neural-ODEs (which lacked inherent interpretability).
Future Directions: The paper suggests that the two-population interpretation (2PoP Model) may offer a way to overcome topological constraints (e.g., trajectory crossing) inherent in lower-dimensional ODE classifiers, though this is noted as a topic for future investigation.

The authors remain modest regarding performance, acknowledging that ConsNODEs does not outperform static MLPs in raw accuracy or efficiency on standard benchmarks. Instead, the value proposition is the ability to perform classification via a transparent dynamical process that is theoretically guaranteed to be expressive enough to approximate any velocity field compatible with the prescribed attractors.