💻 computer science

Stability and Geometry of Attractors in Neural Cellular Automata

This paper challenges the assumption that Neural Cellular Automata learn fixed-point attractors by using dynamical systems theory to demonstrate that the growing gecko NCA actually exhibits oscillatory, periodic, and quasi-periodic behaviors with distinct stability characteristics and secondary modes under perturbation.

Original authors: Mia-Katrin Kvalsund, James Stovold

Published 2026-04-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mia-Katrin Kvalsund, James Stovold

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a digital ecosystem where tiny, simple cells talk to their neighbors to build complex shapes, like a digital gecko growing and healing itself. This is a Neural Cellular Automaton (NCA). Think of it like a colony of ants or a school of fish: no single ant knows the whole plan, but together, they create a beautiful, organized structure.

For a long time, scientists assumed that once these digital geckos learned their shape, they would just sit there perfectly still, like a statue. They thought the system had found a "fixed point"—a final, unchanging state.

This paper says: "Not so fast!"

The authors, Mia-Katrin and James, decided to look closer at these digital geckos using tools usually reserved for studying weather patterns and planetary orbits. Here is what they found, explained simply:

1. The Statue is Actually a Dancing Robot

The big surprise? The geckos aren't statues. They are dancers.

Even after the gecko has "learned" its shape and looks perfect, it never actually stops moving. It's like a dancer who has memorized the choreography so well that they can perform the same routine over and over again, forever.

The Discovery: Instead of freezing in place, the cells are constantly shifting in a rhythmic pattern. Some geckos spin in a simple circle (like a waltz), while others do complex, multi-layered dances (like a tango with a partner).
The Metaphor: Imagine a clock. You might think the hands stop when they hit 12:00. But in this digital world, the hands keep spinning around the clock face forever, never quite stopping, even though the time looks the same every second.

2. How They Measured the "Dance"

To prove the geckos were dancing and not just standing still, the authors used two special "microscopes":

The Lyapunov Test (The "Shake Test"): Imagine you have a ball balanced on a hill. If you nudge it, does it roll away (chaos), or does it roll back to the bottom (stable)?
- They nudged the digital gecko slightly. The gecko wobbled but always rolled back to its dance routine. This proved the system is stable (it won't fall apart) but not static (it keeps moving).
The Fourier Spectrum (The "Music Analyzer"): If you record the dance and turn it into sound, what does it sound like?
- A "fixed point" would be silence.
- A simple dance would be a steady drumbeat (one frequency).
- A complex dance is a symphony with many instruments playing together (multiple frequencies).
- The Result: They found that the geckos were playing complex symphonies. Some were simple beats, but others were intricate, multi-layered rhythms that never quite repeated the exact same pattern twice, yet stayed perfectly organized.

3. The "Second Life" Surprise

Here is the most fascinating part. The researchers decided to mess with the geckos. They applied a "shock" or a big perturbation—like throwing a rock at the dancing robot.

The Expectation: They thought the robot would stumble, shake, and then return to its original dance.
The Reality: Sometimes, the robot didn't go back to the original dance. Instead, it found a new dance.
- Imagine a dancer who, after being bumped, suddenly starts doing a completely different routine. It's still a stable, organized dance, but it's not the one they were doing before.
- This suggests the system has multiple "modes" or personalities. If you damage it enough, it doesn't just break; it switches to a backup plan that is just as stable as the first one.

4. Why Does This Matter?

For a long time, people thought these AI systems were learning to be perfect, frozen statues. This paper changes the story.

It's Alive: These systems are more like living organisms than static pictures. They breathe, pulse, and shift.
Robustness: The fact that they can switch to a "secondary dance" when hit is actually a superpower. It means the system is incredibly resilient. If one way of being breaks, it can find another way to stay stable.
The Future: The authors hope this helps other scientists understand that "stability" doesn't mean "stillness." It can mean a constant, rhythmic flow.

The Big Picture Analogy

Think of a school of fish.

Old View: We thought the fish swam until they reached a specific spot and then froze in a perfect formation.
New View: The fish are constantly swirling and turning in a perfect, endless loop. If you throw a net at them, they might not just scatter and reform; they might suddenly decide to swim in a different, equally beautiful pattern.

This paper teaches us that in the world of artificial life, stability is a dance, not a statue.

1. Problem Statement

Neural Cellular Automata (NCAs) are widely used for modeling self-organizing systems, ranging from biological growth to image segmentation. A common assumption in the literature is that trained NCAs converge to fixed-point attractors (stable states where the system stops changing). However, this assumption is often based solely on visual inspection of the system after a limited number of timesteps, lacking rigorous dynamical systems analysis.

Key gaps in current understanding include:

The precise type of attractor learned (fixed point, limit cycle, quasi-periodic, or chaotic).
The stability of these attractors over very long time horizons.
The system's behavior when subjected to perturbations: Does it return to the original state, or does it enter a different dynamic mode?
The lack of tools to visualize and classify NCA attractor dynamics beyond dimensionality reduction.

2. Methodology

The authors conducted a case study using the growing gecko NCA (Mordvintsev et al., 2020), modified to use deterministic cell updates (100% update probability) to simplify the analysis to a deterministic dynamical system. They trained three model versions (V1, V2, V3) with different random seeds.

The analysis employed tools from dynamical systems theory:

Attractor Visualization:
- Used Principal Component Analysis (PCA) to reduce the high-dimensional substrate (40×40×16 channels = 25,600 dimensions) to 2D/3D for visualization.
- Implemented a "burn-in" phase (2,000–4,000 timesteps) to ensure the system reached the attractor before recording data.
Stability Analysis (Lyapunov Spectrum):
- Calculated the Lyapunov exponents ( $\lambda$ ) to measure sensitivity to initial conditions.
- $\lambda < 0$ : Stable (perturbations decay).
- $\lambda = 0$ : Neutral (periodic or quasi-periodic).
- $\lambda > 0$ : Chaotic.
- Estimated the top exponents by evolving two nearby trajectories and measuring the divergence rate.
Frequency Analysis (Fourier Spectrum):
- Computed the Power Spectral Density of the time series to identify dominant frequencies.
- Distinguished between periodic (single base frequency + harmonics), quasi-periodic (multiple incommensurate frequencies), and chaotic (broadband noise) behaviors.
Perturbation Testing:
- Applied small perturbations (Gaussian noise) and large perturbations (circle masks) to states on the attractor.
- Tracked the system's trajectory to determine if it returned to the original attractor or settled into a secondary mode.
Training Dynamics:
- Analyzed the formation of attractors at various training epochs (1 to 8,000) to observe how dynamics evolve during learning.

3. Key Contributions

First Visualizations of NCA Attractors: The paper presents the first explicit visualizations of NCA attractor dynamics, revealing that they are not static fixed points but complex, evolving geometries.
Refutation of Fixed-Point Assumption: The study demonstrates that NCAs trained for "stable" tasks often converge to oscillatory, periodic, or quasi-periodic attractors, challenging the prevailing belief that they learn fixed points.
Discovery of Secondary Attractors: The authors show that large perturbations can push the system into a secondary attractor (a distinct dynamic mode) rather than returning to the original state, suggesting the existence of multiple stable basins of attraction.
Methodological Toolkit: The paper establishes a framework for analyzing NCAs using Lyapunov spectra, Fourier analysis, and PCA, moving beyond empirical observation to rigorous mathematical characterization.

4. Results

A. Attractor Geometry and Type

Non-Fixed Points: None of the three model versions converged to a fixed point. All displayed continuous motion even after long training.
Version 1 (Periodic):
- Lyapunov: $\lambda_1 \approx 0$ , others negative.
- Fourier: One clear dominant frequency ( $f=0.183$ ) with harmonics.
- Conclusion: True periodic behavior (Limit Cycle).
Version 2 (Quasi-Periodic):
- Lyapunov: $\lambda_1, \lambda_2$ near zero (numerically difficult to distinguish from 0), others negative.
- Fourier: 4 dominant independent frequencies.
- Conclusion: Quasi-periodic behavior (motion on a 4-torus).
Version 3 (Quasi-Periodic):
- Lyapunov: $\lambda_1$ slightly negative, others negative.
- Fourier: 3 dominant independent frequencies.
- Conclusion: Quasi-periodic behavior (motion on a 3-torus).
Dimensionality: The attractors are low-dimensional manifolds compared to the full substrate (e.g., V1 requires only 2 PCA components to explain >95% variance), yet they are complex enough to require scaling for visualization.

B. Stability and Perturbations

Lyapunov Stability: All systems are Lyapunov stable (no positive exponents), meaning small perturbations decay.
Large Perturbations: When subjected to large perturbations (circle masks), the systems did not always return to the original attractor.
- Version 1: Settled into a secondary limit cycle with a mean distance of 0.6 (unscaled units) from the original.
- Version 3: Settled into a secondary wavy limit cycle with a mean distance of 1.7 from the original.
- This indicates the existence of multiple attractors within the system's subspace, where the system can "get stuck" in a different stable mode depending on the perturbation.

C. Learning Dynamics

Oscillatory dynamics emerge early in training (by epoch 2,000).
The system transitions from a state that looks like a limit cycle to a weakly dissipative state (slightly negative $\lambda_1$ ) over time, but the oscillation persists. The "fixed point" is never truly reached within reasonable timeframes.

5. Significance and Future Directions

Robustness Implications: The existence of secondary attractors implies that NCAs may be less robust to large structural damage than previously thought. A system might "heal" into a different, potentially non-functional or altered state rather than the original goal.
Explainable AI (XAI): This work provides a necessary toolkit for understanding the "black box" nature of NCAs, moving from visual similarity to mathematical characterization of stability.
Future Research:
- Investigating whether these oscillatory attractors are universal across all NCAs.
- Analyzing Stochastic NCAs (which use random updates), as the current tools are designed for deterministic systems.
- Exploring Strange Non-Chaotic Attractors (SNAs), which combine fractal geometry with non-chaotic dynamics, as a potential classification for the observed dampened quasi-periodic motion.
- Studying the synchronization of local cell dynamics to form global attractors.

In summary, the paper fundamentally shifts the understanding of NCAs from static, fixed-point systems to dynamic, oscillatory systems with complex basins of attraction, providing the mathematical framework to analyze their long-term stability and robustness.