Mass Conservation as an Inductive Bias for Self-Organized Criticality in NCA Reservoirs

Technical Summary: Mass Conservation as an Inductive Bias for Self-Organized Criticality in NCA Reservoirs

Problem Statement

Reservoir computing (RC) leverages the intrinsic dynamics of fixed, high-dimensional systems to map inputs into feature spaces solvable by simple linear readouts. A promising theoretical foundation for RC is Self-Organized Criticality (SOC), a dynamical regime characterized by scale-free avalanche dynamics and power-law statistics, often associated with the "edge-of-chaos" hypothesis where computational capacity is maximized. While recent work has demonstrated that Neural Cellular Automata (NCA) can be evolved toward critical avalanche dynamics to serve as effective RC substrates, achieving robust and consistent criticality remains a challenge. This paper investigates whether mass conservation—a local redistribution rule that preserves total lattice mass—can serve as a reliable inductive bias to promote SOC in evolved NCA reservoirs without compromising their utility in downstream tasks.

Methodology

The authors propose a modified NCA architecture based on the work of Pontes-Filho et al. (2025), integrating a mass-conservation mechanism inspired by the MaCE framework (Papadopoulos and Guichard, 2025).

1. NCA Architecture: The system is a one-dimensional, binary, deterministic CA with a grid width $W=1000$ and $C=5$ channels (1 visible, 4 hidden). State updates are governed by a lightweight 3-layer Convolutional Neural Network (CNN) with 1,565 parameters.
2. Mass-Conservation Mechanism: Unlike standard NCAs, the "conserving" variant applies a local donate-then-collect redistribution rule to the visible channel only after the CNN update but before binarization.
   • Each cell $j$ donates mass to its neighbors ($i \in N(j)$) proportionally to a softmax-weighted affinity based on the raw visible values.
   • Each cell $i$ collects donations from neighbors whose neighborhood includes $i$.
   • This ensures the total visible activation $\sum s_i$ is exactly preserved at every timestep. The four hidden channels remain unconstrained to allow for arbitrary internal representations.
3. Evolutionary Training: Both baseline (non-conserving) and conserving variants are evolved using CMA-ES (Covariance Matrix Adaptation Evolution Strategy) over 500 generations.
   • Fitness Function: A composite score rewarding power-law behavior in the visible channel's avalanche statistics. It combines the coefficient of determination ($R^2$), Kolmogorov–Smirnov (KS) distance, bin occupancy, state uniqueness, and log-likelihood ratios comparing power-law vs. exponential models.
   • Experimental Setup: Three independent random seeds (42, 43, 44) were used for each variant.
4. Evaluation Metrics:
   • Criticality: Assessed via Goodness-of-Fit (GOF) tests on six avalanche distributions (size, duration, and inter-arrival time for both 0-state and 1-state). A distribution passes if $p \ge 0.1$. "Perfect criticality" is defined as all six distributions passing.
   • Downstream Tasks: Performance was evaluated on three benchmarks using a fixed NCA reservoir and a trained linear readout:
     • 5-bit Sequential Memory: Retention of a 5-bit pattern after a 200-step distractor.
     • MNIST Classification: Image classification using a LinearSVC readout.
     • CartPole-v1 Temporal Control: A reinforcement learning task using a linear Q-network.

Key Contributions

1. Inductive Bias for SOC: The study demonstrates that mass conservation acts as a reliable inductive bias, significantly increasing the likelihood of evolving NCA reservoirs that exhibit perfect criticality (6/6 GOF passes).
2. Training Efficiency: The conserving variant achieved a 7.3% higher mean fitness and converged 1.27× faster in wall-clock time compared to the baseline.
3. Preservation of Utility: The introduction of conservation constraints did not degrade downstream performance. Both variants achieved comparable results on memory and classification tasks, with the conserving variant showing superior performance in temporal control after checkpoint selection.
4. Criticality-Computation Link: The seed with the strongest criticality (conserving seed 44) also achieved the highest temporal control score, suggesting a positive correlation between SOC quality and sequential computation capabilities.

Results

• Criticality Characterization:
  • Conserving Variants: Seeds 43 and 44 achieved perfect criticality (6/6 GOF passes). Seed 42 achieved 3/6.
  • Baseline Variants: No baseline seed achieved perfect criticality. The best baseline (seed 44) achieved 5/6 passes; seeds 42 and 43 achieved 3/6.
  • Avalanche Statistics: The conserving seeds exhibited power-law scaling across all distributions. Notably, the 1-state duration distribution for seed 44 had a steep exponent ($\hat{\alpha} \approx 10.3$), passing the GOF test trivially due to low variance, but still contributing to the perfect score.
• Downstream Performance:
  • 5-bit Memory: Both variants achieved perfect scores (1.0) across all seeds.
  • MNIST: Both variants outperformed the raw-pixel baseline (~91.1%). The baseline led by a marginal average of +0.27 percentage points (93.81% vs. 93.54%), a difference within seed-to-seed variance. However, conserving features were 2–6× faster to classify, suggesting better linear separability.
  • CartPole-v1: The conserving variant achieved a higher average reward (120.6 vs. 88.6). Crucially, seed 44 (conserving, perfect criticality) achieved the highest individual score (163.4).
• Checkpoint Alignment: A notable asymmetry was observed in checkpoint selection. For the conserving variant, the final checkpoint (gen 499) simultaneously maximized fitness and criticality. For the baseline, checkpoints with the best avalanche statistics did not coincide with those yielding the best downstream performance, indicating that conservation aligns SOC quality with functional utility.

Significance and Claims

The paper claims that mass conservation is a simple, effective, and zero-parameter mechanism for promoting robust criticality in evolved NCA reservoirs. By preventing mass explosion or extinction, conservation regularizes the dynamics, narrowing the fitness landscape around fully critical attractors and making them easier for evolutionary algorithms to find.

The authors assert that this approach successfully decouples the pursuit of criticality from the sacrifice of downstream utility. The results provide tentative empirical support for the edge-of-chaos hypothesis in the context of reservoir computing, suggesting that systems operating near criticality (facilitated by conservation) may offer superior capabilities for sequential computation and temporal control. The study concludes that conservation not only aids in achieving SOC but also ensures that the resulting critical dynamics are functionally aligned with task performance, eliminating the tension often seen between optimizing for criticality and optimizing for task utility.

Note: The authors acknowledge limitations, including the use of only three seeds per variant (limiting statistical power) and the application of conservation only to visible channels. They suggest future work should explore larger seed populations, hidden channel conservation, and co-optimization of criticality and task fitness.