💻 computer science

Benefits of Low-Cost Bio-Inspiration in the Age of Overparametrization

This paper demonstrates that in robot control tasks with limited input-output spaces, bio-inspired controllers with fewer parameters, such as shallow MLPs and densely connected CPGs, often outperform overparametrized deep learning architectures, suggesting that evolutionary strategies are more effective than reinforcement learning in these constrained contexts.

Original authors: Kevin Godin-Dubois, Anil Yaman, Anna V. Kononova

Published 2026-04-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Kevin Godin-Dubois, Anil Yaman, Anna V. Kononova

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot spider how to walk. You have two main ways to give it a "brain":

The "Instinct" Brain (CPG): Think of this like a biological reflex. It's a simple, rhythmic system that just knows how to move legs in a wave, like a heartbeat or a breathing pattern. It's cheap, simple, and doesn't need to think too hard.
The "Overachiever" Brain (MLP): This is a deep neural network, like the massive AI models we hear about today. It has millions of connections and tries to "learn" everything from scratch. It's powerful, but it's also heavy, expensive, and sometimes gets confused by its own complexity.

The Big Question:
In the world of AI, everyone is obsessed with making models bigger and bigger (adding more parameters). But does a bigger brain actually make a robot walk better, especially if the robot is simple and doesn't have many sensors?

This paper says: No. Sometimes, bigger is worse.

Here is the breakdown of their findings using simple analogies:

1. The "Too Many Cooks" Problem

The researchers tested a robot spider with only 8 legs and very limited sensors (it only knows where its legs are, not the whole world).

The Result: When they gave the "Overachiever" brain (the deep neural network) too many parameters to play with, it actually performed worse.
The Analogy: Imagine trying to solve a simple math problem (like 2+2) but you are forced to use a supercomputer with a billion processors. The computer gets overwhelmed by all the options, gets stuck in loops, and takes forever to give you the answer. Meanwhile, a simple calculator (the "Instinct" brain) gives you the answer instantly.
The Lesson: For simple robots, shallow, simple networks work better than deep, complex ones.

2. The "Gym vs. The Wild" (Training Methods)

They tried two different ways to teach the robot:

Evolutionary Strategy (CMA-ES): This is like natural selection. You create 1,000 random brains, let them try to walk, kill the ones that fall over, and mix the DNA of the winners to make the next generation. It's slow but very robust.
Reinforcement Learning (PPO): This is like training a dog with treats. The robot tries things, gets a "treat" (points) for moving forward, and learns to repeat the good moves. This is the method used by big tech companies today.

The Surprise:
The "Natural Selection" method (Evolutionary) actually worked better than the "Dog Training" method (RL) for this specific robot.

Why? The "Dog Training" method (PPO) requires a massive "critic" brain to judge the robot's moves. This adds thousands of extra parameters. Since the robot is simple, this extra weight just slows it down. The "Natural Selection" method didn't need that extra weight, so it was more efficient.

3. The "Frugal" vs. The "Fast"

They tested three different goals for the robot:

Goal A: Go Fast! (Speed)
- Winner: The simple "Instinct" brain (CPG) and small neural networks.
Goal B: Be Efficient (Don't hit the ground hard, save energy).
- Winner: The "Instinct" brain (CPG) was the clear champion. The complex brains struggled to learn how to be gentle because they were too busy overthinking.
Goal C: Be Balanced (Walk smoothly and steadily).
- Winner: A tie between simple brains.

4. The "Diversity" Trade-off

There was one downside to the simple "Instinct" brain.

The Analogy: The simple brain is like a jazz musician who only plays one perfect song. It's amazing at that song, but it can't improvise. The complex brain is like a musician who can play any song, but they might play the wrong one for the occasion.
The Finding: The complex brains (MLPs) could generate a wider variety of weird walking styles (diversity). The simple brains (CPGs) were stuck in a rhythmic loop. However, for the specific task of walking a robot spider, being stuck in a good rhythm was actually a good thing.

The Bottom Line

In the age of "Bigger is Better," this paper argues for Low-Cost Bio-Inspiration.

If you are building a simple, low-cost robot (like a disaster rescue bot or a toy), you don't need a supercomputer brain. You don't need millions of parameters.

Use simple, rhythmic controllers (CPGs).
Use evolutionary training (Natural Selection).
Keep it small.

Adding more complexity doesn't make the robot smarter; it just makes it slower, more expensive, and harder to train. Sometimes, the best way to move forward is to keep it simple.

1. Problem Statement

The paper addresses a critical gap in Evolutionary Robotics (ER) and Reinforcement Learning (RL): the lack of systematic guidelines for selecting controller architectures when robots have limited sensory capabilities.

Context: Modern RL trends favor "overparametrization" (using massive neural networks), assuming more parameters lead to better performance. However, in robotics, especially with low-cost modular robots, input spaces are often small (e.g., 8 dimensions), while output spaces are similarly constrained.
Hypothesis: In low-dimensional input/output scenarios, having an excessive number of parameters may actively hinder the learning process rather than empower it.
Goal: To empirically compare the efficiency and performance of two bio-inspired paradigms—Central Pattern Generators (CPGs) and Multi-Layer Perceptrons (MLPs)—across different parameter spaces and training regimes.

2. Methodology

Experimental Setup

Robot Morphology: The study uses the ARIEL (formerly Revolve) modular robot platform, specifically a "spider" morphology with 8 active hinges (4 hips, 4 knees).
Constraints: The robot has limited proprioception, providing only an 8-dimensional input (joint positions) and an 8-dimensional output (joint commands). This contrasts with high-end RL benchmarks (like the MuJoCo Ant) which use ~105 observations.
Architectures Tested:
1. CPGs: Coupled oscillators with varying neighborhood connectivity ( $c_0$ to $c_6$ ). The parameter space ranges from 8 (no connections) to 36 (fully connected).
2. MLPs: Artificial Neural Networks with varying depth ( $d \in \{0, 1, 2\}$ ) and width ( $w \in \{1, \dots, 128\}$ ).
Optimization Algorithms:
- CMA-ES (Covariance Matrix Adaptation Evolutionary Strategy): A gradient-free method used for both CPGs and MLPs.
- PPO (Proximal Policy Optimization): A gradient-based RL method used only for MLPs (as off-the-shelf CPG RL solutions were unavailable).
Reward Functions: Three distinct objectives were used to test different gait behaviors:
1. Speed ( $R_s$ ): Maximizes forward velocity.
2. Gymnasium ( $R_g$ ): Penalizes large control signals and ground impact forces, encouraging "frugal" locomotion.
3. Kernel-based ( $R_k$ ): Encourages specific target velocity, zero lateral/vertical speed, and a specific body elevation (stable gait).

Evaluation Metrics

Raw Performance: The absolute score achieved on the reward functions.
Parameter Impact ( $I_F$ ): A novel metric introduced to measure efficiency. Defined as $I_F = \hat{f} / \log_{10}(p)$ , where $\hat{f}$ is normalized performance and $p$ is the number of parameters. This quantifies how much performance is gained per order of magnitude of added parameters.
Cross-Performance: Evaluating a controller trained on one reward function against the other two to test generalization.
Diversity: Analyzed via Principal Component Analysis (PCA) on fitted sinusoidal parameters of foot trajectories to measure gait variety.

3. Key Contributions

Systematic Comparison: A comprehensive benchmark comparing CPGs and MLPs under identical low-dimensional constraints, moving beyond single-architecture studies.
Parameter Impact Metric: Introduction of a metric to empirically demonstrate that increasing parameter count does not linearly correlate with performance in low-dimensional robotics tasks.
Efficiency Findings: Evidence that shallow MLPs and densely connected CPGs outperform deeper, overparametrized networks in this specific context.
Algorithmic Insight: Demonstration that CMA-ES generally outperforms PPO in terms of efficiency and robustness for this specific morphology and sensory setup.

4. Results

Performance vs. Parameter Count

CPGs: Performance generally increased with connectivity. The fully connected CPG ( $c_6$ , 36 parameters) or near-fully connected ( $c_4$ ) yielded the best results.
MLPs: Exhibited a "bell-shaped" performance curve.
- CMA-ES: Best performance was achieved with very small networks (e.g., $m^2_8$ with 2 layers of 8 neurons). Increasing parameters by ~50x only improved speed performance by ~3%.
- PPO: Larger networks often failed to outperform smaller ones, particularly on the "frugal" ( $R_g$ ) and "stable" ( $R_k$ ) rewards.
Overparametrization: For the "frugal" reward ( $R_g$ ), MLPs trained with PPO performed poorly, often converging to non-moving solutions due to the difficulty of the penalty landscape. CPGs, with their inherent rhythmic bias, handled this constraint better.

Efficiency (Parameter Impact)

CMA-ES vs. PPO: CMA-ES was significantly more efficient. For the speed reward, PPO required ~27,000% more parameters to achieve a 47% performance gain over CPGs, resulting in a much lower Parameter Impact score.
Optimal Architectures:
- Speed ( $R_s$ ): MLPs (trained with PPO) slightly outperformed CPGs in raw speed, but CPGs were far more parameter-efficient.
- Frugality ( $R_g$ ): CPGs and shallow Perceptrons ( $m^0_0$ ) dominated. Deep MLPs failed to learn effective gaits.
- Stability ( $R_k$ ): CPGs and shallow MLPs performed similarly.

Cross-Performance and Diversity

Generalization: CPGs showed superior cross-performance. A CPG trained on one reward function tended to perform reasonably well on the others. MLPs were highly specialized; a network trained for speed often failed completely on stability or frugality tasks.
Diversity: MLPs produced a wider variety of gaits (higher diversity in PCA space) due to their larger parameter space. CPGs were more constrained to rhythmic patterns, which, while limiting diversity, contributed to their robustness and cross-task performance.

5. Significance and Conclusion

The paper challenges the prevailing "bigger is better" paradigm in robotics control, specifically for low-cost, low-sensory robots.

Low-Cost Bio-Inspiration: The study validates that bio-inspired, low-parameter architectures (like CPGs) are not just biologically plausible but computationally superior for specific robotic constraints. They offer a better balance between expressivity and performance.
Training Regime: CMA-ES is shown to be a more robust and efficient optimizer than PPO for this class of problems, largely because it avoids the overhead of Actor-Critic architectures and handles the non-differentiable nature of CPGs naturally.
Practical Implication: For modular robotics with limited sensors, designers should prioritize shallow networks or structured CPGs over deep, overparametrized MLPs. This reduces computational load, energy consumption, and the risk of overfitting or learning failure in sparse reward landscapes.

In summary, the authors argue that in the age of overparametrization, frugality in parameters is a critical design principle for efficient and robust robot control.