🔬 physics

Emergent swimming strategies of a smart three-bead swimmer

Using reinforcement learning and neuroevolution, researchers trained a simple three-bead microswimmer model to autonomously discover five distinct swimming gaits, demonstrating that efficient, adaptive locomotion at low Reynolds numbers can be achieved with minimal computational complexity.

Original authors: Ruma Maity, Maximilian Huebl, Julian Lemmel, Benedikt Hartl, Gerhard Kahl

Published 2026-06-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ruma Maity, Maximilian Huebl, Julian Lemmel, Benedikt Hartl, Gerhard Kahl

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a tiny, invisible robot made of three beads connected by stretchy arms, floating in a thick, sticky fluid like honey. This is the "three-bead swimmer" the researchers studied.

In our normal world, if you want to move, you push against something (like walking on the ground or paddling a boat). But in this tiny, sticky world, the rules are different. If you just wiggle back and forth in the same way you started, you won't go anywhere; you'll just end up exactly where you began. This is known as the "Scallop Theorem." To move, you have to wiggle in a clever, non-repeating pattern.

The Problem: How do you teach a robot to wiggle?
Usually, scientists have to manually program these robots with complex instructions on exactly how to move their arms. But real life is messy. A robot needs to be smart enough to figure out how to move on its own, adapting to its environment without a human holding a remote control.

The Solution: A "Brain" that Learns by Trial and Error
The researchers gave this three-bead robot a very simple "brain" (a tiny computer program called a neural network) and let it learn how to swim using a method called Reinforcement Learning.

Think of it like training a dog, but for a robot:

The Goal: The robot wants to swim as far as possible.
The Reward: Every time the robot moves forward, it gets a "treat" (a digital reward score). If it spins in circles or stays still, it gets no treat.
The Evolution: The researchers started with 200 random "brains." Most were terrible and didn't move. The ones that moved even a little bit were kept. Their "brains" were then slightly tweaked and mixed together (like breeding animals) to create a new generation.
The Result: Over thousands of generations, the robots evolved from clumsy wiggles into highly efficient swimmers.

The Discovery: Five Unique Swimming Styles
By adjusting the "rules" of the reward (telling the robot to value speed, straight lines, or spinning differently), the researchers discovered five distinct swimming styles that the robots invented on their own:

The Flapping Mode (The Sprinter): This was the most efficient. The robot moved in a straight line, very fast. It looked like a swimmer doing a breaststroke but with a specific, rhythmic pattern that never repeated itself exactly. It used a brain so simple it had almost no internal parts—just a few direct connections.
The Chiral Mode (The Spiral): This robot moved forward but also spun slightly, creating a spiral path. It was like a corkscrew moving through the honey.
The Walking Mode (The Hiker): This was the most complex style. The robot moved in a straight line but with a very intricate, "stepping" motion. It required a slightly more complex brain to coordinate the timing of its three arms.
The Rotational Mode (The Top): When the researchers changed the rewards to encourage spinning, the robot learned to spin in place. Its center stayed fixed, but the beads orbited around it. It didn't go anywhere, but it mastered the spin.
The Circular Mode (The Drifter): Here, the robot's center of mass moved in a perfect circle. It was moving, but it never got closer to a destination; it just circled endlessly.

Why This Matters
The most surprising part of the study is how simple the brains were. Even though the robots learned complex movements, the "neural networks" controlling them were tiny—some had fewer than ten connections.

This proves that you don't need a supercomputer or a massive, complex brain to make a tiny robot swim efficiently. You just need a simple set of rules and a way to learn from mistakes. The paper suggests that this approach could help scientists design better artificial micro-swimmers for the future, and it helps us understand how real microscopic creatures (like the Chlamydomonas algae) might have evolved their own simple, effective ways to navigate the world.

Technical Summary: Emergent Swimming Strategies of a Smart Three-Bead Swimmer

Problem Statement
Low-Reynolds-number microswimmers are ubiquitous in biology and hold significant potential for biomedical applications, such as targeted drug delivery and noninvasive surgery. However, a critical barrier to the deployment of artificial microswimmers is their lack of autonomy. Existing models often rely on fixed swimming strategies that cannot adapt to dynamically changing environments. While Machine Learning, specifically Reinforcement Learning (RL), offers a path to autonomous control, current approaches typically employ deep neural networks with hundreds of thousands of parameters. These large-scale networks are computationally expensive, operationally brittle, and often fail to generalize to out-of-distribution scenarios, making them unsuitable for resource-constrained, real-world in vivo environments. Furthermore, the internal decision-making processes of such complex networks are often opaque.

Methodology
The authors investigate a two-dimensional model of a triangular microswimmer consisting of three beads connected by three actuating arms. The system operates in a viscous fluid where hydrodynamic coupling is described by the Oseen tensor. The motion of the beads is governed by the balance of viscous forces and the forces generated by the arms, which expand and contract.

To achieve autonomous swimming, the authors employ a Reinforcement Learning (RL) framework based on the NeuroEvolution of Augmented Topologies (NEAT) algorithm. Unlike traditional RL that optimizes weights within a fixed network architecture, NEAT simultaneously evolves both the network topology (structure) and the connection weights.

Agent-Environment Interaction: The "agent" is an Artificial Neural Network (ANN) that takes the current lengths of the three swimmer arms as input and outputs the forces to be applied to those arms.
Reward Scheme: A flexible, weighted linear combination of four partial rewards ( $r_1$ $r_{1}$ to $r_4$ $r_{4}$ ) is used to guide the evolution. These rewards quantify:
1. Instantaneous center-of-mass (COM) velocity ( $r_1$ ).
2. Average COM velocity over time ( $r_2$ ).
3. Geometric constraints preventing the swimmer from collapsing into a line ( $r_3$ ).
4. Coupling between translational and rotational degrees of freedom ( $r_4$ ).
Evolutionary Process: A population of 200 networks is initialized and evolved over 1000 generations. Networks are evaluated by simulating 6000 interaction steps. High-performing networks are selected, mutated, and recombined to form the next generation. The study systematically varies the weights ( $\rho_i$ ) of the partial rewards to explore the space of possible swimming strategies.

Key Contributions and Results
The study successfully trains the three-bead swimmer to discover five distinct, emergent swimming gaits by tuning the reward weights. The results demonstrate that highly efficient and robust locomotion can be achieved using extremely simple neural architectures (often fewer than ten nodes and weights).

Non-Reciprocal Modes (Directed Locomotion): Three strategies result in net displacement, adhering to the requirements for swimming at low Reynolds numbers (non-reciprocal shape deformation):
- Flapping Mode: The most efficient strategy, characterized by a straight-line trajectory. It involves a 5-stroke cycle where the swimmer maintains an isosceles triangle shape. The associated ANN is remarkably simple, containing no hidden nodes and only five connections.
- Chiral Mode: A 4-stroke cycle resulting in linear propagation superimposed with rotational motion. The swimmer maintains an isosceles shape with a specific phase shift in arm length expansion/contraction. The ANN requires no hidden nodes and six connections.
- Walking Mode: The most complex non-reciprocal gait, involving a 6-stroke cycle with intricate interplay between arm lengths and forces. This mode results in straight-line propagation with periodic reorientations. The corresponding ANN is more complex, featuring three internal nodes and twelve connections.
Reciprocal Modes (Inefficient/Localized Motion): Two strategies result in no net displacement, consistent with the Scallop Theorem, which states that reciprocal motion cannot produce net propulsion in a viscous fluid:
- Rotational Mode: The beads orbit a fixed center of mass in a 3-stroke cycle.
- Circular Mode: The center of mass traces a circular path in a 4-stroke cycle.
  In both cases, the shape deformations are reciprocal, and the resulting ANNs are simple.

Significance and Claims
The paper claims that combining neuroevolution-based RL with a targeted reward scheme allows for the training of microswimmers that adapt their strategies to highly viscous environments without prior knowledge of the system dynamics. The primary significance lies in the demonstration that minimal "computational power" (simple, sparse neural networks) is sufficient to generate robust, efficient, and diverse swimming behaviors.

The authors emphasize that these simple architectures are promising candidates for the experimental design of artificial microswimmers, as they are easier to implement physically than deep learning controllers. The results provide a transparent view of the decision-making strategies (coordination between arm lengths and forces) and suggest that such approaches could be extended to more complex geometries and ensembles of microswimmers. The work also offers insights into the locomotion of biological microorganisms, such as Chlamydomonas reinhardtii, which share similar triangular structural features.

Technical Summary: Emergent Swimming Strategies of a Smart Three-Bead Swimmer

More like this