Optimum control strategies for maximum thrust production in underwater undulatory swimming

This study leverages a biomimetic robotic swimmer combined with machine learning and intuitive models to identify optimal control strategies for maximizing thrust production, offering a practical, model-free implementation for autonomous underwater locomotion that bridges fluid dynamics, robotics, and biology.

The Gist

Imagine trying to push a heavy shopping cart through a crowded supermarket. You could push it gently and steadily, or you could give it a sharp, hard shove, wait for it to slow down, and then shove it again. This paper explores which method is best for a robot fish trying to swim as fast as possible through water.

Here is the story of their discovery, broken down into simple concepts:

The Problem: How to Swim Like a Fish

Real fish, whales, and tadpoles swim by wiggling their bodies back and forth. This creates a wave that pushes against the water, propelling them forward. Scientists have long wondered: What is the perfect "wiggle" to go the fastest?

Is it a smooth, gentle wave (like a sine wave)? A jagged, triangular wave? Or something else entirely? To find out, the researchers built a robotic fish and let a computer learn the best way to move it.

The Experiment: Teaching a Robot with "Reinforcement Learning"

The team built a robot fish with a flexible tail made of soft plastic. They attached a motor that could pull cables to bend the tail, just like muscles pull on bones in a real fish.

Instead of programming the robot with a specific rule (like "wiggle at 2 hertz"), they used Reinforcement Learning. Think of this like training a dog:

• The robot tried different movements.
• Every time it pushed harder against the water (creating more "thrust"), the computer gave it a "reward."
• Every time it moved inefficiently, it got no reward.

Over time, the computer figured out the perfect pattern to maximize that reward.

The Big Discovery: The "Square Wave"

The computer didn't find a smooth, gentle wave. Instead, it discovered that the fastest way to swim is to use a Square Wave.

The Analogy: Imagine you are on a playground swing.

• The Smooth Way: You gently push the swing forward and backward in a slow, rhythmic circle.
• The Square Wave Way: You push the swing as hard as you can to the very back, hold it there for a split second, and then immediately shove it as hard as you can to the very front. You are constantly switching between "Full Speed Forward" and "Full Speed Backward" with no in-between.

The robot found that switching the motor between its two extreme limits (maximum left and maximum right) created the most thrust. It's like a "Bang-Bang" controller: you are either "Bang" (full power) or "Bang" (full power in the other direction). There is no "maybe."

Why Does This Work?

The researchers built a mathematical model to understand why this works. They found two main reasons:

1. The Motor's Limits: The robot's motor has a maximum speed. If you ask it to move smoothly, it spends a lot of time accelerating and decelerating. By switching instantly between the extremes, the motor spends almost all its time spinning at its top speed.
2. The Water's Rhythm: The water and the tail have a natural "resonance" (like a swing has a natural rhythm). The square wave hits this rhythm perfectly, keeping the tail moving as fast as possible without wasting energy fighting against the water's resistance.

The "Swinging" Strategy: No Math Required

The researchers realized that to use the perfect square wave, you usually need to know exactly how heavy the robot is, how stiff the tail is, and how fast the motor spins. This is hard to know in the real world.

So, they came up with a clever, "model-free" trick they call "Swinging Control."

The Analogy: Think of a child on a swing who doesn't know physics. They don't calculate the perfect time to push. Instead, they just wait until the swing slows down at the top of its arc, and then they push again.

• The robot does the same thing. It watches the tail.
• As long as the tail is moving fast, it keeps the motor in one direction.
• The moment the tail starts to slow down too much, the robot instantly flips the motor to the other side.

This strategy works almost as well as the perfect mathematical solution, but it doesn't require any prior knowledge of the robot's physics. It just reacts to what is happening in the moment.

The Final Proof

To make sure this wasn't just a fluke with their specific robot, they ran a massive computer simulation of a fish swimming in a virtual tank of water. They tested smooth waves, jagged waves, and the "switching" strategy.

The Result: The "switching" strategy (the square wave) consistently made the virtual fish swim faster than any other method.

The Takeaway

To swim as fast as possible underwater, you don't need to be smooth and gentle. You need to be decisive. Switch your power between the two extremes, and flip the direction the moment your speed starts to drop. It's a simple, powerful rule that bridges the gap between how robots move and how nature swims.

Technical Summary

Technical Summary: Optimum Control Strategies for Maximum Thrust Production in Underwater Undulatory Swimming

Problem Statement
Underwater undulatory swimming, utilized by fishes and cetaceans, involves a complex interplay between internal dynamics (muscle contraction, decision-making) and external fluid-structure interactions. While biological swimmers achieve high efficiency, replicating this in autonomous robotic systems remains a challenge. Specifically, there is a lack of conclusive understanding regarding the optimal control strategy required to maximize thrust production in artificial swimmers. Previous studies have explored correlations between speed and tail-beat frequency but have not definitively identified the control signal shape or the precise mechanism for maximizing thrust without extensive prior system knowledge.

Methodology
The authors employed a multi-faceted approach combining experimental robotics, machine learning, theoretical modeling, and numerical simulation:

1. Experimental Platform: A biomimetic robotic fish with a deformable polymer skeleton and a fin was constructed. A waterproof servomotor, connected via cables, induced body deformation. The robot was fixed at the head to a force sensor to measure longitudinal thrust ($F_x$).
2. Machine Learning (Reinforcement Learning - RL): Deep RL algorithms were used to identify the control signal ($\phi_c(t)$) that maximizes average thrust. Two input sets were tested:
   • State-based inputs: Servomotor instruction angle, lateral force ($F_y$), and its derivative.
   • Vision-based inputs: Sequences of camera images.
     The RL agent was constrained to vary the instruction angle within $[-\Phi, \Phi]$.
3. Theoretical Modeling: A mathematical model was developed combining:
   • A damped harmonic oscillator equation describing the fin angle dynamics ($\alpha$) driven by the servomotor.
   • A nonlinear equation describing the servomotor's internal dynamics (saturation effects) relative to the instruction angle.
   • A thrust equation proportional to the square of the fin angular velocity ($\dot{\alpha}^2$).
     The model was also optimized using RL to verify experimental findings.
4. Analytical Derivation: Pontryagin's maximum principle was applied to the model to explain the optimality of specific control strategies under different limits (fast vs. slow servomotors).
5. Numerical Simulation: Full 2D direct fluid-structure interaction (FSI) simulations were conducted to validate the strategies in a realistic aquatic environment, modeling the swimmer as a viscoelastic beam.

Key Results

• Optimal Control Signal Shape: Both experimental RL and theoretical modeling converged on a square wave function as the optimal control signal. The instruction angle switches abruptly between the two extreme allowed values ($\pm \Phi$). This "bang-bang" control consistently outperformed sinusoidal and triangular waveforms across various frequencies.
• Optimal Frequency: The maximum thrust is achieved at a specific frequency ($f^*$) that depends on the system's internal dynamics and the instruction amplitude ($\Phi$).
  • For fast servomotors (where the motor can track the command), the optimal frequency approaches the natural undulation frequency of the system ($f^* \approx \omega_0/2\pi$).
  • For slow servomotors, the optimal frequency is determined by the time required to sweep the angle at maximum angular speed ($f^* = \Omega/4\Phi$).
• Model-Free "Swinging" Strategy: The authors proposed a practical, model-free control strategy called "swinging control." This strategy switches the instruction sign when the fin angular velocity drops to a fraction ($C$) of its maximum observed value.
  • This approach requires no prior knowledge of system parameters (mass, stiffness, damping).
  • It achieves over 95% of the maximum possible thrust efficiency for a wide range of system parameters when the threshold $C$ is set to 0.6.
• Validation: The 2D FSI simulations confirmed that square wave forcing yields the highest cruising speed. Furthermore, the "swinging" controller in simulation automatically selected a frequency that propelled the swimmer to near-maximum speeds.

Significance and Claims
The paper claims to bridge fluid dynamics, robotics, and biology by providing a unified framework for understanding and optimizing aquatic locomotion. The primary contributions are:

1. Identification of the Optimal Strategy: The study definitively identifies a square wave (bang-bang) control signal as the most effective method for generating maximum thrust in undulatory swimmers, driven by the interplay between internal motor saturation and external fluid resonance.
2. Practical Implementation: The introduction of the "swinging control" strategy offers a robust, model-free solution for autonomous robotic swimmers. It eliminates the need for complex system identification or prior knowledge of physical equations, making it highly applicable for real-world deployment.
3. Theoretical Insight: The work explains why this strategy works, linking the optimal control to the maximization of tail speed and the system's resonance properties, validated through both simplified models and complex numerical simulations.

The authors conclude that these findings provide valuable insights for designing efficient artificial swimmers and potentially enhancing the performance of human aquatic athletes, though they frame these as potential applications of the derived physical principles rather than immediate, tested outcomes.