💻 computer science

A New Evolutionary Strategy: Learn From the Best

This paper introduces Learn From the Best Evolution Strategy (LFB-ES), a novel black-box optimizer that enhances high-dimensional neural network training by iteratively guiding the population to learn from elite individuals, thereby achieving superior convergence and accuracy compared to classical methods like OpenAI-ES and CMA-ES.

Original authors: Zhijian Mo, Qihua Xiao, Zhihui Shan, Xueli Ban

Published 2026-08-31

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhijian Mo, Qihua Xiao, Zhihui Shan, Xueli Ban

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the vast landscape of artificial intelligence, there is a specific kind of puzzle that stumps even the most advanced learning systems. Imagine trying to teach a robot to walk, or a computer program to predict a complex weather pattern, but you are forbidden from giving it any hints along the way. You cannot tell it, "your left foot is too high" or "your prediction for Tuesday was slightly off." You can only wait until the very end of the task and hand it a single number: a score. This is known as a black-box optimization problem with sparse rewards. The system must figure out how to improve its internal settings based solely on that final grade, without any step-by-step feedback. This scenario is common in real-world engineering, from tuning the parameters of a new engine to fitting mathematical curves to noisy data, yet it remains one of the most difficult challenges because the path to improvement is invisible.

For years, scientists have relied on evolutionary strategies to solve these puzzles. These methods mimic natural selection: they create a population of digital agents, each with slightly different internal settings, let them try the task, and keep the ones that get the best scores. The two most famous of these methods, OpenAI-ES and CMA-ES, have been the standard tools for this work. However, they struggle when the tasks become highly complex and the number of settings to tune grows large. They often get stuck in local loops, moving slowly and failing to capture the fine details of the problem, much like a hiker wandering in a dense forest who keeps circling the same small clearing instead of finding the mountain peak.

A team of researchers from Lenovo has proposed a new way forward called "Learn From the Best Evolution Strategy," or LFB-ES. Instead of relying on random chance to stumble upon better solutions, this new method introduces a structured form of learning within the population. In every generation of the experiment, the single best-performing agent is chosen as a teacher. The rest of the group, acting as students, then study the teacher's behavior. They do not just copy the teacher's final score; they try to mimic the specific sequence of actions and outputs the teacher produced during the task. By using a mathematical process to minimize the difference between their own outputs and the teacher's, the students rapidly adjust their internal settings to be more like the winner. This creates a cycle where the population collectively climbs toward a better solution, guided by the current champion rather than wandering blindly.

The researchers tested this approach on a difficult curve-fitting challenge. They asked the algorithms to predict a highly complex, rapidly oscillating wave pattern, a task that requires capturing thousands of tiny peaks and valleys. The only information the algorithms received was the total error of their entire prediction at the end of the run. The results were striking. The new LFB-ES method converged on the correct answer much faster than the traditional methods and achieved a level of precision that the others could not reach. While the older algorithms produced flat, inaccurate lines that missed the intricate details of the wave, the new method reproduced the complex pattern with remarkable accuracy, matching the real data almost perfectly.

A crucial part of this success was the choice of the internal "switches" the computer uses to process information, known as activation functions. The researchers found that a specific type of switch that repeats in a cycle, similar to the way a sine wave rises and falls, worked far better than the standard switches used in most modern AI. When they swapped the standard switches for this repeating, periodic type within their new learning framework, the system's ability to explore and find the best solution improved dramatically. However, they also discovered that this advantage was not universal. When they applied the same method to a different type of problem involving discrete choices, like playing a video game where the agent must choose between moving left or right, the new method did not show the same overwhelming superiority. It performed slightly better than the old random methods but did not dominate the way it did in the continuous curve-fitting task.

The study also examined the cost of this new approach. Because the method requires the student agents to learn from the teacher through a series of calculations, it takes more time to run than the simplest random methods. However, the researchers showed that this extra time is manageable and scales reasonably well even as the number of settings to tune increases. In contrast, one of the older, more complex methods became impossible to run on large problems because it ran out of computer memory. The new strategy offers a middle ground: it is more computationally demanding than the simplest random search but far more efficient and capable than the heavy, memory-hungry alternatives, making it a practical tool for solving high-dimensional problems where only a final score is available.

Ultimately, this work demonstrates that even in a completely closed environment where no intermediate guidance is given, a population of agents can learn to improve rapidly if they are allowed to learn from their best performers. By combining this social learning dynamic with the right mathematical tools, the researchers have created a system that navigates the fog of black-box optimization with much greater clarity and speed than before. While the method is not a magic solution for every type of problem, it provides a powerful new tool for engineers and scientists who need to tune complex systems without knowing the internal rules of the game.

Technical Summary: Learn From the Best Evolution Strategy (LFB-ES)

1. Problem Definition

The paper addresses the challenges inherent in black-box optimization scenarios characterized by sparse terminal rewards. In these settings, an agent interacts with an unknown environment where no intermediate stepwise feedback is available; performance is evaluated solely via a single global scalar score (or error) after a complete trajectory. This paradigm is prevalent in offline parameter tuning, unknown nonlinear function regression, and offline agent evaluation.

While gradient-free evolutionary strategies like OpenAI-ES and CMA-ES are standard solutions for such tasks, the authors identify specific bottlenecks when applying them to high-dimensional neural networks:

CMA-ES: Suffers from quadratic storage complexity ( $O(n^2)$ ) and cubic computational complexity ( $O(n^3)$ ) due to covariance matrix updates, making it computationally prohibitive for high-dimensional deep networks.
OpenAI-ES: Relies on fixed Gaussian perturbations without adaptive search radius adjustment. This often leads to entrapment in local optima, slow convergence, and degraded fitting precision in complex, highly nonlinear tasks.
General Limitations: Both methods exhibit low sample utilization and struggle to achieve high fitting accuracy in complex curve regression tasks due to a lack of efficient inter-individual information exchange.

2. Methodology: LFB-ES

The authors propose Learn From the Best Evolution Strategy (LFB-ES), a novel population-level evolutionary optimizer that operates strictly under black-box constraints without requiring internal environmental priors.

Core Mechanism: Teacher-Student Knowledge Transfer

Unlike traditional evolutionary strategies that rely on undirected random perturbation, LFB-ES implements an iterative elite demonstration learning mechanism:

Population Initialization: $N$ agents are initialized with differentiated parameters. The network architecture utilizes cosine periodic activation functions.
Black-Box Evaluation: Each agent executes a full trajectory in the environment, receiving a global terminal score.
Elite Screening: The agent with the highest score (lowest loss) in the current generation is selected as the Teacher. Its state-output trajectory serves as the demonstration dataset.
Student Learning: All other agents (Students) update their parameters via gradient descent (using the Adam optimizer) to minimize the Mean Squared Error (MSE) between their outputs and the Teacher's outputs.
Iteration: This cycle repeats. Crucially, the algorithm constrains output consistency rather than parameter uniformity, allowing students to retain diverse parameter distributions while approximating the teacher's behavior. Outstanding students can eventually surpass the current teacher, creating a self-improving cycle.

Role of Periodic Activation Functions

A critical component of the methodology is the use of cosine periodic activation functions. The authors argue that periodic mapping allows networks with distinct internal parameters to generate highly consistent outputs. This property enables student networks to fit teacher trajectories while preserving sufficient parameter-space diversity, preventing the population homogeneity and performance saturation often observed with aperiodic activations (like Tanh or ReLU) in evolutionary settings.

3. Key Contributions

The paper outlines three primary contributions:

Novel Population Paradigm: Establishment of a teacher-student knowledge transfer mechanism within the population. This approach guides parameter updates using the generation-wise elite individual, mitigating the low exploration efficiency and slow convergence of traditional evolutionary strategies without leaking black-box information.
Validation of Periodic Activations: Systematic verification that periodic activation functions (specifically Cosine) are superior to mainstream aperiodic alternatives (Tanh, ReLU) within the LFB-ES framework. This combination maintains inter-individual diversity and enhances global exploration capabilities.
Open-Source Implementation: The authors have open-sourced the complete algorithm code to facilitate reproducibility.

4. Experimental Results

The authors conducted comprehensive experiments on complex black-box curve regression tasks, specifically high-frequency oscillatory functions ( $y = \sin(100x)$ ), comparing LFB-ES against OpenAI-ES and CMA-ES.

Convergence and Fitting Accuracy: LFB-ES demonstrated significantly faster convergence and substantially lower final fitting errors compared to baselines. In a 1,000-generation test, LFB-ES achieved a fitting error of 0.0027, whereas OpenAI-ES and CMA-ES (even with various activation functions) hovered around 0.50–0.65.
Activation Function Sensitivity: Replacing the activation functions of OpenAI-ES and CMA-ES with Cosine did not improve their performance, confirming that the synergy between the periodic function and the LFB-ES learning mechanism is unique.
Computational Overhead:
- CMA-ES failed to run on networks with 97k parameters due to out-of-memory errors.
- OpenAI-ES was the fastest but suffered from poor fitting accuracy.
- LFB-ES maintained linear time complexity, successfully running on 97k-parameter networks with acceptable runtime (101s for 100 generations), though slower than OpenAI-ES.
Scenario Limitations (Ablation): When tested on a discrete-action control task (Atari Pong-v5), LFB-ES showed only marginal improvement over OpenAI-ES and did not exhibit the overwhelming advantage seen in continuous curve fitting. This suggests the method is highly effective for continuous-output black-box tasks where every output influences the final feedback, but less so for discrete-action scenarios.

5. Significance and Claims

The paper claims that LFB-ES represents a significant advancement for high-dimensional neural network optimization in pure black-box environments. By shifting from random exploration to intra-population knowledge learning, the method effectively addresses the sample inefficiency and convergence stagnation of classical evolutionary strategies.

The authors emphasize that LFB-ES achieves high-precision network updating and progressive parameter optimization without requiring intermediate supervisory signals or domain-specific prior knowledge. The work suggests that the combination of elite demonstration learning and periodic activation functions offers a robust solution for complex nonlinear regression tasks where traditional gradient-based methods fail and standard evolutionary strategies are too slow or inaccurate.

Future work is identified as optimizing the demonstration learning mechanism and integrating discrete exploration strategies to extend applicability to discrete decision-making tasks like game control.