💻 computer science

A Classification-Regression Cooperative Fuzzy Surrogate- Assisted Evolutionary Algorithm for Expensive High- Dimensional Multi-Objective Optimization

This paper proposes HDFC-ASS, a classification-regression cooperative fuzzy surrogate-assisted evolutionary algorithm that integrates a variable-correlation-guided Kriging strategy, a fuzzy classifier-assisted local exploitation mechanism, and a convergence-diversity-uncertainty cooperative criterion to effectively solve expensive high-dimensional multi-objective optimization problems under strict evaluation budgets.

Original authors: Yishan Zhao, Xianwen Wei, Guoliang Sun, Kaiping Song

Published 2026-08-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yishan Zhao, Xianwen Wei, Guoliang Sun, Kaiping Song

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a chef trying to invent the perfect recipe for a new dish. You have a list of goals: it must be the spiciest, the cheapest, and the most nutritious all at the same time. But here's the catch: every time you want to taste a potential recipe to see if it works, you have to wait three days for the ingredients to grow and the kitchen to be cleaned. You only have enough time to taste about 500 dishes before your funding runs out. This is the world of "expensive optimization." In science and engineering, finding the best solution often means running complex computer simulations or physical experiments that take hours or days. When you have to juggle multiple conflicting goals (like speed vs. cost) and the variables are incredibly complex (like thousands of ingredients), trying to find the perfect answer by just guessing and tasting is impossible. You need a smarter way to guess.

Enter "surrogate-assisted evolutionary algorithms." Think of these as a team of sous-chefs who build a quick, rough sketch of what the final dish might taste like based on the few samples you've already tasted. Instead of waiting three days for a real taste, the sous-chef gives you a prediction in seconds. The problem is, if the kitchen is huge (high-dimensional) and you've only tasted a few dishes (small data), the sketch can be wildly wrong, leading you to waste your precious tasting time on bad ideas. This paper tackles that exact mess: how do you keep your sketch accurate when the recipe is complicated and you have very few real samples to work with?

The researchers, Yishan Zhao and their team from Wuwei Vocational and Technical University, propose a new method called HDFC-ASS. You can think of this algorithm as a very smart, two-stage detective trying to solve a mystery with very few clues.

Stage 1: The "Random Room" Explorer
In the beginning, the detective doesn't try to look at the whole giant mansion (the entire complex problem) at once. That would be too confusing and the clues would be too scattered. Instead, the algorithm uses a trick called "variable-correlation-guided random-subspace incremental Kriging." In plain English, it picks a few related clues (variables) that seem to matter most, mixes them with a few random ones, and builds a small, focused map of just that corner of the mansion. It does this in several different "rooms" (subspaces) and combines the maps. This helps the algorithm explore the vast, high-dimensional space without getting lost or building a shaky map based on too little data. It's like exploring a huge forest by looking at small, manageable patches of trees rather than trying to see the whole forest from a helicopter when you only have a few binoculars.

Stage 2: The "Fuzzy" Judge
Once the detective has gathered enough clues and the maps start to look reliable, the strategy shifts. Now, instead of just guessing numbers (regression), the algorithm switches to a "fuzzy classifier." Imagine a judge who doesn't just say "Yes, this is a good dish" or "No, it's bad." Instead, the judge gives a score of "how likely" a dish is to be good. If the dish is clearly great, the score is high. If it's clearly bad, the score is low. If it's right on the edge, the score is somewhere in the middle. The paper emphasizes that this switch only happens when the judge is proven to be reliable. If the judge is still confused (early in the process), the algorithm sticks to the map-making stage. This prevents the algorithm from making a premature decision based on a shaky guess.

The Magic Switchboard
The real genius of HDFC-ASS is how it manages its limited budget of "tastes" (expensive evaluations). It has a special rulebook called "convergence-diversity-uncertainty model management." It constantly asks three questions:

Convergence: Are we getting closer to the best possible solution?
Diversity: Are we looking at a wide variety of options, or just the same few?
Uncertainty: Is our map blurry in this area? Do we need to taste a dish here to make the map clearer?

Based on the answers, the algorithm decides whether to explore new areas, fix the blurry parts of the map, or focus on the most promising candidates. It's like a smart budget manager that knows exactly when to spend money on new research and when to invest in refining a winning idea.

What Did They Find?
The team tested their new detective on a series of tough challenges called DTLZ, WFG, and MaF. These are like standard "obstacle courses" for optimization algorithms, featuring problems that are multimodal (full of traps), biased (unfairly skewed), or disconnected (broken paths). They compared HDFC-ASS against nine other popular methods.

The results showed that HDFC-ASS was generally better at finding high-quality solutions, especially in the most difficult scenarios where other algorithms struggled to balance exploring new ideas and refining good ones. In the tests, where the algorithms were allowed only 500 expensive evaluations to solve problems with up to 200 variables, HDFC-ASS consistently produced better results than its competitors. However, the paper notes that this power comes with a cost: HDFC-ASS takes a bit more computer time to run than some simpler methods, though it is still much faster than the most complex alternatives.

In short, the paper suggests that by switching between "map-making" and "fuzzy judging" only when the time is right, and by breaking big problems into smaller, manageable pieces, you can solve incredibly complex, expensive puzzles much more efficiently. It's not a magic wand that solves everything instantly, but it's a very effective tool for navigating the foggy, high-dimensional forests of modern engineering design.

Technical Summary: A Classification-Regression Cooperative Fuzzy Surrogate-Assisted Evolutionary Algorithm for Expensive High-Dimensional Multi-Objective Optimization

1. Problem Statement
The paper addresses the challenge of solving expensive high-dimensional multi-objective optimization problems (MOPs) where objective function evaluations are computationally costly (e.g., requiring physical experiments or high-fidelity simulations) and the number of available evaluations is strictly limited.

A specific challenge identified is the model-state mismatch in existing Surrogate-Assisted Evolutionary Algorithms (SAEAs). In high-dimensional spaces with sparse data:

Full-dimensional regression models (like Kriging) become unstable and unreliable.
Early-stage classification boundaries are often imbalanced and unstable, leading to premature exploitation based on unreliable labels.
Existing methods often treat global exploration (regression) and local exploitation (classification) separately, lacking a unified mechanism to coordinate these tasks based on the current reliability of the surrogate models.

2. Methodology: HDFC-ASS
The authors propose HDFC-ASS (Classification-Regression Cooperative Fuzzy Surrogate-Assisted Evolutionary Algorithm). The framework operates through two sequential stages connected by a shared evaluated archive, utilizing a Convergence-Diversity-Uncertainty (CDU) model management criterion to allocate expensive evaluations.

Stage 1: Variable-Correlation-Guided Random-Subspace Incremental Kriging (Global Exploration)
- Instead of building a full-dimensional model, the algorithm estimates the relevance of decision variables to objectives using Pearson and Spearman correlations.
- It constructs informative low-dimensional subspaces by combining core relevant variables with randomly sampled complementary variables.
- Incremental Kriging models are trained and updated within these subspaces. An ensemble approach aggregates predictions from multiple subspaces to estimate the mean and uncertainty (variance), reducing the instability of full-dimensional modeling under small-sample conditions.
Stage 2: Fuzzy Classifier-Assisted Local Exploitation
- A fuzzy classifier (MLPM-FKNN) is trained to output the membership degree of a candidate solution belonging to the "promising" class, rather than a hard binary label.
- Reliability-Based One-Way Transition: The algorithm does not permanently fuse regression and classification. It remains in the global exploration stage until the fuzzy classifier satisfies specific reliability conditions (measured by balanced accuracy and class support). Only then does it transition to local exploitation. This prevents unreliable early classification boundaries from misleading the search.
Model Management and Infill Sampling
- A cooperative criterion adaptively selects candidates for true expensive evaluation based on three factors:
  1. Convergence: Distance to the estimated ideal point.
  2. Diversity: Distance to the nearest evaluated solution.
  3. Uncertainty: Prediction variance from the Kriging ensemble.
- Depending on the stage and classifier reliability, the algorithm prioritizes:
  - Global Exploration: Selecting candidates with high uncertainty and good convergence/diversity scores.
  - Boundary Correction: If the classifier is unreliable, sampling near the fuzzy decision boundary (membership $\approx 0.5$ ).
  - High-Membership Exploitation: If the classifier is reliable, selecting candidates with high positive-class membership and good convergence/diversity.

3. Key Contributions
The paper outlines three primary contributions:

Variable-Correlation-Guided Random-Subspace Incremental Kriging: A strategy to handle high-dimensional spaces by reducing effective modeling dimensions while preserving search directions, updated incrementally to improve efficiency.
Fuzzy Classifier-Assisted Local Exploitation with One-Way Transition: A mechanism that uses fuzzy membership values and triggers the switch to local exploitation only when the classifier reaches a state of high reliability (balanced accuracy), avoiding premature exploitation.
Convergence-Diversity-Uncertainty Cooperative Model Management: A unified criterion that dynamically allocates the limited evaluation budget among global exploration, boundary correction, and high-confidence exploitation based on the current state of the surrogate models.

4. Experimental Results
The proposed algorithm was evaluated on the DTLZ, WFG, and MaF benchmark suites with decision variables ranging from 50 to 200 and objectives from 3 to 11. The evaluation budget was strictly limited to 500 expensive function evaluations.

Performance: HDFC-ASS demonstrated competitive or superior performance compared to nine representative SAEAs (including classification-assisted, regression-assisted, and ensemble methods) in terms of Inverted Generational Distance (IGD) and Hypervolume (HV).
Robustness: The algorithm showed particular strength on difficult problem characteristics, such as multimodal landscapes (DTLZ1, DTLZ3), biased transformations (WFG1), disconnected Pareto fronts (WFG2), and irregular fronts (DTLZ5-7).
Visual Analysis: Distribution plots indicated that HDFC-ASS achieved a broader individual range and better solution distribution compared to competitors.
Computational Cost: While HDFC-ASS requires more computation time than some simpler methods (like MCEA/D) due to the maintenance of random-subspace models and fuzzy classification, it is significantly more efficient than complex heterogeneous ensemble methods (like HeEMOEA) in high-dimensional settings.

5. Significance and Claims
The paper claims that the performance improvement of HDFC-ASS stems primarily from its model-state-aware cooperation mechanism. Rather than simply fusing model outputs, the algorithm activates regression and classification surrogates according to their trustworthy states.

The random-subspace incremental Kriging effectively guides global exploration in high-dimensional spaces where full models fail.
The reliability-controlled fuzzy classifier strengthens local screening only after the classification boundary becomes trustworthy.

The authors conclude that HDFC-ASS offers an effective trade-off between model-management cost and optimization performance, making it suitable for expensive high-dimensional MOPs under strictly limited evaluation budgets. Future work is suggested to focus on adaptive parameter control and real-world engineering applications.

More like this