Finding Sets of Pareto Sets in Real-World Scenarios -- A Multitask Multiobjective Perspective

1. Problem Statement

The paper addresses the challenge of generating a Set of Pareto Sets (SOS) for real-world optimization problems.

• Context: Traditional Multiobjective Optimization (MOP) typically solves for a single Pareto Front (PF) under fixed parameters. However, in real-world scenarios (e.g., engineering design, supply chain), task settings (load conditions, material properties, demand fluctuations) vary dynamically.
• Limitation of Current Approaches: Solving a new MOP for every specific environment is computationally expensive and inefficient. Furthermore, existing Evolutionary Multitasking (EMT) research has largely focused on synthetic benchmark problems rather than complex real-world applications.
• Goal: To generate a unified repository of specialized solution models (a "Set of Pareto Sets") in a single optimization pass. This allows decision-makers to select the most suitable Pareto-optimal solution for a specific, potentially unknown, task setting without re-running the optimization.

2. Methodology

The authors frame the SOS generation as a Multitask Multiobjective Optimization (MTMO) problem and evaluate the performance of current EMT algorithms on this formulation.

A. Problem Formulation

The SOS is defined as the union of Pareto Sets ($PS_k$) for $K$ distinct task settings ($T_k$).

• Objective: Minimize $F_k(x_k) = (f_{k,1}(x_k), \dots, f_{k,m}(x_k))$ for each task $k$.
• SOS Definition: $S = \bigcup_{k=1}^{K} PS_k$, where $PS_k$ contains the non-dominated solutions for task $k$.

B. Real-World Case Studies

The study evaluates three distinct domains, each with multiple task variations:

1. Engineering Design Problems:
   • Four-Bar Truss Design (EO1): Minimizing structural volume and joint displacement under varying loads and materials.
   • Hatch Cover Design (EO2): Minimizing weight and stress constraints under different material strengths and deflection limits.
   • Welded Beam Design (EO3): Minimizing cost and deflection under varying load forces ($P$), lengths ($L$), and Young's modulus ($E$).
2. Inventory Management Problems (IM1, IM2, IM3):
   • Optimizing order quantity ($Q$) and reorder point ($u$) in a continuous review system.
   • Objectives: Minimize total annual cost and minimize stockout frequency/quantity.
   • Variations based on changes in annual demand, lead time standard deviation, and setup costs.
3. Hyperparameter Optimization (HPO):
   • Optimizing LeNet-5 architecture for MNIST classification (even vs. odd digits).
   • Objectives: Maximize classification accuracy and minimize the number of parameters (computational cost).

C. Algorithms Evaluated

The authors compare four algorithms (implemented via the MTO-Platform):

1. MO-MFEA: Multiobjective Multifactorial Evolutionary Algorithm (Implicit transfer via crossover).
2. MO-MFEA-II: An improved version with adaptive transfer parameter estimation.
3. EMT-ET: A method utilizing explicit knowledge transfer (learning search mappings).
4. NSGA-II: A classical single-task multiobjective algorithm used as a baseline.

D. Evaluation Metrics

• Cumulative Hypervolume (CHV): Measures the quality of the union of all obtained Pareto sets.
• Relative Mean-Minimum Distance (RMMD): A novel metric developed to quantify the similarity between Pareto sets of different tasks. A lower RMMD indicates higher similarity, suggesting that the tasks share underlying structures that EMT can exploit.
• Visualization: Analysis of solutions in both Decision Space and Objective Space.

3. Key Contributions

1. Extension of SOS to Real-World Domains: The paper moves beyond machine learning (where SOS was originally proposed) to demonstrate its applicability in engineering, operations research, and hyperparameter tuning.
2. Empirical Validation of EMT: It provides a comprehensive benchmark of current EMT algorithms (MO-MFEA, MO-MFEA-II, EMT-ET) on complex, non-synthetic real-world problems, filling a gap in existing literature.
3. Novel Similarity Metric (RMMD): Introduces a quantitative method to measure the "closeness" of Pareto sets across different tasks, validating the theoretical premise that related tasks share solution structures.
4. Insight into Design-Performance Interplay: Demonstrates how analyzing the SOS allows engineers to understand how optimal designs shift in response to environmental changes (e.g., how weld dimensions must change as load increases).

4. Experimental Results

• Algorithm Performance:
  • MO-MFEA-II consistently achieved the best Cumulative Hypervolume (CHV) results, outperforming MO-MFEA in 6 out of 7 problem sets. This is attributed to its adaptive transfer parameter estimation.
  • EMT vs. Single-Task: EMT methods generally outperformed the single-task baseline (NSGA-II), confirming that leveraging latent synergies between tasks improves solution quality and convergence for SOS generation.
• Visualization Findings:
  • Decision Space: Solutions for different tasks (even with different parameters) tend to cluster in similar regions of the decision space. This confirms that EMT can effectively capture task similarities.
  • Objective Space: The SOS manifests as multiple space curves (one for each task), clearly showing the trade-offs specific to each environment.
• RMMD Analysis:
  • The RMMD values for tasks within the same problem category (e.g., EO3, IM1) were significantly lower than 1 (the baseline for random populations). This confirms high similarity between the Pareto sets of related tasks, validating the effectiveness of the multitask approach.
• Case Study Insight (Welded Beam):
  • By analyzing the SOS for the Welded Beam problem under varying loads ($P=6000, 7000, 8000$ lb), the authors observed a systematic trend: as the load increased, the optimal weld seam length and melt depth slightly increased. This provides actionable engineering intuition that would be difficult to derive by solving each task in isolation.

5. Significance

• Practical Utility: The SOS concept offers a "one-time solve" strategy for dynamic environments. Engineers and managers can pre-compute a repository of solutions and instantly select the optimal configuration when environmental conditions change, saving significant computational time.
• Enhanced Decision Making: It shifts the paradigm from finding a single "best" solution to understanding the spectrum of optimal behaviors across a range of scenarios.
• Theoretical Advancement: The paper validates that EMT is not just a theoretical curiosity for benchmarks but a robust tool for handling the complexity and variability inherent in real-world optimization problems.
• Interpretability: The ability to visualize shifts in Pareto optimal designs helps users understand the causal relationships between design variables and performance metrics under different constraints.