A Firefly Algorithm for Mixed-Variable Optimization Based on Hybrid Distance Modeling

This paper proposes a hybrid distance-based Firefly Algorithm (FAmv) that effectively handles mixed-variable optimization problems by integrating continuous, ordinal, and categorical variables into a unified attractiveness mechanism, demonstrating superior performance on benchmark and engineering design tasks compared to state-of-the-art methods.

The Gist

The Big Picture: The "Mixed-Up" Puzzle

Imagine you are trying to solve a massive, complex puzzle. But here's the catch: some of the puzzle pieces are smooth, round numbers (like temperature or speed), while others are distinct, separate categories (like "Red," "Blue," or "Green," or specific integer settings like "1," "2," or "3").

In the world of computer science, this is called a Mixed-Variable Optimization Problem. Most computer algorithms are like specialized tools: some are great at moving smooth, round pieces (continuous variables), and others are great at snapping together distinct blocks (discrete variables). But very few are good at handling a mix of both at the same time without getting confused.

This paper introduces a new version of a popular algorithm called the Firefly Algorithm, specifically designed to handle this "mixed-up" puzzle. The authors call it FAmv.

---

The Original Firefly Algorithm: A Nightlight Dance

To understand the new version, we first need to understand the original Firefly Algorithm.

Imagine a dark forest filled with fireflies.

• The Goal: Every firefly wants to find the brightest light (the best solution).
• The Rule: Fireflies are attracted to brighter lights. The brighter the firefly, the more attractive it is.
• The Movement: If a firefly sees a brighter one nearby, it flies toward it. If it's far away, the attraction is weaker. If it's close, the attraction is strong.
• The Randomness: Sometimes, fireflies just flutter around randomly to explore new areas.

In the original algorithm, all fireflies live in a world of smooth, continuous numbers. They measure distance using a standard ruler (Euclidean distance). If Firefly A is at (1.5, 2.3) and Firefly B is at (1.6, 2.4), the distance is easy to calculate.

The Problem: What happens if Firefly A has a "Red" shirt and Firefly B has a "Blue" shirt? You can't measure the distance between "Red" and "Blue" with a standard ruler. The original algorithm gets stuck because it doesn't know how to compare different types of variables.

---

The New Solution: The "Hybrid Distance" Model

The authors realized they needed a new way to measure "distance" that works for both smooth numbers and distinct categories. They invented a Hybrid Distance Model.

Think of it like a Universal Translator for distance. Instead of just using a ruler, the algorithm now uses a two-part measuring tape:

1. The Ruler Part: Measures the difference between smooth numbers (e.g., how far apart 5.0 and 5.5 are).
2. The Switch Part: Measures the difference between categories (e.g., is "Red" the same as "Blue"? If not, that's a big jump. If it's the same, the jump is zero).

They tested two ways to combine these:

• The Hamming Approach: This is like counting how many switches are different. If two fireflies have different colors or different integer settings, you count a "point" of difference.
• The Gower Approach: This is a more sophisticated method that normalizes everything. It ensures that a huge difference in a smooth number (like 1 vs. 1000) doesn't completely drown out a small difference in a category (like "Red" vs. "Blue"). It balances the score so both types of variables get fair treatment.

How the New Fireflies Move

In the new FAmv algorithm, the fireflies have a split personality when they move:

1. The Smooth Move: For the continuous parts (numbers), they fly just like the original fireflies, gliding smoothly toward the brighter light.
2. The Discrete Move: For the categorical parts (colors, types), they don't "fly" smoothly. Instead, they perform a probabilistic swap.
   • Analogy: Imagine you are wearing a "Red" hat. You see a brighter firefly wearing a "Blue" hat. The algorithm calculates: "How close are we?"
   • If you are very close, there is a high chance you will instantly swap your hat to "Blue" to match the brighter firefly.
   • If you are far away, there is a low chance you will swap. You might keep your "Red" hat and just flutter randomly to see if you can find a better spot.

This ensures that the fireflies respect the rules of the puzzle (you can't have a "half-red" hat) while still learning from the best solutions.

The "Smart" Firefly: Adapting on the Fly

The authors also added a "smart" feature. In the original algorithm, the fireflies have fixed settings for how much they explore (wander) versus how much they exploit (follow the leader).

In FAmv, the fireflies are like smart explorers.

• At the start of the journey: They are very adventurous. They wander far and wide (high exploration) to find where the good stuff might be.
• As they get closer to the finish line: They become more focused. They stop wandering so much and start fine-tuning their position to get the perfect result (high exploitation).

The algorithm automatically adjusts this behavior based on how much "time" (computing power) is left.

Did It Work?

The authors tested their new algorithm on two types of challenges:

1. Math Puzzles (CEC2013): They took standard math problems and forced them to have mixed variables. The new Firefly algorithm competed very well against other top-tier algorithms, often finding better solutions.
2. Real-World Engineering: They tested it on designing things like:
   • A Pressure Vessel (where thickness must be a specific multiple of a number, but radius can be any number).
   • A Welded Beam.
   • A Coil Spring.

In these real-world tests, the new Firefly algorithm was often the winner or tied for the best, proving it can handle messy, real-life engineering problems.

The Takeaway

This paper is about teaching a computer algorithm to be multilingual. Just as a human needs to speak both English and Spanish to navigate a diverse city, this new Firefly Algorithm learned to speak the language of "smooth numbers" and "distinct categories" simultaneously.

By creating a new way to measure distance and a new way to move, the authors created a tool that is better at solving complex, real-world puzzles where variables don't all fit into the same neat box. It's a step forward in making artificial intelligence more flexible and practical for the messy, mixed-up problems we face in engineering and science.

Technical Summary

1. Problem Statement

The paper addresses Mixed-Variable Optimization Problems (MVOPs), a class of real-world optimization challenges where the decision space contains heterogeneous variable types: continuous, ordinal, integer, and categorical variables.

• The Challenge: Most existing metaheuristic algorithms (e.g., standard Particle Swarm Optimization, Differential Evolution, or Genetic Algorithms) are designed primarily for either continuous or discrete spaces.
• Limitations of Current Approaches:
  • Relaxation/Discretization: Common strategies involve relaxing discrete variables into continuous spaces (with rounding) or discretizing continuous variables. These methods often lead to precision loss, suboptimal solutions, or fail to capture the structural nuances of categorical variables.
  • Firefly Algorithm (FA) Specifics: The standard FA relies on Euclidean distance to calculate "attractiveness" between fireflies. Euclidean distance is ill-suited for heterogeneous spaces because it cannot naturally measure the distance between categorical or discrete variables, leading to misleading attraction mechanisms.

2. Methodology: The FAmv Algorithm

The authors propose FAmv (Firefly Algorithm for mixed-variable optimization), a novel adaptation that preserves the original firefly dynamics while introducing type-specific operators and hybrid distance modeling.

A. Hybrid Distance Modeling (Attractiveness)

The core innovation is replacing the standard Euclidean distance with mixed-distance metrics to compute the attractiveness ($\beta$) between two fireflies ($X_i$ and $X_j$). Two specific formulations are proposed:

1. Euclidean-Hamming Distance ($d_{EH}$):

   • Combines the Euclidean distance for continuous components ($d_E$) and the Hamming distance for discrete/categorical components ($d_H$).
   • Formula: $r_{i,j} = \frac{1}{D} (d_E(x^{(c)}_i, x^{(c)}_j) + d_H(x^{(d)}_i, x^{(d)}_j))$.
   • Pros: Simple aggregation; Cons: May not perfectly balance the scale contributions of continuous vs. discrete variables.

2. Gower Distance ($d_G$):

   • A robust metric widely used in cluster analysis for heterogeneous data. It normalizes the contribution of each variable independently before aggregation.
   • For continuous variables, it uses normalized absolute difference; for discrete/categorical, it assigns 0 if values match and 1 if they differ.
   • Pros: Ensures balanced interaction regardless of variable scales or types.

B. Mixed-Variable Movement Mechanism

The movement of a firefly is decoupled into two distinct phases based on variable type:

1. Continuous Update:

   • Follows the standard FA update rule using the calculated attractiveness $\beta$ and a random perturbation term ($\alpha$).
   • $x^{(c)}_i(t+1) = x^{(c)}_i(t) + \beta(r_{i,j})(x^{(c)}_j(t) - x^{(c)}_i(t)) + \alpha(rand - 0.5)$.

2. Discrete Update:

   • $\beta$-Step (Guided Attraction): Instead of a direct vector addition, the discrete component moves probabilistically. If $x^{(d)}_{i,k} \neq x^{(d)}_{j,k}$, the value of firefly $i$ is replaced by firefly $j$'s value with probability $\beta = \exp(-\gamma r^2_{i,j})$. Closer fireflies have a higher probability of exchanging discrete values.
   • $\alpha$-Step (Random Exploration):
     • For integer variables (compact intervals): Uses a perturbation mechanism ($x = INT(x + \alpha \cdot \epsilon)$).
     • For categorical/non-compact variables: Uses a probabilistic replacement mechanism where the probability of replacement is mapped via a sigmoid function of $\alpha$, allowing for global exploration in non-continuous domains.

C. Parameter Adaptation Strategy

To balance exploration (global search) and exploitation (local refinement), the control parameters $\alpha$ (randomness) and $\gamma$ (light absorption/decay) are dynamically adjusted based on the optimization progress:

• Both parameters start at high values to encourage exploration.
• They are linearly decreased as the ratio of consumed function evaluations ($FE$) to the maximum budget ($FEmax$) increases.
• A lower bound (0.01) is enforced to prevent premature convergence.

3. Key Contributions

1. Type-Aware Adaptation: The first unified adaptation of the Firefly Algorithm specifically designed for mixed-variable search spaces without relying on relaxation or discretization heuristics.
2. Hybrid Distance Formulations: Introduction of Euclidean-Hamming and Gower-based distance metrics to model interactions in heterogeneous spaces, enabling coherent search trajectories.
3. Probabilistic Discrete Movement: A novel mechanism for discrete variables that interprets attractiveness as a probability of value exchange, ensuring solution feasibility.
4. Comprehensive Evaluation: Extensive testing on the CEC2013 mixed-variable benchmark (28 functions) and three real-world engineering design problems.

4. Experimental Results

The study compared FAmv (and its variants) against state-of-the-art baselines: DEmv (Differential Evolution), GA (Genetic Algorithm), and PSOmv (Particle Swarm Optimization).

• CEC2013 Benchmark (28 Functions):

  • Unimodal Functions: The FAmvH variant (using Euclidean-Hamming distance) achieved the best results on 4 out of 5 functions, demonstrating superior convergence speed and stability.
  • Multimodal Functions: Performance was heterogeneous. FAmv variants were competitive, achieving the best results on functions F8, F9, F15, F16, F19, and F20.
  • Composition Functions: FAmv variants dominated, achieving the best results on 6 out of 8 composition functions.
  • Robustness: FAmv variants showed lower solution dispersion (standard deviation) across independent runs compared to baselines.

• Engineering Design Problems:

  • Welded Beam (BEAM): FAmvH achieved the lowest fabrication cost.
  • Pressure Vessel (VESSEL): FAmvG (Gower distance) outperformed all other algorithms.
  • Coil Spring (CSD): DEmv performed slightly better, but FAmv variants remained highly competitive with low variability.

• Ablation Study:

  • The study confirmed that the mixed-variable movement mechanism alone provides significant performance gains over the standard FA.
  • Parameter adaptation further enhanced performance, particularly on complex multimodal and composition landscapes.
  • Distance Choice: Hamming-based distances generally offered robust performance across categories, while Gower-based distances required adaptive parameters to compensate for their bounded nature (0 to 1) which can lead to overly strong attraction.

5. Significance and Conclusion

This paper provides a significant step forward in applying nature-inspired metaheuristics to complex, real-world optimization problems.

• Theoretical Impact: It demonstrates that the Firefly Algorithm can be effectively generalized beyond continuous spaces by redefining the distance metric and movement operators to respect variable heterogeneity.
• Practical Impact: The method avoids the pitfalls of relaxation strategies, preserving the integrity of categorical and discrete constraints. Its success on engineering design problems (Vessel, Beam, Spring) validates its applicability in industrial settings.
• Future Directions: The authors note that future work should focus on better normalizing the distance contributions when continuous variables span very large ranges and developing more advanced adaptive mechanisms to further stabilize the exploration-exploitation trade-off.

In summary, FAmv offers a robust, flexible, and high-performance solution for mixed-variable optimization, outperforming or matching state-of-the-art algorithms across a wide spectrum of synthetic and real-world benchmarks.