🔢 mathematics

Derivative-free optimization is competitive for aerodynamic design optimization in moderate dimensions

This paper demonstrates through systematic benchmarking on canonical aerodynamic bodies that modern derivative-free optimization methods are competitive with, and often outperform, traditional derivative-based approaches in moderate-to-high dimensions, offering a robust alternative when adjoint-based gradients are unavailable or unreliable.

Original authors: Punya Plaban, Peter Bachman, Ashwin Renganathan

Published 2026-04-17

📖 4 min read🧠 Deep dive

CC BY 4.0

Original authors: Punya Plaban, Peter Bachman, Ashwin Renganathan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to design the perfect airplane wing. Your goal is to make it as aerodynamic as possible (to save fuel) while keeping it strong enough to hold the plane up. This is a bit like trying to find the lowest point in a vast, foggy mountain range, but you can't see the ground, and every time you take a step, you have to send out a drone to map the terrain, which takes a long time and costs a lot of money.

This paper is a race between two different teams of explorers trying to find that lowest point (the best wing shape) using the fewest number of drone flights.

The Two Teams

Team 1: The Map-Makers (Derivative-Based Methods)
This team uses a high-tech, expensive tool called an "Adjoint Solver." Think of this as a magical compass that doesn't just tell you which way is down, but gives you a perfect, detailed map of the slope right where you are standing.

The Pros: If the map is perfect, they can zoom straight to the bottom very quickly.
The Cons: Making the map is hard. If the ground is bumpy, the map gets distorted, and the compass spins wildly. Sometimes, the map takes just as long to make as the drone flight itself. If the terrain is too weird (like during takeoff or landing), the map might not work at all.

Team 2: The Intuitive Hikers (Derivative-Free Methods)
This team doesn't have a compass or a map. Instead, they use a mix of trial-and-error, smart guessing, and building a "mental model" of the mountain based on the few spots they've already visited.

The Pros: They don't need the expensive, fragile map. They are flexible and can handle bumpy, weird terrain where the compass breaks.
The Cons: Without a map, they have to take more steps to figure out which way is down. They might wander a bit before finding the bottom.

The Experiment

The researchers set up a series of challenges using different wing shapes (from simple 2D airfoils to complex 3D wings) with varying levels of complexity (from 4 to 32 "knobs" or variables to turn). They asked: "Who finds the best wing shape faster, given a limited budget of drone flights?"

The Surprising Results

The "Map" isn't always magic: In many cases, the Map-Makers (Team 1) got stuck. Their compasses got confused by the bumpy terrain (mathematical noise in the simulation), and they ended up taking huge, inefficient steps or giving up entirely. They spent a lot of time trying to calculate the perfect slope, only to realize the slope didn't exist in a way they could use.
The Hikers won the race: The Intuitive Hikers (Team 2), especially the ones using a smart "mental model" (called Bayesian Optimization), often found better wing shapes using fewer total drone flights.
- In the simpler 2D tests, they were neck-and-neck with the Map-Makers.
- In the more complex 3D tests (with more "knobs" to turn), the Hikers pulled ahead significantly. They were better at navigating the foggy, high-dimensional mountain without getting lost.

The Big Takeaway

For a long time, the engineering world believed that if you wanted to design a plane, you had to use the expensive, complex "Map-Maker" tools. This paper says, "Not necessarily!"

It turns out that for many real-world design problems, the "Intuitive Hiker" approach is just as good, and often better. It's more robust (less likely to break when things get messy) and can actually save you time and money by avoiding the need for those fragile, expensive maps.

The Analogy in a Nutshell

Imagine you are trying to find the best recipe for a cake.

Team 1 tries to use a complex chemical formula to calculate exactly how much sugar and flour to add. If the formula is slightly off, the cake is ruined, and they have to start over.
Team 2 just bakes a few cakes, tastes them, and uses their brain to guess, "Okay, next time, I'll add a little more vanilla and less salt."

The paper shows that for complex cakes (airplane wings), the "tasting and guessing" team often finds the delicious winner faster than the team trying to do the complex math, especially when the ingredients are tricky.

Conclusion: We don't always need the most complex, high-tech math to design the future of flight. Sometimes, a smart, flexible approach that learns as it goes is the most competitive tool in the toolbox.

1. Problem Statement

Aerodynamic Shape Optimization (ASO) is critical for improving aircraft efficiency (e.g., maximizing lift-to-drag ratios). Traditionally, this field relies on derivative-based optimization methods (e.g., gradient descent, quasi-Newton) coupled with the adjoint method to compute sensitivities efficiently.

However, the adjoint method faces significant practical challenges:

Implementation Complexity: Requires rigorous mathematical treatment and complex numerical implementation.
Robustness Issues: Convergence is highly sensitive to mesh quality and boundary condition changes.
Cost: While theoretically cheap (one forward + one adjoint solve), in practice, adjoint solvers often fail to converge or require multiple evaluations for step-length/Hessian approximation, making them prohibitively expensive.
Inapplicability: In scenarios like Large Eddy Simulations (LES) or hypersonic reacting flows, adjoints may be stochastic or impossible to compute.

The paper addresses the question: Are derivative-free optimization (DFO) methods competitive with derivative-based methods for ASO in moderate dimensions (up to ~50 variables), particularly when adjoint gradients are unavailable or unreliable?

2. Methodology

Experimental Setup

The authors conducted a systematic benchmarking study using high-fidelity Computational Fluid Dynamics (CFD) via the open-source solver SU2.

Test Cases: Three canonical configurations were used:
1. NACA0012: 2D airfoil (Inviscid/Euler).
2. RAE2822: 2D airfoil (Viscous/RANS with Spalart-Allmaras turbulence model).
3. ONERAM6: 3D wing (Inviscid/Euler).
Parametrization: Free-Form Deformation (FFD) was used to define design variables (control point displacements).
Dimensions: Studies were conducted with design spaces ranging from 4 to 32 dimensions.
Constraints: Both unconstrained (drag minimization) and constrained (drag minimization subject to lift, moment, and thickness constraints) variants were tested.

Algorithms Compared

The study compared a suite of widely used optimization algorithms from the SciPy library:

Derivative-Based (Gradient-Based):
- L-BFGS-B (Quasi-Newton)
- SLSQP (Sequential Least Squares Quadratic Programming)
- TNC (Truncated Newton Conjugate Gradient)
- Trust-constrained (Trust-region method)
Derivative-Free (DFO):
- Nelder-Mead (NM): Direct search simplex method.
- COBYLA: Constrained Optimization BY Linear Approximations.
- Bayesian Optimization (BO): Surrogate-based method using Gaussian Process (GP) priors with Expected Improvement (EI) or Scalable Constrained BO (SCBO) acquisition functions.

Performance Metrics

Objective: Drag coefficient ( $C_d$ or $C_D$ ).
Constraints: Lift ( $C_l$ ), Moment ( $C_m$ ), and thickness.
Evaluation Budget: Performance was measured against the total number of CFD evaluations (objective + gradient calls). The authors assumed the cost of an adjoint (gradient) call is roughly equivalent to a forward (objective) call for fairness, though they note adjoints often take longer in practice.
Stopping Criteria: Algorithms were run until convergence tolerances were met or a fixed evaluation budget was exhausted.

3. Key Contributions

Systematic Benchmarking: This is one of the first studies to provide a direct, systematic comparison between classic derivative-based methods and modern DFO methods (specifically BO) across a suite of aerodynamic problems with varying dimensions and physics (Euler vs. RANS).
Challenge to the "Gold Standard": The paper challenges the assumption that gradient-based methods are always superior for ASO, demonstrating that the cost and fragility of adjoint solvers can negate their theoretical advantages.
Feasibility of DFO: It establishes that DFO methods, particularly surrogate-based approaches like BO, are not only viable but often superior in finding the best feasible designs within a fixed CFD budget.
Open Reproducibility: The authors released all code, mesh files, and configuration files (SU2) to ensure full reproducibility of the results.

4. Key Results

Unconstrained Optimization

NACA0012 (2D, Inviscid): Derivative-based methods (L-BFGS-B, TNC) converged quickly in terms of iterations but required many gradient evaluations. BO performed comparably to the best derivative-based methods (Trust-constrained) and outperformed Nelder-Mead in higher dimensions (8 and 16 variables).
RAE2822 (2D, Viscous): In the more complex RANS case, derivative-based methods (SLSQP, TNC, L-BFGS-B) frequently failed to converge or terminated early with high objective values due to line-search failures and ill-conditioned Hessians caused by noisy gradients. BO and Nelder-Mead found the lowest drag values. BO consistently outperformed all gradient-based methods.
ONERAM6 (3D, Inviscid): In the 12-dimensional wing case, BO demonstrated the fastest convergence in terms of function evaluations, finding the lowest drag coefficient compared to all gradient-based solvers.

Constrained Optimization

RAE2822 & ONERAM6: In constrained scenarios, derivative-based methods (especially SLSQP) struggled with feasibility, often experiencing extreme constraint violations or failing to find a feasible solution within the budget.
DFO Superiority: COBYLA and BO consistently found feasible designs with lower objective values.
- BO excelled by balancing exploration and exploitation, using global surrogate models to navigate narrow feasible corridors without getting stuck in local infeasible regions.
- Gradient-based methods often failed because their local linear/quadratic approximations of constraints were inaccurate when the CFD solver was sensitive or non-smooth.

Gradient Norm Analysis

The study observed that even when derivative-based methods terminated, they often did not achieve strict first-order optimality (gradient norm $\approx 0$ ). The gradient norms fluctuated significantly, suggesting that the adjoint solver's lack of convergence or the non-smoothness of the CFD objective prevented the optimizer from finding a true stationary point.

5. Significance and Conclusion

Main Conclusion:
For aerodynamic design optimization in moderate dimensions (up to ~50), derivative-free methods are competitive and often superior to derivative-based methods. Specifically, Bayesian Optimization (BO) and COBYLA consistently outperformed gradient-based solvers (L-BFGS-B, SLSQP, TNC, Trust-constrained) in terms of:

Finding the lowest feasible objective value.
Handling constraints robustly.
Surviving the noise and non-smoothness inherent in high-fidelity CFD solvers.

Implications:

Paradigm Shift: The reliance on adjoint solvers is not always necessary. In cases where adjoints are difficult to implement, unstable, or where the CFD solver is noisy (e.g., LES, RANS with turbulence models), DFO offers a robust alternative.
Practical Viability: Modern DFO strategies, particularly those using Gaussian Process surrogates, provide a practical, "plug-and-play" solution for ASO that avoids the heavy mathematical overhead of adjoint implementation.
Future Work: The authors plan to extend this study to high-dimensional problems ( $>50$ variables) and explore non-stationary surrogates (e.g., Deep Gaussian Processes) to further enhance performance.

In summary, the paper argues that the "gold standard" of gradient-based ASO is not universally applicable and that derivative-free methods offer a promising, often superior, alternative for practical engineering design problems.