Optimal drift optimizer for non-convex optimization

Technical Summary: Optimal Drift Optimizer for Non-Convex Optimization

Problem Statement
The paper addresses the global minimization of a possibly non-convex objective function $f: \mathbb{R}^d \to \mathbb{R}$. The authors frame this optimization problem within the context of finite-horizon stochastic control. The core criterion is the minimization of a cost functional $J_\lambda(u)$ that balances the expected terminal value of the objective function against a quadratic control cost (drift energy):
$$J_\lambda(u) = \mathbb{E}\left[ f(X_T) + \frac{\lambda}{2T} \int_0^T |u_t|^2 dt \right]$$
where the state process $X_t$ follows the dynamics $dX_t = u_t dt + \sqrt{2\beta} dB_t$. The goal is to identify the optimal feedback control (drift) $u^*_\lambda(t, x)$ that minimizes this criterion and to analyze its properties as an optimization algorithm.

Methodology
The authors approach the problem by solving the associated Hamilton-Jacobi-Bellman (HJB) equation. Rather than relying solely on the classical Hopf-Cole transformation to linearize the HJB equation, they isolate the specific drift selected by the criterion and reorganize the solution to highlight its optimization-theoretic structure.

1. Deterministic Warm-up: The analysis begins with a deterministic open-loop toy model, demonstrating that the quadratic running cost induces a proximal penalization of the endpoint, effectively turning the problem into a Moreau envelope minimization.
2. Stochastic Feedback Solution: The paper solves the stochastic control problem using the Fokker-Planck formulation. The value function $V_\lambda$ is shown to satisfy a backward viscous eikonal equation. Through the Hopf-Cole transform, the value function is expressed via a heat convolution of a Gibbs density.
3. Conditional Terminal Law: The central object of the analysis is the conditional terminal law $\eta_{\lambda,t,x}$ of the optimal process given the current state $(t, x)$. The authors prove this law is a Gibbs measure for a "proximally penalized" energy, which combines the objective function $f$ with a quadratic penalty based on the distance from the current state.

Key Contributions and Results

• Three Exact Representations of the Drift: The optimal feedback $u^*_\lambda(t, x)$ is derived in three equivalent forms, all rooted in the conditional terminal law:

  1. Potential Form: $u^*_\lambda = -\nabla \Phi_\lambda$, where $\Phi_\lambda$ is a heat-regularized Gibbs landscape.
  2. Averaged-Gradient Form: The drift is the Gibbs-weighted average of the gradients $\nabla f(y)$ over the state space.
  3. Barycentric Form: The drift points from the current state $x$ toward the barycenter (weighted average) of the candidate terminal points. This form is gradient-free, relying only on function values.

• Endogenous Scales: Unlike Gaussian homotopy or consensus-based methods where smoothing scales and temperatures are prescribed externally, this criterion selects its own scales endogenously. The search radius is $\sqrt{2\beta(T-t)}$ and the effective temperature is $2\lambda\beta/T$.

• Asymptotic Regimes:

  • Local Consistency ($t \to T$): As the terminal time is approached, the drift recovers a scaled gradient-descent field ($-\frac{T}{\lambda}\nabla f(x)$), ensuring local consistency with first-order methods.
  • Global Selection ($\lambda \to 0$): In the low-temperature regime, assuming a unique global minimizer $x^*$, the conditional terminal law concentrates exponentially on $x^*$, even in the presence of non-global local minima. Consequently, the drift converges to an affine attraction field $-\frac{x - x^*}{T-t}$ pointing directly toward the global minimizer.
  • Non-Commutativity: The paper notes that the limits $t \to T$ and $\lambda \to 0$ do not commute, reflecting the tension between global exploration and local exploitation.

• Laplace Asymptotics: In the nondegenerate case (where the Hessian at the minimizer is positive definite), the authors derive precise Laplace asymptotics for the drift, value function, and the covariance of the conditional terminal law. The covariance scales as $O(\lambda)$, indicating that the uncertainty shrinks as the temperature lowers.

• Gradient-Free Discretization: Leveraging the barycentric representation, the paper proposes a simple gradient-free discretization scheme. This involves sampling candidate terminal points from a Gaussian distribution centered at the current state, reweighting them by the exponential of the negative objective function, and moving toward the weighted average.

Significance and Claims
The paper explicitly states that it does not claim to provide a universal optimal algorithm for non-convex optimization. Instead, its significance lies in:

1. Isolating Optimization Content: It extracts the specific optimization mechanisms inherent in classical linearly solvable control formulas (Hopf-Cole, Schrödinger-Föllmer) and presents them in a coherent framework on $\mathbb{R}^d$.
2. Theoretical Global Selection: It provides a rigorous proof that the exact continuous-time criterion selects the unique global minimizer asymptotically as $\lambda \to 0$, without requiring assumptions that exclude non-global local minima.
3. Structural Insight: It clarifies the role of the conditional terminal law as a Gibbs measure for a penalized energy, unifying the potential, gradient, and barycentric perspectives.
4. Endogenous Parameterization: It highlights that the smoothing and temperature scales are intrinsic to the control criterion rather than external hyperparameters.

The authors position this work as a theoretical foundation that explains the "why" behind certain exploration-exploitation mechanisms, offering a precise global-selection theorem for the exact drift in the asymptotic regime, while acknowledging that finite-time and computational complexity guarantees for practical discretizations remain a subject for future work.