📊 statistics

Deep Neural Networks for Mean-Field Stochastic Optimal Control with Applications to Mean Variance Optimization

This paper proposes three deep learning-based algorithms to numerically solve high-dimensional mean-field stochastic optimal control problems derived via the stochastic maximum principle, demonstrating their accuracy and efficiency through applications to mean-variance portfolio selection.

Original authors: Hamid OULGHAZI, Mustapha Hamchichi, Sidi Mohamed Douiri

Published 2026-10-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Hamid OULGHAZI, Mustapha Hamchichi, Sidi Mohamed Douiri

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to steer a ship through a storm where the wind and waves do not just react to your current position, but also to the average behavior of every other ship in the ocean. This is the essence of a complex mathematical challenge known as mean-field stochastic control. It appears in fields ranging from physics, where it helps describe how gases behave, to finance, where it models how the collective actions of investors influence market prices. The core difficulty lies in the sheer number of variables involved. When a system depends on the average behavior of millions of individual parts, the mathematics required to find the perfect strategy to control it becomes so vast that traditional computers simply cannot handle the load. This is a problem known as the "curse of dimensionality," where the effort needed to solve a problem explodes as the number of factors increases, rendering many real-world scenarios impossible to calculate with standard methods.

A team of researchers from Université Moulay Ismail de Meknes has developed a new way to tackle this problem using artificial intelligence. Instead of trying to calculate every single possibility, they trained deep neural networks—computer systems modeled after the human brain—to learn the best path forward. Their work focuses on a specific type of control problem where the rules of the game change based on the average state of the system. By applying a mathematical principle known as the stochastic maximum principle, which provides a set of rules for finding the best possible outcome in uncertain environments, the researchers transformed the original difficult problem into a new, equivalent format. This new format allows them to treat the search for the best strategy as a learning task, where the computer adjusts its internal settings to minimize errors until it finds the optimal solution.

The researchers tested three different approaches to see which worked best. The first method used a single neural network to guess the necessary adjustments at each step, then used a separate mathematical tool to refine the final answer. The second method was used in a specific scenario where the perfect solution could be written down as a simple formula; in this case, the neural network only had to learn the remaining parts of the puzzle, making the process very fast and accurate. The third and most flexible method used two neural networks working together. One network guessed the adjustments, while the other guessed the control actions, and both were trained simultaneously. Crucially, this third approach included a special rule in its training process that forced the networks to obey the fundamental laws of optimality, even when no simple formula existed for the solution.

To see if these methods actually worked, the team applied them to a classic problem in finance: managing a portfolio of investments to balance risk and return. They simulated a market with a safe asset and a risky one, asking the computer to find the best way to invest money over a specific period to maximize returns while keeping risk low. The simulations showed that all three methods could find a solution, but they performed differently. The method that relied on a known formula was the most precise, closely matching the theoretical ideal. The method using two networks was nearly as accurate but had a significant advantage: it did not need a pre-existing formula to work. It could handle situations where the perfect strategy was too complex to write down on paper. The single-network method worked as well, though it required more time to converge on a good answer.

The results demonstrate that deep learning can successfully navigate high-dimensional control problems that were previously out of reach. The study confirms that by reformulating the problem and using neural networks to approximate the solution, it is possible to achieve high accuracy and computational efficiency. The most significant finding is that the two-network approach offers a powerful, general-purpose tool. It can solve complex, real-world problems where the optimal strategy is hidden and cannot be expressed in a simple equation, opening the door to solving similar challenges in economics, engineering, and physics that were once considered too difficult to compute.

Technical Summary: Deep Neural Networks for Mean-Field Stochastic Optimal Control

Problem Statement
This research addresses high-dimensional optimal control problems governed by mean-field stochastic differential equations (SDEs). Unlike classical stochastic control, where coefficients depend solely on the state process, mean-field models incorporate dependencies on the probability distribution (or expectation) of the state. The specific problem formulated is the minimization of a cost functional of the form:
$\inf_{u \in U} \mathbb{E} \left[ \int_0^T h(t, x_t, \mathbb{E}[\phi(x_t)], u_t) dt + g(x_T, \mathbb{E}[\chi(x_T)]) \right]$
subject to a mean-field SDE where the drift and diffusion coefficients depend on the state $x_t$ and its expectation $\mathbb{E}[\psi(x_t)]$ . The authors note that such mean-field dependencies introduce time inconsistency and nonlinear performance criteria, distinguishing them from classical linear expectation-based problems.

A primary challenge identified is the "curse of dimensionality," which renders traditional numerical methods (e.g., finite difference or Monte Carlo methods based on grid discretization) impractical for high-dimensional settings due to exponential growth in computational complexity.

Methodology
The authors leverage the Stochastic Maximum Principle (SMP) to transform the original control problem into an equivalent variational problem. By applying the SMP, they derive an extended Hamiltonian system expressed as a coupled Forward-Backward Stochastic Differential Equation (FBSDE) coupled with a minimum condition on the Hamiltonian.

The core methodological innovation is the reformulation of the optimality conditions into a new control problem where the goal is to minimize a quadratic terminal cost:
$\inf_{\hat{p}_0, \{\hat{q}_t\}} \mathbb{E} \left[ | \hat{p}_T - g_x(\hat{x}_T, \mathbb{E}[\chi(\hat{x}_T)]) - \mathbb{E}[g_y(\hat{x}_T, \mathbb{E}[\chi(\hat{x}_T)])]\chi_x(\hat{x}_T) |^2 \right]$
subject to the dynamics of the state $\hat{x}$ and the adjoint process $\hat{p}$ . If this terminal cost converges to zero, the solution corresponds to the optimal control of the original problem.

To solve this high-dimensional FBSDE numerically, the paper proposes three distinct Deep Neural Network (DNN) based algorithms:

Algorithm 1 (Single DNN with Numerical Optimization):
- A single DNN approximates the control process $\hat{q}_t$ .
- The optimal control $\hat{u}_t$ is determined at each time step by numerically solving the minimization condition $\arg\min_v H(\cdot)$ using optimization techniques like L-BFGS or SLSQP.
- This approach treats the minimization condition as an external optimization step within the training loop.
Algorithm 2 (Single DNN with Explicit Control):
- Applicable when the function $\bar{H}$ (the minimized Hamiltonian) and the optimal control $\hat{u}_t$ admit an explicit analytical representation.
- A single DNN approximates $\hat{q}_t$ .
- The system is solved as a standard FBSDE without the need for iterative numerical minimization at every step, significantly reducing computational cost.
Algorithm 3 (Dual DNN with Penalty Term):
- Designed for cases where an explicit expression for the optimal control is unavailable.
- Two DNNs are employed simultaneously: one to approximate $\hat{q}_t$ and another to approximate $\hat{u}_t$ .
- The first-order optimality condition ( $H_u = 0$ ) is enforced directly by adding a penalty term $\lambda \int_0^T |H_u|^2 dt$ to the loss function. This allows the network to learn the optimal control without requiring a closed-form solution.

Key Contributions

Reformulation: The paper successfully reformulates mean-field stochastic optimal control problems into a tractable variational problem suitable for deep learning, utilizing the Stochastic Maximum Principle.
Algorithmic Diversity: It introduces three specific algorithms tailored to different levels of information availability regarding the optimal control (from fully explicit to fully implicit).
Handling Mean-Field Terms: The methodology explicitly accounts for the mean-field nature of the problem, where coefficients depend on the expectation of the state process, extending deep learning applications beyond standard SDEs.
Penalty-Based Constraint: Algorithm 3 introduces a novel approach to handling the maximum principle constraint in high dimensions by embedding the first-order condition directly into the loss function, avoiding the need for explicit control formulas.

Results
The proposed methods were evaluated using the Mean-Variance Portfolio Selection problem, a classic application in financial mathematics. The simulations involved a market with a risk-free and a risky asset, with parameters set to standard values (e.g., risk-free rate 0.05, volatility 0.20).

Convergence: All three algorithms demonstrated convergence toward the analytical optimal value of the cost functional.
Accuracy:
- Algorithm 2 achieved the highest accuracy and smallest relative error, attributed to the simplification provided by the explicit optimal control.
- Algorithm 3 provided highly accurate results, closely matching the analytical solution, despite not requiring an explicit control formula. The inclusion of the penalty term was credited with guiding the training toward solutions that satisfy optimality conditions more robustly.
- Algorithm 1 showed the lowest accuracy among the three, with larger deviations from the analytical optimum, largely due to the computational overhead and potential errors introduced by the iterative numerical minimization at each step.
Efficiency: The study highlights that while Algorithm 2 is the most efficient when explicit controls exist, Algorithm 3 offers a robust and flexible alternative for general high-dimensional problems where explicit solutions are unavailable.

Significance and Claims
The paper claims that the proposed deep learning framework effectively overcomes the curse of dimensionality in mean-field stochastic optimal control. It asserts that the reformulation into a quadratic terminal cost problem provides a "more tractable framework" for verifying optimality.

The authors position Algorithm 3 as the most general and significant contribution, as it enables the solution of high-dimensional stochastic control problems without relying on the existence of explicit analytical expressions for the optimal control. They conclude that these methods show "good accuracy and computational efficiency in high dimensions," making them viable for complex applications like the mean-variance portfolio optimization where traditional methods fail. The work bridges the gap between theoretical stochastic control (via the SMP) and modern computational techniques (via DNNs), offering a practical toolkit for high-dimensional mean-field problems.

Technical Summary: Deep Neural Networks for Mean-Field Stochastic Optimal Control

More like this