Duality and DeepMartingale for High-Dimensional Optimal Switching: Computable Upper Bounds and Approximation-Expressivity Guarantees

1. Problem Formulation

The paper addresses the finite-horizon optimal switching problem under uncertainty.

• Context: A controller manages a system that can operate in one of $J$ distinct regimes. Switching between regimes incurs a cost $l_{ij}(t)$. The system generates running payoffs $f^i(t)$ depending on the current regime and a terminal payoff $\Phi^i$.
• Setting: The problem is formulated on a general filtration with discrete intervention dates ($t_n$) but allows for continuous-time observations between these dates. This covers both intrinsically discrete models and discretely exercisable continuous-time models.
• Challenge: In high dimensions, the problem is computationally intractable due to the "curse of dimensionality." At each decision date, the controller must optimize jointly over the intervention time and the post-intervention regime, while continuation values are coupled across all regimes. Traditional grid-based methods (PDE/QVI) fail in high dimensions, and existing regression-based dynamic programming methods lack rigorous high-dimensional approximation guarantees or computable upper bounds.

2. Methodology

The authors propose a DeepMartingale-based dual framework that combines stochastic duality theory with deep learning.

A. Dual Representation and Strong Duality

• Regime-Decision Reformulation: The authors transform the switching control problem into an equivalent regime-decision problem, where the controller selects a sequence of regimes for each time interval $[t_m, t_{m+1})$.
• Martingale Penalties: They introduce a family of martingale penalties to derive a dual representation. Unlike previous works where the upper bound depended on the unknown value function, this paper derives a fully computable dual upper bound.
• Doob Martingale Characterization: The minimal penalty is characterized by the Doob martingales of the continuation values.
• Strong Duality: The paper proves strong duality, showing that the primal value equals the essential infimum of the expected dual upper bound over all admissible martingales.
  $$Y^i_{t_n} = \text{ess inf}_{M \in (\mathcal{M}_{n,N})^J} \mathbb{E}_{t_n} \left[ \hat{U}^i_n(M) \right]$$
  where $\hat{U}^i_n(M)$ is the dual functional involving running payoffs, switching costs, and martingale increments.

B. DeepMartingale Solver

• Parametrization: The Doob martingales (which are optimal penalties) are parametrized using Deep Neural Networks. Specifically, the martingale integrands $Z^i_t$ are approximated by feedforward neural networks $z_{\theta}(t, X_t)$.
• Training Objective: The solver minimizes the dual upper bound (or an $L^2$-surrogate loss) backward in time.
  • Loss (D1): Minimize the expectation of the dual upper bound.
  • Loss (D2): Minimize the $L^2$ distance between the dual upper bound and a known lower bound (surrogate).
• Reference Regime Strategy: To reduce computational cost, the authors propose training the martingale network only for a single reference regime ($i_0$). Theoretical results suggest this single set of martingales is simultaneously optimal (or near-optimal) for all regimes, significantly lowering memory usage.

C. Primal Benchmarking

To validate the dual upper bounds, the authors implement a primal deep policy-based approach (DeepPD) to compute feasible lower bounds. This involves parameterizing the switching policy with a softmax network and evaluating it out-of-sample to generate empirical upper–lower gaps.

3. Key Contributions

1. Computable Dual Upper Bounds: The paper establishes the first fully computable martingale-dual theory for finite-horizon optimal switching with discrete interventions. The resulting upper bound does not depend on unknown value functions.
2. Extension of DeepMartingale: The framework extends the DeepMartingale method (previously limited to optimal stopping) to the more complex optimal switching setting, handling the coupling of regimes and the maximum operator over multiple choices.
3. Convergence and Expressivity Guarantees:
   • Convergence: The authors prove convergence of the solver under both the upper-bound loss and the $L^2$-surrogate loss.
   • Curse of Dimensionality Avoidance: Under specific structural assumptions (affine Itô diffusions, Lipschitz coefficients, and polynomial growth), they prove that for any target accuracy $\epsilon$, there exist neural networks with size at most $c d^q \epsilon^{-r}$ (where $c, q, r$ are independent of dimension $d$). This implies the method avoids the curse of dimensionality.
4. Delta-Hedging Interpretation: The learned dual martingale integrand is shown to correspond to the delta hedge ratio in a financial context, providing a practical strategy for risk management.

4. Numerical Results

The authors tested the method on Brownian and Brownian–Poisson models with dimensions up to $d=100$.

• Performance: The DeepPD (Dual + Primal) framework produced small upper–lower gaps (often close to 0.1) across all dimensions, demonstrating high accuracy.
• Scalability: Unlike competing methods (e.g., DeepOSJ) which ran out of memory at $d=20$, the proposed method remained stable and accurate up to $d=100$.
• Hedging: The learned martingale yielded robust delta-hedging strategies with low tail risk (CVaR).
• Switching Boundaries: Visualizations of the switching regions showed that the dual-induced rules captured the correct switching geometry, producing smoother and more coherent boundaries compared to primal-only or other deep learning baselines.

5. Significance

This work bridges the gap between stochastic duality theory and high-dimensional machine learning for optimal control.

• Theoretical Impact: It provides rigorous approximation guarantees for deep learning solutions to optimal switching, a problem previously lacking such theoretical backing in high dimensions.
• Practical Impact: It offers a scalable, computable tool for pricing and hedging complex real options (e.g., energy storage, natural resource management, firm entry/exit) where traditional methods fail due to dimensionality.
• Methodological Innovation: The "reference-regime" training strategy and the integration of dual upper bounds with primal lower bounds create a robust framework for verifying solution quality in high-dimensional stochastic control.