Technical Summary: Structure-Preserving Reconstruction of Convex Lipschitz Functionals

Problem Statement

In applied analysis, economics, finance, and machine learning, many phenomena are modeled by convex functionals (e.g., value functions, risk measures, option prices, loss functionals). However, in practice, these functionals are rarely known in closed form; they are only observed through a finite set of exact pointwise evaluations.

The central problem addressed is whether a convex, $L$-Lipschitz functional $\rho$ defined on a compact convex subset $C$ of a separable Hilbert space $H$ can be reconstructed from finite samples such that the reconstruction:

1. Preserves Convexity: The approximation remains convex for any error tolerance $\epsilon > 0$.
2. Preserves Regularity: The approximation remains $L$-Lipschitz.
3. Achieves Universal Approximation: It can uniformly approximate any such functional over $C$.
4. Is Computationally Feasible: It is exactly implementable by a ReLU-multilayer perceptron (MLP) using only finitely many linear measurements $\langle b, \cdot \rangle_H$ where $b$ lies in a finite-dimensional subspace.

Existing methods often fail one or more of these criteria: deep learning approximations typically violate convexity; explicit formulas involving suprema/infima are not closed-form or require infinite parameters; and finite-dimensional methods do not generalize to infinite-dimensional Hilbert spaces.

Methodology

The paper proposes a three-stage approach to solve this problem, moving from an exact theoretical form to a discretized, neural-network-implementable structure.

1. Exact Primal Form (Convexified Whitney-McShane Extension)

The authors begin by constructing an exact, though in-computable, extension of the data. Using the Fenchel duality theorem, they define a function $f_N$ based on the upper McShane extension. This function is the convex envelope of a finite minimum of Lipschitz functions. While $f_N$ is convex, $L$-Lipschitz, and interpolates the data, its semi-closed form involves a minimization over the probability simplex $\Delta_N$, which is not explicitly computable by standard neural networks.

2. Dual Discretization

To achieve computability while preserving structure, the authors switch to the dual formulation of the extension. They express the functional as the supremum of affine hyperplanes over the dual ball $B_H(0, L)$.

• Finite-Dimensional Projection: Since the dual ball in an infinite-dimensional space is not compact, they project the problem onto a finite-dimensional subspace $V_d \subset H$. This subspace is chosen such that the data points and the domain $C$ are well-approximated within this subspace (controlled by an effective linear dimension $d$).
• Discretization of the Dual: The continuous supremum over the dual ball is replaced by a maximum over a finite $\eta$-net of the projected dual ball.
• Resulting Formula: This yields an explicit formula (Equation 3.1) of the form:
  $$f_{\epsilon, d}(x) = \max_{m} \left( \langle p_m, x \rangle + \min_{n} (\rho(\xi_n) - \langle p_m, \xi_n \rangle) \right)$$
  This formula is a pointwise maximum of affine functions, guaranteeing convexity, and the slopes $p_m$ are bounded by $L$, guaranteeing the Lipschitz property.

3. Neural Network Implementation

The authors demonstrate that this explicit formula can be exactly computed by a neural network architecture.

• ReLU-MLP Construction: The inner minimum and outer maximum operations are implemented using ReLU-MLPs with explicitly bounded size, width, and depth.
• Convex Neural Functionals (CNFs): To ensure that any trainable configuration of the network preserves convexity and Lipschitzness (not just the specific reconstruction configuration), the authors introduce a structured class of networks called Convex Neural Functionals (CNFs).
  • CNFs utilize parametric ReLU (PReLU) activations with non-negative gating parameters.
  • They incorporate max-pooling layers.
  • The architecture enforces non-negative weights in specific layers and specific activation constraints.

Key Contributions and Results

Theoretical Guarantees

1. Theorem 3.1 (Reconstruction Formula): Proves the existence of an explicit finite-sample reconstruction formula that is convex, $L$-Lipschitz, and uniformly $\epsilon$-accurate on $C$. The formula relies on a finite number of linear measurements from a finite-dimensional subspace.
2. Theorem 3.2 (Neural Representability): Establishes that the reconstruction formula from Theorem 3.1 can be exactly computed by a ReLU-MLP. The paper provides explicit bounds on the network complexity (size, width, depth) in terms of the number of samples $N$ and the discretization parameters.
3. Theorem 3.3 (Certifiable Structure): Proves that the broader class of Convex Neural Functionals (CNFs) is "certifiably" convex and Lipschitz. Crucially, every admissible parameter configuration within this class (not just the specific one derived from the data) yields a convex and Lipschitz functional. This provides a principled foundation for training, ensuring the structural constraints are never violated during optimization.

Numerical Validation

The paper includes a "toy" numerical validation:

• Experiment: A CNF was trained to approximate a randomly generated convex ReLU-MLP target.
• Results: The CNF, with approximately 32% of the target's parameters, successfully learned the target.
• Convexity Preservation: Throughout the training process (gradient descent), the CNF iterates remained within the admissible architecture class, preserving convexity. The Jensen gap (a measure of non-convexity) remained at the scale of numerical precision or strictly negative, confirming the theoretical certificate.
• Dimensional Ablation: Experiments across dimensions $d \in \{1, 20, 50, 100\}$ showed consistent performance and parameter efficiency.

Significance and Claims

The paper claims to provide a structure-preserving reconstruction theory for convex Lipschitz functionals from finite data in Hilbert spaces. Its significance lies in bridging the gap between theoretical approximation guarantees and practical, trainable deep learning models:

• Principled Architecture: Unlike standard ReLU-MLPs where convexity is a property of specific weights (and easily broken during training), the CNF architecture guarantees convexity and Lipschitzness by construction for all valid parameters.
• Infinite-Dimensional Applicability: The theory operates in separable Hilbert spaces, making it applicable to functional data and infinite-dimensional problems, unlike methods restricted to finite-dimensional Euclidean spaces.
• Exact Computability: The reconstruction is not an asymptotic limit or an implicit optimization; it is an explicit formula exactly realizable by a neural network with finite measurements.

The authors position this work as a foundation for learning convex functionals in applications such as risk management and mathematical finance, where preserving convexity is a hard constraint rather than a desirable heuristic. They explicitly state that future work will extend these results to Banach spaces and specialized financial applications.