⚡ electrical engineering

LEAF: A Learning-Enabled ADMM Framework for Accelerated Convex Optimization

This paper introduces LEAF, a learning-enabled ADMM framework that utilizes Input Convex Neural Networks to approximate the Moreau envelope of convex objectives, thereby achieving significant computational speedups and maintaining theoretical convergence guarantees while reducing model complexity compared to existing methods.

Original authors: Binh Nguyen, Trinh Tran, Truong X. Nghiem

Published 2026-06-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Binh Nguyen, Trinh Tran, Truong X. Nghiem

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a giant, complex puzzle every single day. The puzzle is a "convex optimization problem," which is just a fancy math way of saying: "Find the best possible solution while following a strict set of rules."

For decades, mathematicians have used a specific tool called ADMM (Alternating Direction Method of Multipliers) to solve these puzzles. Think of ADMM as a very reliable, hardworking robot. It breaks the big puzzle into smaller pieces, solves them one by one, and checks its work. It always finds the right answer, but it can be slow. Every time it hits a tricky piece, it has to stop and do a heavy calculation to figure out the next move.

The authors of this paper, Binh Nguyen, Trinh Tran, and Truong X. Nghiem, asked a simple question: "What if we could teach the robot to recognize the shape of the puzzle pieces so it doesn't have to calculate them from scratch every time?"

They created a new system called LEAF (Learning-Enabled ADMM Framework). Here is how it works, using simple analogies:

1. The "Magic Map" (The Moreau Envelope)

In the robot's job, there is a specific step where it has to find the "smoothest path" down a hill. In math, this is called finding the "proximal operator." Doing this calculation is like climbing a steep mountain to find the lowest valley; it takes a lot of energy and time.

The authors realized that instead of teaching the robot to climb the mountain every time, they could teach it to read a map of the mountain.

The Old Way: The robot calculates the exact height and slope of the mountain for every single step.
The LEAF Way: The robot uses a "Magic Map" (called the Moreau Envelope) that tells it exactly where the bottom of the valley is and how steep the path is, without needing to climb it.

2. The "Smart Student" (Input Convex Neural Networks)

To create this Magic Map, they didn't just use any old computer program. They used a special type of artificial intelligence called an Input Convex Neural Network (ICNN).

Think of a normal AI as a student who might guess the answer. Sometimes the student guesses right, but sometimes they guess something impossible (like a solution that breaks the rules).

The ICNN Student: This student is trained with a strict rulebook. They are forced to only learn shapes that are "convex" (like a bowl shape). This guarantees that the map they draw will never have "holes" or "traps" that would break the math.
The Result: The student learns a scalar value (a single number representing the "cost" or "height" of the path) instead of trying to memorize a giant, complex 3D model. This makes the student much faster to train and requires far fewer examples to learn.

3. The Two New Algorithms

The paper introduces two ways to use this new "Smart Student" and "Magic Map":

MEL-ADMM: This is the standard version. The robot still does its job, but whenever it needs to calculate that heavy "mountain climb," it just asks the Smart Student for the answer. The student gives the answer instantly.
sMEL-ADMM (The Splitter): This is an even faster version. It breaks the puzzle into even smaller chunks. It uses the Smart Student for the hard part and solves the easy parts with simple math formulas. It's like having a team where one person is a genius at the hard stuff, and the rest of the team handles the easy stuff instantly.

Why is this a big deal?

The authors tested their new system on three real-world scenarios:

Microgrid Energy: Managing electricity for a neighborhood with solar panels and batteries.
Entropy Maximization: A complex math problem about probability distributions.
Minimum Volume Enclosing Ellipsoid: Finding the smallest possible oval shape that can wrap around a group of points.

The Results:

Speed: In the energy management test, their new method was 10 times faster than the best existing tools. In some cases, it solved the problem in less than a millisecond.
Accuracy: Even though it was using a "learned" shortcut, the answers were still incredibly accurate (within 0.01% of the perfect answer).
Safety: Because of the special "rulebook" (the ICNN) they used, the solutions never broke the rules. The robot never suggested an impossible solution.

The Bottom Line

The paper claims that by teaching a computer to learn a simplified "map" of the problem (the Moreau Envelope) instead of trying to learn the whole complex solution directly, we can make optimization much faster without losing accuracy or safety.

It's like upgrading from a robot that has to measure every step of a journey to a robot that has a GPS that knows the route perfectly. The destination is the same, but the journey takes a fraction of the time.

Technical Summary: LEAF: A Learning-Enabled ADMM Framework for Accelerated Convex Optimization

Problem Statement

The paper addresses the computational challenges inherent in solving large-scale, parametric convex optimization problems, particularly those involving non-smooth objectives and varying parameters (e.g., in Model Predictive Control). Traditional iterative solvers like the Alternating Direction Method of Multipliers (ADMM) are robust but can be computationally expensive when the "primal update" step (specifically the proximal operator) requires solving a difficult subproblem at every iteration.

Existing Learning-to-Optimize (L2O) approaches attempt to accelerate these solvers but often suffer from three main limitations:

Lack of Structural Guarantees: Many methods fail to guarantee feasibility or convergence, especially on out-of-distribution problem instances.
High Model Complexity: Approaches that learn high-dimensional operators (mapping inputs directly to decision variables) require massive models and data, limiting scalability.
Data Inefficiency: End-to-end learning methods often require extensive training data covering the entire distribution of problem instances to generalize well.

Methodology: The LEAF Framework

The authors propose LEAF (Learning-Enabled ADMM Framework), a structured L2O approach that integrates supervised learning into ADMM iterations to accelerate the primal update while preserving theoretical guarantees.

Core Concept: Learning the Moreau Envelope

Instead of learning the high-dimensional proximal operator directly, LEAF learns a scalar-valued approximation of the Moreau Envelope (ME) of the objective function.

Theoretical Basis: The proximal operator $prox_{\lambda f}(x)$ is related to the gradient of the Moreau envelope $M_{\lambda f}(x)$ by the identity: $prox_{\lambda f}(x) = x - \lambda \nabla M_{\lambda f}(x)$ .
Advantage: By learning the scalar ME function $M_{\lambda f}$ , the framework reduces the learning output dimensionality from $n$ (decision variables) to 1, significantly lowering model complexity and data requirements.

Architecture: Input Convex Neural Networks (ICNNs)

To ensure the learned model respects the convexity of the underlying optimization problem, LEAF employs Input Convex Neural Networks (ICNNs).

Structure: The ICNN architecture constrains weights and activation functions (non-decreasing, convex) to guarantee that the learned function is convex with respect to its input.
Properties: This design ensures the learned ME is convex and smooth (Lipschitz continuous gradient), which are critical for the stability and convergence of the ADMM algorithm.
Training: The model is trained offline using data generated by standard ADMM iterations. The loss function combines output error (ME value) and gradient error, with an additional penalty term to enforce the property that the learned ME is bounded above by the original objective function.

Algorithms

The framework yields two specific algorithms:

MEL-ADMM: Replaces the expensive proximal update in standard ADMM with a forward pass through the trained ICNN to compute the gradient of the learned ME.
sMEL-ADMM (Splitting MEL-ADMM): An extension that splits constraints (e.g., separating affine equality constraints from convex set constraints) to further reduce computational overhead, allowing for closed-form solutions in specific update steps.

Key Contributions

Structured Learning Framework: A general framework that integrates supervised learning into ADMM to bypass expensive primal updates via efficient forward computations, rather than learning the full solution map.
Convexity-Informed Learning: The use of ICNNs to approximate the Moreau envelope, explicitly enforcing convexity and smoothness. This leads to high accuracy with fewer parameters and improved data efficiency compared to generic neural networks.
Theoretical Guarantees: The paper provides rigorous analysis proving that both MEL-ADMM and sMEL-ADMM converge to an optimal solution of a modified problem (defined by the learned ME) and maintain feasibility for the original constraints. The convergence rate is shown to be comparable to classical ADMM.
Scalability and Efficiency: By learning a scalar function rather than a vector-valued operator, the framework significantly reduces computational and memory requirements.

Experimental Results

The authors validate LEAF through three numerical experiments:

Microgrid Energy Management (MPC): A non-smooth optimization problem involving battery storage and load scheduling.
- Result: MEL-ADMM achieved a ~2x speedup over Interior Point Methods (IPOPT) for a 24-hour horizon. sMEL-ADMM achieved a >10x speedup (reaching ~0.5ms solving time) while maintaining feasibility and low optimality gaps.
Entropy Maximization: A problem with randomized linear inequality constraints.
- Result: Across 10,000 randomized instances with varying dimensions ( $n$ up to $10^4$ ), sMEL-ADMM consistently outperformed IPOPT, MadNLP, and Clarabel, demonstrating superior scalability and stability.
Minimum Volume Enclosing Ellipsoid (MVEE): A semidefinite programming problem.
- Result: sMEL-ADMM was approximately 7.5x faster than Mosek and 4x faster than Clarabel for 55 constraints, maintaining this advantage as constraints increased.

In all cases, the proposed methods achieved solutions with optimality gaps below 0.1% (or within specified tolerances) in significantly less time than state-of-the-art solvers, with stable execution times even as problem dimensions or accuracy requirements changed.

Significance and Claims

The paper claims that LEAF represents a significant advancement in Learning-to-Optimize by successfully bridging the gap between data-driven speed and algorithmic reliability.

Structural Integrity: Unlike "black-box" L2O methods, LEAF embeds problem structure (convexity) directly into the learning architecture, ensuring that the learned model behaves predictably and preserves key properties like feasibility.
Efficiency: By shifting the learning target from high-dimensional operators to a scalar Moreau envelope, the framework overcomes the scalability and data-efficiency bottlenecks of previous approaches.
Practical Impact: The framework is particularly suited for real-time and large-scale applications (such as Model Predictive Control) where fast, reliable, and feasible solutions are critical, offering up to an order-of-magnitude speedup over traditional solvers without sacrificing solution quality.

The authors conclude that integrating optimization structures into learning architectures is a viable path toward robust, high-performance optimization, and they identify future work in mixed-integer and distributed convex optimization.