📈 economics

Multi-objective Portfolio Optimization Via Gradient Descent

This paper introduces a flexible, gradient descent-based benchmark framework for multi-objective portfolio optimization that leverages automatic differentiation to handle complex constraints and diverse objectives, demonstrating competitive performance against standard solvers across various experimental scenarios.

Original authors: Christian Oliva, Pedro R. Ventura, Luis F. Lago-Fernandez

Published 2026-08-15

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Christian Oliva, Pedro R. Ventura, Luis F. Lago-Fernandez

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the captain of a massive ship, but instead of sailing on water, you are navigating a sea made entirely of money. Your goal is to get from point A to point B with the most treasure possible, but there's a catch: the ocean is full of storms (risk), and you have a strict map (regulations) that says you can't carry too much gold in one chest, and you must stay close to a specific lighthouse (a benchmark index). This is the world of portfolio optimization. For decades, the standard way to steer this ship was to use a rigid, mathematical compass called "Modern Portfolio Theory." It worked well for calm waters, but when the sea got choppy with complex rules or conflicting goals—like wanting maximum treasure and minimum risk and strict safety limits—the old compass often got stuck or broke.

Enter a new kind of navigation tool: Gradient Descent. Think of this not as a rigid map, but as a hiker feeling their way down a foggy mountain. The hiker (the computer) takes small steps, constantly checking which way is steepest downhill (towards less risk or more profit) and adjusting their path. Usually, this "hiking" method is used to teach computers how to recognize cats in photos or drive cars. But what if we taught the computer to hike through the financial mountains instead? That is exactly what this paper explores. The researchers built a new framework that uses this "hiking" technique, powered by modern AI tools, to solve the messy, multi-goal problems of investing that old methods struggle with. They wanted to see if this flexible, step-by-step approach could build a better, safer, and more rule-abiding treasure chest than the traditional, rigid calculators.

The Paper's Journey: Hiking the Financial Mountains

The authors, a team from the Autonomous University of Madrid and March Asset Management, set out to test if this "hiking" method (Gradient Descent) could handle the complex job of Multi-Objective Portfolio Optimization (MPO). In the real world, investors don't just want "high returns." They want high returns while keeping risk low, while following strict laws (like the European UCITS rules), while making sure they don't put all their eggs in one basket, and while tracking a specific market index closely. Traditional tools often struggle when you ask them to juggle all these balls at once, especially if the rules are tricky or the data is huge.

The researchers built a digital playground using data from the S&P 500 (a list of about 500 of the biggest US companies) from the year 2020. They didn't just build one test; they built six different scenarios, ranging from simple "find the best path" tasks to incredibly complex "find the best path while dodging rocks, staying within speed limits, and carrying a specific weight" challenges.

The Simple Tests: Walking the Straight Line
First, they tested the basics. In one scenario, they asked the computer to simply find the path with the best balance of risk and reward (maximizing the "Sharpe ratio"). They compared their new "hiking" method against a famous, rigid mathematical solver called CVXPY. The result? The hiker and the rigid calculator arrived at almost the exact same spot. The weights of the assets (how much money went into each stock) were nearly identical, differing only in the sixth decimal place. This proved that the new method was just as accurate as the old, trusted tools for simple jobs.

In a second test, they asked the computer to minimize "extreme losses" (a concept called CVaR). They compared this against another tool called SKFOLIO. Here, the "hiker" took a slightly different path than the rigid calculator, picking different stocks and assigning them slightly different weights. However, when they checked the final result—the actual amount of extreme loss avoided—the two methods were practically twins. The difference was so tiny (in the fifth decimal place) that for all practical purposes, the hiker found the same destination, just by feeling the ground rather than following a pre-drawn map. This was a big deal because the "extreme loss" problem is mathematically messy and hard for rigid tools to solve perfectly; the hiker handled it with ease.

The Complex Tests: Juggling and Obstacle Courses
Then, things got spicy. The researchers added layers of complexity, like a video game adding new levels.

The Rule-Follower: They asked the computer to minimize extreme losses while obeying UCITS regulations. These are strict European rules that say, for example, "No single stock can take up more than 10% of your portfolio," and "The total of your big bets (over 5%) can't exceed 40%." Traditional tools often choke on these rules because they make the math "non-convex" (a fancy way of saying the map has holes and cliffs that break the rigid calculators). But the hiker? It just kept walking, adjusting its steps until it found a path that stayed within the lines. The result was a portfolio that perfectly followed the rules while still keeping risk low.
The Tightrope Walker: In another test, they added a Tracking Error constraint. This means the portfolio's performance had to stay very close to the S&P 500 index (like a tightrope walker staying on the wire). The hiker managed to minimize extreme losses and stay glued to the index, proving it could balance conflicting goals without falling off.
The Ultimate Challenge: The final, most complex test asked the computer to do everything at once: maximize returns, minimize extreme losses, follow the UCITS rules, keep a minimum number of stocks, ensure no stock was too small, and stay close to the index. The computer successfully navigated this chaotic obstacle course. It selected 23 stocks, kept every single one above a 1% weight, ensured the "big bets" stayed under the 40% limit, and delivered a portfolio that outperformed the standard index in risk-adjusted returns.

The "Mask" Mystery
In a final, creative test, the researchers created "masks"—groups of stocks that had to stay under a certain total weight (like saying, "The tech group can't have more than 27% of your money"). The hiker successfully distributed the money so that no group broke its limit, while still minimizing the overall risk of the portfolio. It showed that the method could handle custom, arbitrary rules that would be a nightmare for older software.

What This Means for the Future

The paper doesn't claim to have discovered a "magic bullet" that guarantees you'll get rich. Instead, it suggests something more practical: flexibility. The authors show that using Gradient Descent with automatic differentiation (a tool that lets computers calculate slopes instantly) is a powerful, scalable way to solve investment problems that are too messy for traditional math.

The key finding is that this method is competitive with the best existing tools for simple tasks but superior for complex, multi-rule scenarios. It doesn't just find an answer; it finds a good answer that respects all the messy, real-world constraints investors actually face. The researchers emphasize that the "hiker" needs a guide (the user) to set the right "weights" (how much to care about risk vs. reward), but once those settings are chosen, the computer can handle the heavy lifting of balancing dozens of conflicting goals simultaneously.

In the end, this paper opens the door for a new kind of financial tool—one that is as adaptable as a human hiker, capable of navigating the foggy, rocky terrain of modern investing where rigid maps fail. It suggests that by borrowing techniques from the world of artificial intelligence, we can build portfolios that are not just mathematically sound, but also practically robust and ready for the real world.

Technical Summary: Multi-objective Portfolio Optimization Via Gradient Descent

Problem Statement
Traditional portfolio optimization (PO), rooted in Modern Portfolio Theory (Markowitz, 1952), often relies on quadratic programming or evolutionary algorithms. While effective for single-objective problems, these approaches struggle with scalability and flexibility when addressing complex, real-world scenarios involving large datasets, non-convex constraints, and multiple conflicting objectives (e.g., simultaneously minimizing Conditional Value-at-Risk (CVaR) and maximizing the Sharpe ratio). Furthermore, exact solution algorithms face difficulties when problems become NP-hard due to non-convex constraints or high dimensionality. The authors identify a gap in the literature where Machine Learning (ML) techniques, specifically gradient-based methods, are underutilized in Multi-objective Portfolio Optimization (MPO) compared to population-based metaheuristics like Genetic Algorithms (GAs) and Swarm Algorithms (SAs).

Methodology
The paper proposes a benchmark framework for MPO utilizing gradient descent with automatic differentiation, implemented via deep learning libraries (TensorFlow and PyTorch). The core methodology involves:

Differentiable Formulation: Instead of solving a static quadratic program, the portfolio weights are treated as the output of a differentiable transformation of learnable "pre-weights" ( $z$ $z$ ).
- To ensure weights sum to 1 and are non-negative, the authors employ the softmax function.
- To address the limitation of softmax (which rarely produces sparse solutions), they utilize sparsemax (Martins and Astudillo, 2016), which truncates small values to zero, facilitating sparse portfolio construction.
Loss Function Construction: The optimization objective is defined as a loss function $L$ $L$ to be minimized. This function combines:
- Primary Objectives: Such as maximizing the Sharpe ratio (formulated as $-(R - r_f)/\sigma(R)$ ) or minimizing CVaR.
- Constraints as Penalties: Regulatory and practical constraints are incorporated as penalty terms added to the loss function, weighted by hyperparameters ( $\lambda$ ).
Handling Non-Differentiable Constraints: To maintain compatibility with gradient descent, the authors introduce specific techniques for non-differentiable operations:
- Threshold Constraints: Implemented using ReLU (e.g., $[w - \alpha]_+$ ) to penalize weights exceeding a limit.
- Conditional Masks: For binary decisions (e.g., asset selection), a "round sigmoid" function is used. During backpropagation, the gradient of the smooth sigmoid is used instead of the non-differentiable round function, allowing the optimizer to learn effective masks.
Experimental Setup: The framework is evaluated on S&P 500 assets (2019–2020) across six scenarios ranging from single-objective optimization to complex multi-objective problems with multiple constraints (UCITS regulations, tracking error limits, active asset counts, and subset weight limits).

Key Contributions

A Flexible MPO Framework: The authors provide an open-source, extensible benchmark that allows researchers to model any financial objective and constraint within a unified gradient descent framework.
Differentiable Constraint Handling: The paper details specific techniques (sparsemax, ReLU penalties, and surrogate gradients for masks) to enforce complex constraints (like UCITS directives and asset count limits) without breaking the differentiability required for backpropagation.
Comparative Analysis: The study benchmarks the proposed method against standard solvers (CVXPY for convex problems, SKFOLIO for CVaR) and metaheuristics, demonstrating that gradient descent can achieve competitive results while offering greater modeling flexibility.

Results
The experimental results across six cases demonstrate the framework's efficacy:

Case 1 (Maximize Sharpe Ratio): The gradient descent (GD) optimizer produced results nearly identical to the exact convex solution provided by CVXPY, with a Mean Squared Error (MSE) of $3.6179 \times 10^{-5}$ in weight vectors across 100 simulations.
Case 2 (Minimize CVaR): While the specific asset weights differed from the SKFOLIO exact solver (due to CVaR being non-convex), the GD optimizer achieved a CVaR value differing only in the 5th decimal place, effectively approximating the global solution.
Case 3 (Minimize CVaR with UCITS): The framework successfully satisfied simplified UCITS constraints (e.g., no single asset >10%, sum of assets >5% <40%) while minimizing CVaR, a scenario where exact solvers like CVXPY cannot find a solution due to non-DCP (Disciplined Convex Programming) violations.
Case 4 & 5 (Multi-Objective with Constraints): The optimizer successfully balanced conflicting objectives (maximizing Sharpe ratio and minimizing CVaR) while adhering to tracking error limits, minimum asset weights, and active asset counts. Case 5 showed a significant improvement in the Sharpe ratio (0.0516 vs. 0.0310 in Case 4) with only a marginal increase in CVaR, demonstrating effective trade-off management.
Case 6 (Subset Constraints): The method effectively minimized volatility while strictly adhering to weight limits on predefined asset subsets (masks), with deviations below 0.01%.

Significance and Claims
The authors position this work not as a novel optimization strategy intended to outperform all existing techniques, but as a practical and extensible tool for researchers and practitioners. The primary significance lies in the framework's ability to:

Model Complexity: Handle arbitrary combinations of objectives and constraints (including regulatory rules like UCITS) that are difficult to formulate for traditional solvers.
Leverage Automatic Differentiation: Utilize the computational simplicity and adaptability of deep learning frameworks to solve large-scale portfolio problems.
Facilitate Research: Provide a reproducible baseline (via open-source code) for future studies in multi-objective portfolio optimization, potentially serving as a foundation for comparing gradient-based methods against metaheuristics or integrating forward-looking models like Black-Litterman.

The paper concludes that the proposed gradient descent benchmark is a robust, reliable, and flexible alternative for constructing portfolios in real-world conditions where multiple, conflicting objectives and strict constraints must be managed simultaneously.

The Paper's Journey: Hiking the Financial Mountains

What This Means for the Future

More like this