Decision-focused Sparse Tangent Portfolio Optimization

Technical Summary: Decision-focused Sparse Tangent Portfolio Optimization

Problem Statement
Sparse tangent portfolio optimization seeks to construct interpretable, low-cardinality portfolios that maximize the Sharpe ratio (excess return per unit of volatility) along the mean-variance frontier. However, this task faces two primary challenges:

1. Computational Complexity: Enforcing a cardinality constraint (limiting the number of non-zero asset weights) transforms the optimization problem from a smooth continuous task into an NP-hard combinatorial search.
2. Prediction-Decision Misalignment: Conventional "predict-then-optimize" (PFL) pipelines train predictive models using statistical losses (e.g., Mean Squared Error) to forecast asset returns. These models are then fed into a separate optimizer. This approach fails to account for the downstream decision structure; small forecast errors can cascade through discrete asset selection steps, leading to significant suboptimality in the final portfolio performance, particularly under tight sparsity constraints.

Methodology
The authors propose an end-to-end Decision-Focused Learning (DFL) framework that integrates prediction, asset selection, and portfolio re-optimization into a single differentiable pipeline. The core methodology involves three technical innovations:

1. Differentiable Sharpe Ratio Reformulation:
   To enable gradient flow through the optimization layer, the authors reformulate the non-convex Sharpe ratio maximization problem into a Disciplined Parametrized Programming (DPP)-compliant convex program.

   • They utilize a homogenization technique, introducing a scaling variable $t$ and defining $y = tw$.
   • By applying Cholesky factorization ($\Sigma = L_\Sigma L_\Sigma^\top$) to the covariance matrix, they transform the problem into a Quadratically Constrained Quadratic Programming (QCQP) form:
     $$\max_{y,t} y^\top \mu \quad \text{s.t.} \quad \mathbf{1}^\top y = t, \quad \|L_\Sigma y\|_2 \leq 1, \quad t \geq 0$$
   • This formulation satisfies DPP rules, allowing the use of solvers like CVXPYlayers to compute gradients with respect to the predicted mean returns $\hat{\mu}$.

2. Differentiable Soft Top-k Operator:
   To handle the discrete nature of cardinality constraints ($k$ assets) while maintaining differentiability, the authors replace the hard top-k selection with a smooth operator.

   • Assets are ranked based on the magnitude of their transformed weights $|L_\Sigma^\top w|$.
   • A soft top-k mask is generated using a logistic sigmoid function $\sigma$ with a learnable shift parameter $t$. The shift is determined via a one-dimensional bisection search to ensure the sum of the mask values equals exactly $k$.
   • The backward pass utilizes a closed-form vector-Jacobian product, ensuring computational efficiency and numerical stability without explicitly forming large Jacobian matrices.

3. End-to-End Training Objective:
   The predictive model is trained to minimize a task-specific loss rather than a pure forecasting loss. The objective function combines:

   • Regret Loss ($L_{DFL}$): The difference between the optimal portfolio return under ground-truth parameters and the return achieved using the model's predictions.
   • Prediction Loss ($L_{MSE}$): A standard Mean Squared Error term to stabilize training.
   • The total loss is a weighted sum: $L_{Task} = \alpha L_{DFL} + (1-\alpha) L_{MSE}$.

Key Contributions

1. First Application of DFL to Sparse Tangent Portfolios: This work bridges decision-focused learning and sparse portfolio optimization, addressing explicit cardinality constraints that traditional DFL applications have largely avoided.
2. Differentiable Decision Layer: The authors construct a novel layer that combines a DPP-compliant convex reformulation of Sharpe ratio maximization with a smooth top-k operator. This enables gradient propagation through prediction, discrete selection, and re-optimization.
3. Empirical Validation: The framework is validated across four major equity markets (EuroStoxx50, FTSE100, KOSPI200, Nikkei225) with varying asset universes and cardinality levels.

Experimental Results
The proposed DFL framework was compared against:

• Baselines: Historic (mean-variance on historical data) and Prediction-focused Learning (PFL) pipelines.
• Optimization Models: OSCAR (heuristic), SD-relaxation (semidefinite programming), and mSSRM-PGA (proximal gradient).

Findings:

• Superior Performance: The DFL framework consistently achieved competitive or superior out-of-sample Sharpe ratios compared to PFL and Historic baselines.
• Scalability: The performance gains were most pronounced in larger asset universes (e.g., KOSPI200 with 162 assets and Nikkei225 with 208 assets), where the combinatorial complexity of sparse selection is highest.
• Stability: While some PFL variants (specifically SD-relaxation) occasionally matched the mean performance in smaller markets, the DFL method exhibited lower standard deviations across random seeds, indicating greater robustness to training randomness.
• Return Time-Series: The DFL portfolios demonstrated higher realized returns over the test period compared to PFL baselines across most time periods.

Significance and Claims
The paper claims that aligning the learning objective directly with downstream portfolio quality (risk-adjusted performance) is critical for sparse optimization. By internalizing the structure of the constrained optimization problem within the predictive model, the framework mitigates the "predict-then-optimize" mismatch. The authors assert that this approach is particularly effective when the selection problem is combinatorially difficult, as is the case with large asset universes and strict cardinality constraints.

Limitations
The authors acknowledge three main limitations:

1. Computational Overhead: Differentiating through the soft top-k operator introduces computational costs that scale with the asset universe size.
2. Constraint Scope: The current formulation allows short selling and does not incorporate practical constraints like long-only positions, leverage limits, or turnover penalties.
3. Generalizability: The approach is tailored specifically to sparse tangent portfolio optimization; extending it to other cardinality-constrained objectives with different downstream goals requires non-trivial adaptation.