📈 economics

CBGA-SAC: A CVaR-Constrained Deep Reinforcement Learning Framework for Portfolio Optimization

This paper proposes CBGA-SAC, a unified deep reinforcement learning framework that integrates CNN-BiLSTM-Attention for temporal risk extraction and Graph Attention Networks for dynamic asset correlations to optimize portfolio rebalancing under Conditional Value-at-Risk constraints, demonstrating superior risk-adjusted returns and robustness during market regime shifts compared to traditional and baseline deep learning models.

Original authors: R Lavanya, Shivani Agarwal, Vinay Dwivedi, Pavan Kumar Varma Kothapalli, Venkata Sandeep Dhullipalla, Megha Sharma, Nisha Kundu

Published 2026-10-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: R Lavanya, Shivani Agarwal, Vinay Dwivedi, Pavan Kumar Varma Kothapalli, Venkata Sandeep Dhullipalla, Megha Sharma, Nisha Kundu

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The world of finance is often imagined as a place of steady growth, but for those who manage money, the reality is a landscape of constant, unpredictable change. Markets do not move in straight lines; they surge, stall, and sometimes crash in ways that defy simple prediction. The core challenge for investors is not just finding assets that grow, but doing so without exposing their capital to catastrophic loss. Traditional methods for managing this risk often rely on historical averages, assuming that the future will look much like the past. However, when markets shift suddenly, these old maps fail. To navigate this, researchers are turning to artificial intelligence, specifically a type of learning that mimics how humans learn from trial and error, hoping to build systems that can adapt to new dangers in real time.

In this context, a team of researchers has developed a new framework called CBGA-SAC, designed to make smarter decisions about how to split money across different stocks. The system was built to solve a specific problem: most existing tools treat the job of predicting risk and the job of deciding how much to invest as two separate tasks. They might use one computer program to guess how volatile a stock will be, and then feed that guess into a second, separate program to decide the portfolio mix. This separation means that if the first program makes a mistake, the second one never knows to adjust. Furthermore, many systems ignore how stocks influence one another, treating them as isolated islands rather than a connected network where a shock to one can ripple through the others.

The researchers proposed a unified solution that combines three distinct capabilities into a single, flowing process. First, the system uses a deep learning structure to scan the history of stock prices, looking for both short-term patterns and long-term trends that human analysts might miss. It pays special attention to the most critical moments in the data, much like a reader skimming a long report to find the most important sentences. Second, instead of assuming that the relationship between stocks is fixed, the system builds a dynamic map of how they are connected. It learns, day by day, which stocks tend to move together and which do not, updating this map as market conditions change. Finally, this combined understanding of time and connection is fed into a reinforcement learning agent. This agent acts like a trader that learns by doing, constantly adjusting its strategy to maximize profit while strictly avoiding the worst possible losses.

To train this system, the researchers used a vast dataset of thirty major American stocks, covering a period that included a significant market downturn in 2022 followed by a recovery. This specific timeframe was chosen because it represented a "regime shift," a sudden change in the rules of the market that would test whether the system could adapt or if it would crumble under pressure. The system was taught with a reward function that did not just look at how much money was made, but heavily penalized the risk of extreme losses, known as tail risk. It also accounted for the cost of trading, ensuring that the system did not waste money on unnecessary changes to the portfolio.

The results of testing this framework against traditional methods and other advanced algorithms were striking. In the simulated environment, the new system achieved a higher total return than its competitors while suffering less severe losses when the market turned. Specifically, it showed a higher ratio of return to risk, meaning it generated more profit for every unit of danger taken. Perhaps most importantly, when the market crashed in 2022, the system recovered faster and lost less value than the other methods. The researchers found that the system's ability to learn the changing connections between stocks and its strict focus on avoiding catastrophic losses were the primary reasons for this success. When they removed these specific features from the model, its performance dropped significantly, confirming that these components were essential.

However, the researchers are careful to note the limits of their findings. The success was demonstrated in a controlled backtesting environment, a simulation where the rules of trading are idealized and do not include the messy realities of the real world, such as the delay in executing orders or the extra cost of moving large amounts of money. While the system proved it could adapt to a changing market in the simulation, the authors state that its performance in a live, real-time trading environment remains to be seen. They suggest that future work must test the system against actual market friction and across different types of assets to see if the advantages hold up outside the laboratory.

Ultimately, this work represents a step toward a more integrated approach to financial safety. By merging the prediction of risk with the decision of how to invest, and by treating the market as a living, changing network rather than a static set of numbers, the researchers have created a tool that is more responsive to the reality of modern finance. The study suggests that the future of risk management lies not in separating these tasks, but in weaving them together into a single, intelligent system that can watch, learn, and adjust as the world changes around it.

Technical Summary of CBGA-SAC: A CVaR-Constrained Deep Reinforcement Learning Framework for Portfolio Optimization

1. Problem Statement

The paper addresses the limitations of current financial risk management and portfolio optimization models in handling rapidly changing market regimes. Classical methods, such as Markowitz mean-variance optimization and parametric Value-at-Risk (VaR), often assume stationary, linear, or elliptically distributed returns, which fail under market stress. Furthermore, existing deep learning solutions typically decouple risk estimation from asset allocation. They often rely on static covariance matrices that do not capture dynamic, asset-specific correlations, and they frequently optimize for variance or return rather than explicitly targeting tail risk (extreme losses). The authors identify four specific gaps in the literature:

Decoupled Processes: Risk assessment and portfolio allocation are treated as separate stages rather than a unified pipeline.
Static Dependencies: Asset correlations are modeled using fixed historical estimates rather than learned, time-varying structures.
Implicit Tail Risk: Reinforcement learning (RL) rewards often lack explicit Conditional Value-at-Risk (CVaR) constraints, relying instead on implicit downside control.
Static Backtesting: Most systems are validated on single-pass historical data without testing adaptability to real-time regime shifts or streaming inference.

2. Methodology: The CBGA-SAC Framework

The authors propose CBGA-SAC, a unified hybrid architecture that integrates temporal feature extraction, dynamic graph-based correlation modeling, and CVaR-constrained reinforcement learning. The framework operates as an end-to-end pipeline with the following components:

Temporal Risk Feature Extraction (CNN-BiLSTM-Attention):
- Input: Multi-scale temporal data including log-returns, rolling volatility, RSI, MACD, and volume.
- Architecture: A 1D Convolutional Neural Network (CNN) captures local short-term patterns. These features are fed into a Bidirectional Long Short-Term Memory (BiLSTM) network to extract long-range temporal dependencies in both forward and backward directions.
- Attention Mechanism: A temporal attention layer weights the hidden states of the BiLSTM, highlighting time steps most relevant to the current risk situation, producing a context vector $c_t$ .
Dynamic Asset Correlation Modeling (Graph Attention Network - GAT):
- Graph Construction: Assets are treated as nodes in a fully connected graph.
- Mechanism: Instead of a static covariance matrix, a Graph Attention Network learns time-varying, data-driven dependency structures. It calculates attention coefficients ( $\alpha_{ij}$ ) between assets $i$ and $j$ based on their learned context vectors.
- Output: A graph embedding $g_i$ for each asset that reflects higher-order, dynamic inter-asset dependencies.
Fused Risk Representation:
- The temporal context vector ( $c_i$ ) and the graph embedding ( $g_i$ ) are concatenated and passed through a fully connected layer to generate a unified risk score. This fused representation is used simultaneously for risk forecasting (VaR/CVaR estimation) and as the state input for the RL agent.
Portfolio Optimization (CVaR-Constrained Soft Actor-Critic):
- Agent: A Soft Actor-Critic (SAC) agent is employed for continuous portfolio weight optimization.
- Action Space: The agent outputs a continuous vector of portfolio weights, projected onto the simplex via a softmax layer.
- Reward Function: The reward is explicitly designed to penalize tail risk and transaction costs:
  $R_t = R_{p,t} - \lambda_1 \cdot \text{CVaR}_\alpha(R_{p,t}) - \lambda_2 \cdot \sum |w_{i,t} - w_{i,t-1}|$
  Where $\lambda_1$ and $\lambda_2$ are tunable coefficients. The CVaR term is calculated using a normal approximation based on the predicted portfolio mean and standard deviation derived from the fused representation.
- Constraint: During training, policies violating a maximum tolerable CVaR threshold receive a "reward clip," effectively penalizing excessive tail risk.
Inference: The system operates on a rolling window, updating risk representations and rebalancing portfolios in real-time as new data arrives.

3. Experimental Setup

Dataset: A diversified universe of 30 US blue-chip stocks (subset of DJIA) with daily OHLCV data from January 2015 to December 2024.
Split: Training (2015–2020), Validation (2021), and Out-of-Sample Testing (2022–2024). The test period specifically includes the 2022 market drawdown and subsequent recovery to evaluate regime shift adaptability.
Baselines: The framework is compared against:
1. Markowitz Mean-Variance Optimization (MVO).
2. LSTM forecasting with MVO overlay.
3. Deep Deterministic Policy Gradient (DDPG).
4. Twin Delayed DDPG (TD3).
Metrics: Cumulative/Annualized Return, Sharpe Ratio, Sortino Ratio, Maximum Drawdown (MDD), 95% CVaR, and Risk Forecasting Error (RMSE).

4. Key Results

Performance: CBGA-SAC outperformed all baselines on the out-of-sample test period (2022–2024).
- Returns: Achieved the highest Cumulative Return (58.3%) and Annualized Return (16.2%).
- Risk-Adjusted Metrics: Attained the best Sharpe Ratio (0.87) and Sortino Ratio (1.19).
- Risk Control: Demonstrated the lowest Maximum Drawdown (-13.6%) and 95% CVaR (-1.68%) compared to baselines (e.g., MVO MDD was -24.1%).
Risk Forecasting: The proposed risk-forecasting head achieved the lowest RMSE (3.21) and highest $R^2$ (0.754) compared to LSTM-only and CNN-LSTM baselines.
Ablation Study:
- Removing the CVaR penalty from the reward function caused the most significant degradation in tail-risk metrics (CVaR worsened from -1.68% to -2.26%), confirming that explicit tail-risk awareness is the most critical design choice.
- Removing the Graph Attention Network (replacing with concatenation) reduced the Sharpe ratio from 0.87 to 0.74, highlighting the importance of dynamic correlation modeling.
- Removing Temporal Attention also led to performance degradation, though less severe than the CVaR removal.

5. Significance and Claims

The paper claims that CBGA-SAC provides a more powerful and partially interpretable framework for financial risk management by:

Unifying Risk and Allocation: It eliminates the decoupling of risk prediction and portfolio allocation, allowing the policy to react to predicted risk rather than lagged realized volatility.
Dynamic Correlation: It replaces static covariance matrices with a learned, time-varying graph structure that adapts to market regime shifts.
Explicit Tail Risk Control: By integrating CVaR directly into the RL reward function, the framework explicitly penalizes tail losses, leading to superior downside protection.
Regime Adaptability: The framework demonstrated faster recovery and shallower drawdowns during the simulated 2022 regime shift compared to traditional and other DRL baselines.

Limitations and Future Directions:
The authors explicitly state that the current validation was performed in a controlled backtesting environment with idealized order execution and fixed transaction costs. They acknowledge that live-deployment validation involving market impact, slippage, and latency constraints remains an open direction. Additionally, the current CVaR estimation relies on a normal approximation; future work suggests using non-parametric or extreme-value theory estimators for fat-tailed distributions. The paper concludes that while the framework addresses three of the four identified research gaps, the fourth (real-time live trading validation) is partially addressed but requires further empirical proof.