Explainable Regime Aware Investing

This paper presents an explainable, regime-aware portfolio construction framework using a strictly causal Wasserstein Hidden Markov Model that dynamically adapts regime complexity while preserving economic interpretability, achieving superior risk-adjusted returns and lower drawdowns compared to traditional benchmarks and nonparametric alternatives.

The Gist

Imagine you are the captain of a ship navigating a vast, unpredictable ocean. Your goal is to reach the destination (making money) as fast as possible without crashing into rocks (losing everything).

This paper presents a new, smarter way to steer that ship. It's called "Explainable Regime-Aware Investing."

Here is the breakdown of the problem, the solution, and why it works, using simple analogies.

1. The Problem: The "Whiplash" Effect

Most investment strategies try to predict the future based on the past.

• The Naive Approach (Equal Weight): This is like driving a car with cruise control set to 60 mph, regardless of whether you are on a highway, a dirt road, or approaching a cliff. You might get there, but you'll crash hard if the road changes.
• The "Smart" but Chaotic Approach (KNN): Imagine a navigator who looks at the last 100 days of weather and says, "Today looks like that day in 2018, so let's do exactly what we did then." The problem? If today looks slightly different, the navigator might suddenly decide it's actually like a day in 2005 and panic, ordering a 90-degree turn. This causes whiplash. In investing, this is called "high turnover"—buying and selling frantically because the model is too sensitive to tiny changes. It costs a fortune in fees and leaves you dizzy.

2. The Solution: The "Weather-Adaptive" Captain

The author, Amine Boukardagha, proposes a system that doesn't just look at the weather; it understands the seasons (Regimes).

• Regimes: Think of market conditions as seasons.
  • Summer (Bull Market): Stocks are hot, everything is growing.
  • Winter (Bear Market): Stocks are freezing, people are scared.
  • Stormy Spring: Volatile, unpredictable.
• The Goal: The ship should sail fast in Summer, slow down in Winter, and take shelter during Storms.

3. The Secret Sauce: The "Wasserstein HMM"

The paper introduces a specific tool called the Wasserstein Hidden Markov Model. Let's break down the fancy name:

• Hidden Markov Model (HMM): This is the engine that guesses which "season" we are in. It looks at the data (returns, volatility) and says, "I'm 80% sure we are in a 'Stormy' regime."
• The "Label Switching" Problem: Imagine you have a team of meteorologists. Every morning, they re-analyze the weather. The problem is, they might swap names. On Monday, "Stormy" was called "Regime A." On Tuesday, they decide "Stormy" is now "Regime B." If your ship's computer gets confused by the name change, it might panic and change course wildly.
• The Wasserstein Fix (The "GPS Anchor"): This is the paper's biggest innovation. Instead of just guessing the name of the season, the model uses Wasserstein Distance (a mathematical way to measure how different two shapes are).
  • The Analogy: Imagine you have a "Master Template" for a Storm. Every morning, the model looks at the new weather data and asks, "How close does this look to our Master Storm Template?"
  • Even if the storm changes slightly (a little more wind, a little less rain), the model recognizes it as the same storm because the shape is similar. It doesn't get confused by name changes. It keeps the identity of the "Storm" stable.

4. How It Works in Real Life

The system does three things every day:

1. Adapts Complexity: If the market is simple, it uses a simple model. If the market is chaotic, it adds more "seasons" to its map. It doesn't force a square peg into a round hole.
2. Tracks Identity: It uses the "Wasserstein Anchor" to ensure that when it says "We are in a Storm," it means the same storm it identified yesterday, not a totally different one.
3. Steers the Ship: Once it knows the season, it adjusts the portfolio.
   • In a "Growth" season: It loads up on stocks (SPX).
   • In a "Stress" season: It drops the stocks and moves to safe havens like Gold, Bonds, or the US Dollar.

5. The Results: The "Liberation Day" Test

The paper tested this on a simulated future scenario called "Liberation Day" (an early 2025 stock market crash).

• The Old Way (SPX Buy & Hold): The ship kept sailing full speed into the cliff. It lost 14.6% of its value.
• The Chaotic Way (KNN): The navigator kept changing its mind every hour, buying and selling frantically. It lost 12.5% and paid huge fees.
• The Wasserstein Way: The model sensed the "Storm" coming early. It smoothly shifted the ship toward defensive assets (Gold/Bonds) before the crash hit its peak.
  • Result: It only lost 5.4%.
  • Bonus: It didn't panic. The ship's course changed smoothly, not with jerky, expensive turns.

Summary

This paper argues that stability is more important than speed.

By using a mathematical "anchor" (Wasserstein distance) to keep the definition of market "seasons" consistent, the strategy avoids the panic-induced trading that destroys profits. It allows the portfolio to adapt to the market's mood without getting dizzy, resulting in higher returns and much less pain when the market crashes.

In one sentence: It's a smart navigation system that knows the difference between a sunny day and a storm, and it keeps its cool so it doesn't make a U-turn every time a cloud passes by.

Technical Summary

Here is a detailed technical summary of the paper "Explainable Regime-Aware Investing" by Amine Boukardagha.

1. Problem Statement

Daily portfolio construction faces a fundamental tension between rapid adaptation to evolving market structures and robustness against estimation noise and trading frictions.

• Instability: At high frequencies, conditional mean and covariance estimates are unstable. Small perturbations in inputs can cause large shifts in optimized weights.
• Regime Challenges: When allocation depends on latent market regimes, two specific operational issues arise in rolling (daily) environments:
  1. Model Order Instability: The appropriate number of regimes may change over time as market complexity shifts.
  2. Label Switching: Repeated re-estimation of Hidden Markov Models (HMMs) can permute latent state identities (labels) across dates. This destabilizes downstream optimization, degrades interpretability, and leads to excessive portfolio turnover.
• Limitations of Existing Methods:
  • Static/Rule-based: Early studies often used fixed regime counts or monotonic emergence rules (e.g., R2-RD), which lack flexibility.
  • Non-parametric (KNN): Local similarity methods (like K-Nearest Neighbors) suffer from discontinuous shifts in neighbor sets at daily frequencies, causing extreme turnover and fragile portfolio behavior.

2. Methodology: The Wasserstein HMM Framework

The paper proposes a Wasserstein Hidden Markov Model (Wasserstein HMM) designed for strictly causal, daily portfolio construction. The framework integrates three core components:

A. Strictly Causal Rolling Estimation

• Data: Daily log returns, 60-day rolling volatility, and 20-day rolling mean returns for a diversified cross-asset universe (Equities, Fixed Income, Commodities, FX).
• Protocol: All features are computed using data up to $t-1$ to ensure strict causality (no look-ahead bias).

B. Predictive Model-Order Selection

Instead of selecting the number of regimes ($K$) once via static criteria (like BIC) or enforcing monotonic growth, the model dynamically selects $K_t$ periodically (e.g., weekly).

• Mechanism: It uses a predictive scoring criterion based on one-step-ahead log-likelihood performance on a validation slice.
• Objective: Selects the $K$ that maximizes out-of-sample predictive value while penalizing complexity ($\lambda_K$). This allows the regime complexity to adapt dynamically to market conditions.

C. Template-Based Identity Tracking (The Core Innovation)

To solve the "label switching" problem without discrete combinatorial optimization, the model uses Wasserstein Template Tracking.

• Persistent Templates: A fixed set of regime templates ($\Theta_g = \{\mu_g, \Sigma_g\}$) represents stable economic identities.
• Wasserstein Mapping: At each re-estimation date, the newly estimated Gaussian HMM components are mapped to the nearest persistent template using the closed-form 2-Wasserstein distance between multivariate normal distributions:
  $$W_2^2(\mathcal{N}_1, \mathcal{N}_2) = \|\mu_1 - \mu_2\|^2 + \text{Tr}\left(\Sigma_1 + \Sigma_2 - 2(\Sigma_2^{1/2}\Sigma_1\Sigma_2^{1/2})^{1/2}\right)$$
• Smoothing: Templates are updated via exponential smoothing. This ensures regime identities evolve continuously and remain stable over time, preventing the "jumping" of labels that plagues standard rolling HMMs.

D. Portfolio Optimization

• Inputs: Conditional moments are aggregated as template-probability-weighted averages.
• Optimization: A Transaction-Cost-Aware Mean-Variance Optimization (MVO) is solved daily.
  • Objective: Maximize $\mu^\top w - \gamma w^\top \Sigma w - \tau \|w - w_{t-1}\|_1$.
  • Constraints: Full investment, long-only, and max-weight limits.
  • Covariance Estimation: Ledoit-Wolf shrinkage is used to stabilize high-dimensional risk estimates.

3. Key Contributions

1. Geometrically Grounded Regime Inference: Introduces a framework using optimal transport geometry (Wasserstein distance) to anchor regime identities, eliminating combinatorial label switching.
2. Adaptive Complexity Control: Replaces static or rule-based regime counting with predictive model-order selection, allowing the system to adapt to changing market complexity.
3. Empirical Validation of Stability: Demonstrates that regime inference stability (specifically identity preservation) is a first-order determinant of portfolio performance, directly reducing turnover and drawdowns compared to non-parametric alternatives.

4. Empirical Results

The strategy was evaluated on a daily cross-asset universe (2005–2026) against benchmarks and a non-parametric KNN baseline.

Performance Metrics (Out-of-Sample)

Metric	Parametric (Wasserstein HMM)	Non-Parametric (KNN)	Equal-Weight	SPX Buy & Hold
Sharpe Ratio	2.18	1.81	1.59	1.18
Max Drawdown	-5.43%	-12.52%	-9.87%	-14.62%
Avg Daily Turnover	0.0079 (0.8%)	0.5665 (56.6%)	N/A	N/A

Key Findings

• Turnover Reduction: The Wasserstein HMM achieved dramatically lower turnover (0.8% daily avg) compared to the KNN approach (56.6% daily avg). The KNN strategy suffered from "discontinuous neighbor sets," causing the optimizer to react aggressively to noise.
• Drawdown Control: The parametric model significantly reduced maximum drawdown (-5.43%) compared to SPX Buy & Hold (-14.62%).
• The "Liberation Day" Event (Early 2025): During a sharp equity selloff in early 2025, the Wasserstein HMM detected a transition to a "stress regime" (Regime E). It systematically reduced equity exposure and shifted to defensive assets (USD, Bonds, Gold), mitigating peak-to-trough losses. In contrast, passive benchmarks suffered full exposure.
• Regime Interpretability: The model identified 5 distinct, stable regimes (e.g., "Balanced Risk-On," "Inflationary/Dollar Strength," "Equity Stress"). The strategy successfully rotated assets based on the conditional Sharpe ratios of these regimes.

5. Significance and Conclusion

The paper establishes that in daily cross-asset allocation, how regimes are inferred is more critical than whether they exist.

• Stability over Complexity: The primary driver of the strategy's success was not just the detection of regimes, but the stabilization of regime identity via Wasserstein template tracking. This prevented the "churn" associated with label switching.
• Implementation Robustness: By smoothing the transition between states and controlling model complexity, the framework produces implementable portfolios with lower transaction costs and smoother capital growth.
• Economic Value: The results prove that structured latent-state models with predictive complexity control and geometric identity anchoring provide materially superior risk-adjusted returns and downside protection compared to both passive strategies and non-parametric local similarity methods.

In summary, the Wasserstein HMM offers a principled, explainable, and robust solution for high-frequency regime-aware investing, transforming latent state inference from a source of instability into a driver of portfolio resilience.