🤖 machine learning

Conformal bandits: bringing statistical validity and reward efficiency under weak arm separability

This paper introduces Conformal Bandits, a novel framework that integrates Conformal Prediction into sequential decision-making to provide finite-sample statistical coverage guarantees while maintaining regret efficiency, particularly excelling in scenarios with weak arm separability such as portfolio allocation where classical policies often fail.

Original authors: Simone Cuonzo, Nina Deliu

Published 2026-08-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Simone Cuonzo, Nina Deliu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but instead of finding a criminal, you are trying to find the best option among a bunch of choices. This is the world of "Multi-Armed Bandits," a famous puzzle in computer science and statistics. Picture a row of slot machines in a casino, each with a different lever (or "arm"). You don't know which machine pays out the most money; you only know that some might be rigged to give you nothing, while others might be jackpot winners. Your job is to figure out which machine is the best by pulling levers one by one. The tricky part is the "dilemma": do you keep pulling the lever that has given you a few coins so far (exploitation), or do you try a new, unknown lever that might be even better (exploration)? If you guess wrong, you lose money, which statisticians call "regret."

Usually, these machines have obvious differences: one is a dud, and one is a goldmine. But in the real world, things are rarely that clear. Sometimes, the difference between the best machine and the second-best is so tiny that it's almost impossible to tell them apart, especially if the machines are also "noisy" (meaning they sometimes give a coin when they shouldn't, or take one when they should give). This is called "weak arm separability." It's like trying to hear a whisper in a hurricane. Traditional methods for solving this puzzle often rely on strict rules about how the noise behaves, which can fail when the real world gets messy. This paper steps into that messy, whispering hurricane to see if a new kind of detective work can find the best machine without getting lost.

The authors, Simone Cuonzo and Nina Deliu, introduce a clever new framework called Conformal Bandits. Think of their approach as giving the detective a super-accurate, flexible "uncertainty shield." Instead of guessing based on rigid mathematical formulas that assume the noise is perfectly predictable, they use a technique called Conformal Prediction. Imagine you are trying to guess the temperature tomorrow. A traditional method might say, "It will be between 60 and 80 degrees," based on a strict formula. Conformal Prediction, however, looks at the actual history of the weather and says, "Based on how the weather actually behaved in the past few days, I can guarantee with 95% certainty that the temperature will fall within this specific range." It doesn't care if the weather is weird or unpredictable; it just guarantees that its prediction box is big enough to catch the truth most of the time.

In this paper, the authors replace the old, rigid "confidence intervals" used in standard bandit strategies with these flexible, data-driven prediction boxes. They call their new strategy Conformal UCB (Upper Confidence Bound). In their simulations, they tested this new method against the classic "UCB1" strategy in scenarios where the difference between the best and second-best options was tiny (like a gap of 0.01 in reward) and the noise was high. The results showed that the old UCB1 strategy struggled, often getting stuck in a loop of confusion and accumulating a lot of "regret" (lost money). In contrast, the Conformal Bandits were much better at distinguishing the tiny differences, learning faster, and making fewer mistakes. They also showed that these new methods could guarantee that their prediction boxes were actually correct (a "statistical guarantee") even when the data was messy, heavy-tailed, or skewed—conditions where the old methods often failed or became overly conservative.

The paper then takes this idea to a real-world playground: portfolio allocation, or how investors decide where to put their money. Here, the "arms" are different investment strategies (like holding only cash, splitting money equally, or using a complex formula to balance risk and reward). The authors found that in the financial world, the differences between these strategies are often incredibly small and hard to spot, just like the tiny gaps in their simulations. They showed that their Conformal Bandit approach could navigate these murky waters better than traditional methods, leading to higher returns and less risk of huge losses.

To make things even smarter, the authors added a layer of "regime awareness." They realized that financial markets change their personality: sometimes they are calm and sunny (Bull markets), sometimes they are stormy and scary (Bear markets). They used a tool called a Hidden Markov Model (think of it as a weather forecast for the market's mood) to detect these shifts. When the market was calm, their algorithm was optimistic and looked for the highest potential gains. When the market turned stormy, the algorithm instantly switched to a defensive mode, focusing on protecting against losses. This "Regime-Aware" version of their strategy outperformed everything else, including the standard methods and even the non-regime-aware version of their own new tool. It proved that by combining the flexibility of Conformal Prediction with an awareness of the market's changing mood, you can make much smarter decisions, even when the differences between your choices are barely visible.

In short, this paper suggests that by swapping out old, rigid rules for flexible, data-driven "uncertainty shields," we can make better decisions in uncertain worlds where the differences between options are tiny and the noise is loud. It doesn't claim to have solved every problem in finance or machine learning, but in their tests and simulations, it showed a clear path toward more reliable and efficient decision-making, especially when the stakes are high and the clues are faint.

Technical Summary: Conformal Bandits

Problem Statement
The paper addresses the Multi-Armed Bandit (MAB) problem, a paradigm for sequential decision-making under uncertainty, specifically focusing on the weak arm separability regime. In many real-world applications, such as medical treatments, digital health interventions, and financial portfolio allocation, the difference in expected rewards ( $\Delta = \mu^* - \mu$ ) between the optimal arm and suboptimal arms is extremely small. This "small-gap" or low signal-to-noise ratio (SNR) regime poses significant challenges for classical regret-minimizing strategies like Upper Confidence Bound (UCB).

Traditional UCB policies rely on distributional assumptions (e.g., sub-Gaussian rewards) and asymptotic guarantees derived from inequalities like Hoeffding's. In small-gap settings, these classical confidence bounds often become unstable or excessively conservative, leading to linear cumulative regret over finite horizons and a failure to distinguish between arms efficiently. Furthermore, standard bandit literature largely neglects finite-sample statistical validity (i.e., guaranteed coverage of future rewards), focusing almost exclusively on regret minimization.

Methodology
The authors propose Conformal Bandits, a framework that integrates Conformal Prediction (CP) into the bandit decision-making process. This approach replaces the classical, distribution-dependent confidence bounds with statistically valid, finite-sample prediction intervals.

Conformal Quantile Regression (CQR): To handle heteroscedasticity and asymmetry in reward distributions (common in finance), the framework utilizes CQR. This involves:
- Fitting conditional quantile models (lower and upper) on a training set.
- Computing conformity scores (residuals) on a calibration set.
- Constructing prediction intervals $C_{k,t+1}^{1-\alpha} = [L_{k,t+1}, U_{k,t+1}]$ that guarantee marginal coverage $P(Y_{k,t+1} \in C_{k,t+1}^{1-\alpha}) \geq 1-\alpha$ under the sole assumption of exchangeability.
- Employing Adaptive Conformal Inference (ACI) to adjust the miscoverage level $\alpha$ dynamically, addressing the violation of exchangeability caused by the sequential nature of bandit arm selection.
Conformal UCB (CP-UCB): The simplest policy in the framework selects the arm with the highest conformal upper bound ( $U_{k,t+1}$ ), preserving the "optimism in the face of uncertainty" principle but using data-driven, distribution-free intervals.
General CP-Bandit: The authors introduce a flexible family of policies parametrized by a mixing coefficient $\lambda \in [0, 1]$ . The selection score is a convex combination of the conformal upper and lower bounds:
$\text{Score}_k = (1-\lambda)U_{k,t+1} + \lambda L_{k,t+1}$
- $\lambda = 0$ : Fully optimistic (equivalent to CP-UCB).
- $\lambda = 1$ : Fully conservative (maximin rule, focusing on the lower bound).
- $0 < \lambda < 1$ : Risk-aware policies balancing upside potential and downside protection. This formulation satisfies shift-equivariance and dominance-consistency.
Regime-Aware Extension: For the portfolio allocation application, the framework is extended to handle non-stationarity. A Hidden Markov Model (HMM) is used to infer market regimes (Bull, Neutral, Bear). The policy adapts dynamically:
- In Bull/Neutral regimes: It acts optimistically (selecting based on upper bounds).
- In Bear regimes: It acts defensively (selecting based on lower bounds to minimize downside risk).

Key Contributions

Framework Integration: The introduction of Conformal Bandits, which bridges the gap between regret-minimizing bandit policies and finite-sample statistical validity.
Risk-Aware Policy Family: The development of the CP-Bandit family, which allows for a principled interpolation between optimistic and conservative strategies via the Hurwicz criterion, offering a distribution-free analogue to Value-at-Risk (VaR) in bandit selection.
Small-Gap Performance: Demonstration that Conformal Bandits achieve superior cumulative regret and best-arm selection rates in small-gap and low-SNR regimes where classical UCB1 fails (exhibiting linear regret).
Financial Application: A novel application to portfolio allocation that addresses the specific challenges of financial markets: small reward gaps, non-stationarity, and the need for risk management. The integration of HMMs allows the bandit to adapt to regime-switching dynamics.

Results

Simulation Studies: In Monte Carlo simulations with Gaussian, Student-t (heavy-tailed), and Skewed Student-t rewards, Conformal Bandit variants (particularly with $\lambda=0.5$ ) consistently outperformed classical UCB1. They achieved lower cumulative regret and faster convergence to the optimal arm. Crucially, while UCB1 produced overly conservative intervals with poor coverage in heavy-tailed scenarios (e.g., achieving 71.42% coverage vs. the 80% nominal target in the Student-t setting specifically at the larger gap $\Delta=0.05$ ), Conformal Bandits maintained nominal coverage guarantees while providing tighter, more informative intervals.
Portfolio Allocation Case Study: Applied to daily ETF data (2018–2025), the Regime-Aware CP policy significantly outperformed stationary bandit strategies and traditional portfolio benchmarks (Equal Weight, Mean-Variance).
- It achieved the highest Total Return (1.54 vs. 0.62 for CP-UCB) and Sharpe Ratio (1.80 vs. 0.60).
- It demonstrated superior risk management, with the lowest Maximum Drawdown (0.086 vs. 0.259) and highest Calmar Ratio (1.66 vs. 0.27).
- The regime-aware mechanism successfully switched to a "Sell-All" (cash) strategy during Bear market episodes, mitigating losses that stationary policies failed to avoid.

Significance and Claims
The paper claims that Conformal Bandits offer a robust solution for sequential decision-making in environments characterized by weak arm separability and complex, non-standard reward distributions. By replacing model-dependent confidence bounds with distribution-free prediction intervals, the framework provides finite-sample statistical guarantees (coverage) without sacrificing regret efficiency.

The authors emphasize that this approach is particularly valuable in domains like finance and healthcare, where small differences in expected outcomes are critical, and where classical assumptions (normality, stationarity) often fail. The integration of regime-switching models further demonstrates the framework's ability to adapt to structural breaks, enhancing risk-adjusted returns while maintaining rigorous uncertainty quantification. The work suggests a path toward unifying statistical validity with decision-theoretic efficiency in sequential learning.

Technical Summary: Conformal Bandits

More like this