Onflow: a model free, online portfolio allocation algorithm robust to transaction fees

This paper introduces Onflow, a model-free, online reinforcement learning algorithm that optimizes portfolio allocation via gradient flows to maximize expected log returns, demonstrating robustness to high transaction costs and superior performance compared to existing dynamic strategies without requiring assumptions about asset return distributions.

The Gist

Imagine you are the captain of a ship navigating a vast, unpredictable ocean. Your goal is to reach the treasure island (maximum wealth) as fast as possible. However, the ocean has two tricky rules:

1. The currents change constantly: You don't know if the wind will blow from the north or south tomorrow.
2. There is a toll for turning: Every time you adjust your sails to face a new wind, you pay a fee. If you turn too often, the fees eat up all your treasure.

For decades, sailors (financial experts) have tried to build maps to predict the currents. Some maps assume the wind blows in a perfect, predictable pattern (like the famous Markowitz model). Others try to learn from the past without assuming a specific pattern (like Cover's "Universal Portfolio").

Enter "Onflow," a new navigation system.

The Core Idea: Flowing Like Water, Not Jumping Like a Frog

Most old navigation systems work like a frog jumping from one lily pad to another. They look at the data, make a big decision, and jump to a new strategy. If the wind changes slightly, the frog jumps again, paying the toll every time.

Onflow is different. It treats your investment strategy like water flowing down a hill.

• Instead of making sudden, jerky jumps, it gently steers your portfolio in the direction that looks most promising.
• It uses a mathematical concept called a "gradient flow." Imagine a ball rolling down a hill to find the lowest point. Onflow is like a ball that rolls smoothly toward the highest peak of profit, but it's heavy enough that it doesn't bounce around wildly. It glides.

The "Soft" Steering Wheel (The Softmax)

How does Onflow decide which way to turn? It uses a "Softmax" function. Think of this as a smart, flexible steering wheel.

• If the wind looks good for Asset A, the wheel turns slightly toward A.
• If Asset B suddenly looks amazing, the wheel turns more toward B.
• Crucially, it never locks the wheel in one spot. It keeps the ship moving, but it does so with such smoothness that it avoids the "toll fees" caused by jerky, frequent adjustments.

The Secret Weapon: Ignoring the "Toll" Until Necessary

The paper highlights a major problem: Transaction Fees.

• Scenario A (No Fees): If turning the wheel is free, Onflow is just as good as the best sailors. It finds the perfect balance of assets, matching the theoretical "best possible" portfolio.
• Scenario B (High Fees): This is where Onflow shines. Imagine a toll booth that charges you 2% of your ship's value every time you turn the wheel.
  • Old Algorithms (The Frogs): They keep jumping back and forth, trying to catch every tiny wind shift. They pay so many tolls that they arrive at the island with almost no treasure left.
  • Onflow (The Water): It realizes, "Hey, this wind shift is small. If I turn, I'll pay a huge fee. I'll just glide a little bit and wait." It becomes patient. It only turns the wheel when the wind shift is so strong that the profit outweighs the toll.

Real-World Tests: The Old NYSE and Crypto

The authors tested this on two very different oceans:

1. The "Old NYSE" (1965–1987): They used data from 36 old stocks. Even when they simulated a massive 2% fee (which usually breaks other strategies), Onflow kept the ship moving forward while others crashed or barely broke even.
2. The "Crypto & Tech" Ocean (2020–2025): They tested it on wild, volatile assets like Tesla, Nvidia, Bitcoin, and Ethereum. These are like storms where the wind changes every second. Even here, Onflow managed to balance the ship, avoiding the fees that would have sunk a more aggressive strategy.

Why This Matters

The beauty of Onflow is that it is model-free.

• Old way: "I assume the wind follows a bell curve, so I will sail this way." (If the wind breaks the bell curve, you crash).
• Onflow way: "I don't care what the wind usually does. I just look at where the water is flowing right now and I glide toward it."

The Bottom Line

Onflow is a smart, patient investor. It doesn't try to outsmart the market with complex predictions. Instead, it uses a smooth, continuous motion to adjust its portfolio. It knows when to be aggressive and, more importantly, when to sit still to avoid paying the "turning fees."

In a world where trading fees can kill your profits, Onflow is the strategy that teaches you how to sail smoothly without burning your fuel.

Technical Summary

Here is a detailed technical summary of the paper "Onflow: a model-free, online portfolio allocation algorithm robust to transaction fees" by Gabriel Turinici and Pierre Brugiere.

1. Problem Statement

The paper addresses the challenge of online portfolio allocation in financial markets. The core problem is to dynamically adjust portfolio weights to maximize expected log-returns while accounting for transaction costs (fees).

• Model Risk: Traditional approaches (e.g., Markowitz) rely on estimating statistical moments (mean, variance) of asset returns, which are often uncertain and lead to suboptimal performance.
• Transaction Costs: High transaction fees (e.g., 2%) often cause standard dynamic rebalancing strategies (like Cover's Universal Portfolio or Helmbold's EG algorithm) to collapse, as the cost of trading outweighs the gains from rebalancing.
• Goal: Develop a model-free, reinforcement learning-based algorithm that adapts to streaming data without assuming a specific distribution for asset returns and remains robust even under high transaction costs.

2. Methodology: The Onflow Algorithm

The authors propose Onflow, a continuous-time gradient flow approach to portfolio optimization.

A. Mathematical Framework

• Market Model: A market with $K$ assets and time steps $t$. The portfolio is defined by weights $\pi$ on the unit simplex ($\sum \pi_k = 1, \pi_k \geq 0$).
• Objective: Maximize the cumulative log-return: $\sum \ln(\langle \pi_t, f_t \rangle)$, where $f_t$ are price relatives.
• Softmax Parameterization: Instead of optimizing weights $\pi$ directly, the algorithm optimizes a vector $H \in \mathbb{R}^K$ and maps it to the simplex via the softmax function:
  $$\pi(k) = S(H)_k = \frac{e^{H_k}}{\sum_\ell e^{H_\ell}}$$
  This ensures weights remain positive and sum to one.

B. Gradient Flow Dynamics

Unlike discrete update rules (e.g., Multiplicative Weights/EG), Onflow treats the optimization as a continuous Ordinary Differential Equation (ODE).

• Reward Function: The objective function combines log-return and transaction costs:
  $$\mathcal{F}_t(H) = \ln(\langle S(H), f_t \rangle) - \text{TransactionCost}(S(H), \pi_{t+})$$
• Transaction Cost Modeling: The cost is modeled as proportional to the turnover. To ensure differentiability (crucial for gradient flows), the authors replace the non-smooth absolute value $|x|$ with a pseudo-Huber loss: $\sqrt{x^2 + a^2} - a$.
• The ODE: The algorithm evolves the parameter $H$ over a time interval $u \in [0, \tau]$ according to:
  $$\frac{d}{du}H(u) = -\nabla_H \mathcal{F}_t(H(u))$$
  The new allocation is obtained by solving this ODE for a duration $\tau$ and setting $\pi_{t+1} = S(H(\tau))$.

C. Reinforcement Learning Context

Onflow is framed as a Policy Gradient method in a model-free Reinforcement Learning setting. It does not require a transition model of the market; it updates the policy based solely on observed price relatives and the immediate reward (log-return minus fees).

3. Key Contributions

1. Continuous-Time Gradient Flow: The paper introduces a novel application of gradient flows (ODEs) to portfolio allocation, contrasting with the discrete updates common in literature (e.g., EG algorithm). This provides smoother, more controllable updates.
2. Robustness to Transaction Fees: By integrating a smooth transaction cost term directly into the gradient flow, Onflow naturally reduces turnover when fees are high, avoiding the "collapse" seen in other algorithms.
3. Theoretical Convergence (Log-Normal Case): The authors prove that in a continuous-time log-normal market with zero transaction costs, the Onflow algorithm converges exponentially to the Markowitz optimal portfolio (the log-optimal portfolio).
4. Model-Free Operation: The algorithm requires no assumptions about the distribution of asset returns (e.g., it works even with heavy-tailed distributions).

4. Numerical Results

The algorithm was tested on the "Old NYSE" dataset (36 stocks, 1965–1987) and modern high-volatility assets (Tech stocks and Crypto, 2020–2025).

A. Performance vs. Benchmarks

• Zero Fees ($\xi=0$): Onflow performs comparably to or slightly better than Cover's Universal Portfolio (UP) and Helmbold's EG algorithm.
• High Fees ($\xi=2\%$): This is the critical differentiator.
  • UP and EG: Performance degrades significantly, often failing to outperform simple "buy-and-hold" strategies due to excessive turnover.
  • Onflow: Maintains superior performance. In the "Iroquois–Kin Ark" pair, Onflow achieved ~50x initial wealth, while EG and UP dropped to ~15x.
  • Mechanism: Onflow drastically reduces turnover (e.g., from ~2% daily to 0.5% daily under high fees) by staying closer to the current allocation unless the gradient strongly dictates a change.

B. High Volatility & Multi-Asset Tests

• Tech/Crypto: Tested on NVDA, TSLA, BTC, and ETH. Onflow successfully tracked the best-performing assets (e.g., switching from TSLA to NVDA) while managing fees.
• Sharpe Ratio: In a 10-asset random sampling test, Onflow achieved a statistically significant higher Sharpe ratio (median 0.237) compared to EG (0.221), Uniform CRP (0.218), and "Split and Forget" (0.209).
• Scalability: Computational analysis shows the ODE solver time stabilizes for portfolios with more than 10 assets, indicating good scalability.

5. Significance and Conclusion

• Practicality: Onflow offers a practical solution for real-world trading where transaction costs are a major friction. It bridges the gap between theoretical optimality and practical feasibility.
• Robustness: The algorithm is robust against model risk (no distributional assumptions) and heavy-tailed return distributions.
• Future Work: The authors suggest extending the method to allow short selling, comparing it with deep learning approaches (LSTMs), and testing on different market regimes.

In summary, Onflow represents a significant advancement in online portfolio optimization by leveraging continuous gradient flows to inherently balance the trade-off between maximizing returns and minimizing transaction costs, outperforming established benchmarks particularly in high-fee environments.