💰 quantitative finance

Machine Learning Enhanced Multi-Factor Quantitative Trading: A Cross-Sectional Portfolio Optimization Approach with Bias Correction

This paper addresses the critical "upstream contamination" flaw in Chinese A-share quantitative trading caused by non-executable price limits by introducing a mask-first design that prevents non-tradable data from entering factor calculations, which, when combined with GPU-accelerated processing and specialized loss functions, significantly improves realized Sharpe ratios and corrects inflated information coefficients.

Original authors: Yimin Du

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yimin Du

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Fake Price" Trap

Imagine you are a chef trying to create a perfect recipe (a trading strategy) based on the prices of ingredients (stocks) in a giant supermarket.

In the Chinese stock market (A-shares), there is a strict rule: if a stock price goes up or down too fast (about 10% or 20% in a day), the market hits a "speed bump." The price stops moving, and nobody can actually buy or sell that stock anymore until the next day.

The paper identifies a sneaky flaw in how most computer programs handle this.

The Mistake: Most programs see the "speed bump" price, write it down, and use it to calculate averages and trends before realizing, "Oh wait, I can't actually buy this!"
The Result: The computer thinks it found a super-predictable pattern because it's looking at prices that are frozen in place. It's like a chef tasting a frozen ice cube and thinking, "This is the perfect temperature for soup!" The computer learns to predict returns it can never actually trade.

The authors call this "Upstream Contamination." It's like pouring dirty water into a clean bucket; even if you try to filter the water later, the bucket is already tainted.

The Solution: The "Do Not Touch" Mask

To fix this, the authors built a system with a "Mask-First" design.

Think of this mask as a set of red "Do Not Touch" stickers that are placed on the price tags the moment the data is loaded, before any math is done.

If a stock hits a speed bump, the sticker goes on immediately.
Every single calculation the computer does (averages, rankings, correlations) checks for the sticker first.
If the sticker is there, the computer ignores that price completely. It doesn't even look at it.

This ensures the computer only learns from prices that were actually tradeable in the real world.

The Toolkit: How They Built It

The authors didn't just fix the mask; they built a high-speed factory to process this data. Here are the key tools they used, explained simply:

The Super-Fast Engine (GPU Vectorization):
- Analogy: Imagine calculating the average speed of 3,000 cars. Doing it one by one with a calculator (like standard software) takes forever. The authors built a system that uses a massive super-computer (GPU) to calculate all 3,000 cars at the exact same time.
- Result: It is 51 times faster than the old way.
The "Wrong Direction" Penalty (Adjusted-MSE Loss):
- Analogy: Imagine you are betting on a horse race.
  - Mistake A: You bet on a horse that wins, but you only bet $1 when you should have bet $100. (You were right about the winner, just wrong about the size).
  - Mistake B: You bet on a horse that loses, when you should have bet on the winner. (You were wrong about the direction).
- The authors realized that Mistake B is much more expensive. So, they programmed the computer to be 11 times more angry when it gets the direction wrong than when it just gets the size wrong. This forces the model to focus on getting the "Up" or "Down" right.
The "Practice Dummy" Generator (GBM Augmentation):
- Analogy: Financial data is rare; you only have a few years of history. It's like a pilot trying to learn to fly with only 10 hours of flight time.
- The authors created a "flight simulator" that generates realistic fake stock data based on real patterns. They let the computer practice on this fake data to smooth out its learning, making it less likely to panic when real market conditions change.
The Smart Portfolio Manager (Markowitz-Ledoit-Wolf):
- Analogy: Once the computer picks the best stocks, it has to decide how much money to put in each. If it puts too much in one, a small problem ruins everything.
- They used a math technique that acts like a "safety net," smoothing out the risks so the portfolio doesn't crash if one stock acts up. They also added a "warm start" feature, which is like remembering your last seat position so you don't have to reset the chair every time you sit down. This makes the daily calculations much faster.

The Results: Did It Work?

The authors tested their system in two ways:

On Fake Data: They created a perfect, controlled simulation of 3,000 stocks over 14 years.
- Result: The system achieved a "Sharpe Ratio" of 2.05. (In finance, a score above 2.0 is considered excellent).
On Real Data: They tested it on actual Chinese stocks from 2022 to 2024.
- Result: It achieved a Sharpe Ratio of 1.63. This is still very strong for a real-world strategy.

The Most Important Discovery:
The authors ran a "what-if" test. They turned off the "Mask" and ran the system the old, contaminated way.

The Illusion: The "fake" system looked better on paper (it had a higher "Information Coefficient"). It seemed to predict the future perfectly.
The Reality: When they tried to trade with it, it lost money. The Sharpe ratio dropped by 0.44 points.

The Lesson: The single biggest improvement didn't come from a fancy new AI model or a complex math formula. It came from simply stopping the computer from looking at prices it couldn't trade.

Summary

This paper is an engineering story about cleaning up the data before feeding it to a smart computer. By putting a "Do Not Touch" mask on untradeable prices, they stopped the computer from learning fake patterns. This simple fix, combined with fast computing and smart penalties for wrong guesses, created a trading system that is both profitable and realistic.

The authors released all their code as open-source, inviting others to see exactly how they built it and to reproduce the results.

Technical Summary: Machine Learning Enhanced Multi-Factor Quantitative Trading

1. Problem Definition: Upstream Contamination

The paper identifies a critical, often overlooked flaw in quantitative trading pipelines for Chinese A-share markets: upstream contamination.

In A-share markets, daily price limits (±10% for main-board, ±20% for STAR/ChiNext) mean that closing prices for stocks hitting these limits are non-executable (unmatched buy orders remain unfilled). Standard industry practice involves computing rolling-window factors (e.g., moving averages, correlations, ranks) on the full dataset and filtering out non-tradable stocks after aggregation.

The authors demonstrate that this "post-hoc" filtering fails because the rolling-window primitives have already ingested the non-executable prices. These contaminated values propagate through the entire factor chain, creating a "phantom alpha."

The Consequence: The model learns to predict returns that are mechanically high (due to limit-up stocks) but impossible to trade.
The Impact: On real data, ignoring this bias inflates the apparent Information Coefficient (IC) by 18% while reducing the realized Sharpe ratio by 0.44 points. This is a form of look-ahead bias where the model predicts untradeable events.

2. Methodology: The Mask-First Architecture

The core contribution is a "mask-first" design pattern where a Boolean tradability mask ( $M \in \{0, 1\}^{T \times N}$ ) is constructed at data load time and threaded through every computational operator. No window ever reads a non-tradable price.

A. System Components

Mask-Aware Primitives (GPU-Vectorized):
- Implemented 18 custom PyTorch primitives using torch.unfold to handle rolling windows on the GPU.
- Speed: Achieves a 51× speedup over standard pandas implementations (10.6s vs. 542s for a 3,000-stock, 213-factor pipeline on an A100).
- Logic:
  - Ranking: Masked cells are filled with $+\infty$ before sorting to rank last, then zeroed. Denominators use the count of valid cells, not total stocks.
  - Correlation: Computes masked Pearson correlation; if any cell in the window is masked, the output is marked unreliable.
  - EWMA: Maintains the previous accumulator value when a masked cell is encountered rather than resetting, preventing drift.
- Precision: Uses float64 for recurrence-based primitives to avoid accumulation errors (observed 0.3% relative error in float32 after 2,500 steps).
Factor Engine:
- 213 Factors: Composed of 9 curated Alpha101 formulas and 204 legacy factors specific to A-share microstructure (e.g., VWAP deviations, turnover dynamics, limit-move liquidity crunches).
- Neutralization: All factors are residualized against industry dummies (29 CSI categories) and log market cap via OLS before cross-sectional z-scoring.
Machine Learning Training:
- Dataset: Bidirectional filtering ensures both the feature date ( $t$ ) and label date ( $t+1$ ) are tradable.
- Adjusted-MSE Loss: A custom loss function that penalizes wrong-sign predictions 11× more heavily than magnitude errors ( $\gamma=0.1$ $γ = 0.1$ ).
  - Rationale: Portfolio managers suffer more from directional errors (buying a stock that falls) than magnitude errors (under-sizing a winning trade).
- Architectures: Tested a small MLP (44K params) and a Factor-axis Transformer (220K params) where factors are tokenized as sequence positions.
- Data Augmentation: Uses Block-Bootstrap GBM to generate synthetic return panels. This preserves short-range serial correlation and smooths the loss landscape, particularly helping the Transformer avoid catastrophic overfitting.
Portfolio Construction:
- Optimization: Mean-variance optimization (Markowitz) with Ledoit-Wolf shrinkage for covariance estimation.
- Constraints: Long-only, max position 3%, risk aversion $\alpha=10$ .
- Efficiency: Uses cvxpy with warm-start caching, reducing solve time from ~1.2s to ~0.2s per day by avoiding repeated problem canonicalization.

3. Key Results

The system was evaluated on a synthetic panel (3,000 stocks, 14 years) and proprietary real A-share data (2022–2024).

Metric	Synthetic Panel (Full System)	Real A-Share (2022–2024)
Annualized Sharpe	2.05	1.63
Max Drawdown	12.0%	11.4%
Information Coefficient	0.049	0.049

Ablation Studies

The paper isolates the contribution of each component, revealing that data engineering outweighs model complexity:

Mask Contract: The single largest contributor (+0.44 Sharpe). Removing it causes a massive drop in realized performance despite higher apparent IC.
Adjusted-MSE Loss: Adds +0.27 Sharpe over standard MSE.
GBM Augmentation: Adds +0.19 Sharpe.
Ledoit-Wolf Shrinkage: Adds +0.18 Sharpe.
Transformer Architecture: Adds +0.12 Sharpe over MLP.

Deflated Sharpe Ratio (DSR): After adjusting for multiple testing (50+ configurations tried), the DSR for the real-data Sharpe of 1.63 is 0.978, indicating a 97.8% probability that the true Sharpe exceeds zero.

4. Significance and Claims

The paper frames its contribution not as a novel theoretical breakthrough, but as a systematic engineering study addressing a specific failure mode in financial ML.

Engineering over Theory: The authors argue that "deceptively small implementation choices" (like handling NaNs or limit-up prices) swing reported Sharpe ratios by full points. They claim that in markets with fill-gap microstructure (like A-shares), data quality engineering dominates model architecture as a source of alpha.
The "Mask-First" Pattern: The paper posits that bias elimination is a first-order performance driver, not a hygiene step. The mask-first approach is presented as a generalizable pattern for any market with non-executable prices (circuit breakers, suspensions, T+1 constraints).
Transparency: The authors explicitly release the full implementation (213-factor engine, training loop, optimizer, backtest, and synthetic data generator) under the MIT license to allow reproducibility without proprietary data.
Honesty in Performance: The system correctly avoids limit-up stocks during euphoric rallies (e.g., September 2024), resulting in underperformance relative to the market index during those specific periods. The authors frame this not as a failure, but as the "honest cost" of a design that refuses to predict untradeable returns.

5. Limitations Acknowledged

Long-Only Constraint: Due to A-share short-selling restrictions, the system cannot capture long-short alpha.
Transaction Costs: Uses a linear cost model (5–8 bps), which is optimistic for large portfolios with convex market impact.
Data Window: The real-data test period (2022–2024) covers only one bear, one sideways, and one bull cycle, which is shorter than standard factor research windows.
Distributional Assumptions: The GBM augmentation assumes log-normal returns, potentially under-representing tail events.

The paper concludes that the "mask-first" contract is the most significant factor in the system's success, proving that in quantitative trading, correctly modeling what cannot be traded is as important as predicting what can.