💰 quantitative finance

Which Portfolios? The Construction Dependence of Factor Model Performance

This paper demonstrates that the performance rankings of factor models are significantly dependent on test-asset construction choices, such as selection, weighting, and rebalancing strategies, rather than being solely determined by the models themselves.

Original authors: Useong Shin

Published 2026-06-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Useong Shin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to judge which of three different weather forecasters is the best. You have a long list of past storms, and you want to see who predicted the rain, wind, and temperature most accurately.

This paper is about how the way you organize your data changes who you declare the winner.

The author, Useong Shin, argues that in the world of finance, we often ask, "Which stock market model is the best?" (like asking "Who is the best weather forecaster?"). The paper says the answer depends entirely on how you build the test (the portfolios). If you change the rules of the game, the winner changes.

Here is a breakdown of the paper's findings using simple analogies:

1. The "Home-Field Advantage"

The paper starts by looking at how models perform on the specific types of stock groups they were originally designed to study.

The Analogy: Imagine a soccer team that practices exclusively on muddy fields. If you test them on a muddy field, they look like geniuses. If you test them on a pristine, dry grass field, they might look clumsy.
The Finding: The "Fama-French" models (a famous family of financial models) do very well when tested on the specific types of stock groups they helped create. The "q5" model (a newer model) does very well on the specific groups it was built to explain. This is called "Home-Field Bias." It doesn't mean the models are fake; it just means they are tuned to a specific type of terrain.

2. The "Random Shuffle" Experiment

To see if these models are actually good at everything, the author didn't use the familiar, pre-organized groups. Instead, he created random portfolios.

The Analogy: Instead of testing the soccer team on their usual muddy field, he threw 1,000 random players into a mix, put them on a random field, and asked them to play. He didn't pick the "best" players; he just grabbed a random handful of stocks.
The Method: He varied the rules of this random game:
- How they picked players: Did they pick big companies (like picking the tallest players) or pick everyone equally (like a lottery)?
- How they managed the team: Did they keep the same lineup every day (Constant Weight), or did they let the players who performed well naturally take up more space in the lineup over time (Buy-and-Hold)?

3. The Surprising Results: The "Rebalancing" Twist

This is the most important part of the paper. The author found that how you manage the portfolio after it's built changes the winner.

The "Buy-and-Hold" Strategy (Letting the team drift):
- The Analogy: You pick a team, and then you let the players who score goals get more playing time automatically, while the ones who miss get less. You don't touch the lineup for a year.
- The Winner: In this scenario, the FF5 and FF6 models (the older, more complex families) usually win. They handle the "drifting" lineup well.
The "Constant Weight" Strategy (Daily Rebalancing):
- The Analogy: Every single morning, you force the team back into the exact same lineup you started with, regardless of who scored yesterday. You are constantly selling the winners and buying the losers to keep the balance.
- The Winner: In this scenario, the FF3 model (a simpler, older model) is the clear winner. It is the most stable.
- The Loser: The q5 model (the newer, "smart" model) performs surprisingly poorly here. It leaves huge "pricing errors" (mistakes) when you force the lineup to reset every day.

4. The "q5" Paradox

The paper highlights a confusing situation with the q5 model.

The Paradox: If you look at the "Maximum Sharpe Ratio" (a fancy way of asking, "Which model offers the best theoretical investment opportunity?"), q5 is the champion. It seems to have the most powerful tools.
The Reality: However, when you actually test it on random portfolios (especially the ones that get rebalanced daily), q5 makes the most mistakes.
The Lesson: Having the "best theoretical tools" (a high Sharpe ratio) doesn't mean the model will predict the returns of a specific, randomly built portfolio accurately. It's like having a Ferrari engine (q5) but putting it in a car that drives poorly on bumpy roads (daily rebalancing).

5. The Main Conclusion: "Which Portfolios?" Matters More Than "Which Model?"

The paper concludes that you cannot simply ask, "Which financial model is the best?"

The Answer: You must ask, "Which model is best for these specific portfolios, managed in this specific way?"

The author argues that the way we build our test portfolios (picking stocks, weighting them, and deciding when to rebalance) is not just a boring technical step. It is a design choice that dictates the results.

If you change the rules (e.g., from "let the winners grow" to "reset every day"), the ranking of the models flips completely.
The FF3 model is the "Swiss Army Knife" that stays stable no matter how you manage the weights.
The FF5/FF6 models are great if you let the portfolio drift naturally.
The q5 model is powerful in theory but struggles when you force daily rebalancing.

Summary

Think of financial models as different types of shoes.

Some shoes are great for running on a track (FF models on their own data).
Some are great for hiking in the mud (FF models on "Buy-and-Hold" portfolios).
Some are great for walking on a treadmill (FF3 on "Constant Weight" portfolios).
And some expensive, high-tech shoes (q5) look amazing in the store and have the best specs, but they might slip and slide if you try to run on a specific type of wet floor (daily rebalancing).

The paper's message is: Don't just look at the shoe specs; look at the terrain you are walking on. The "best" shoe depends entirely on the path you choose to walk.

Technical Summary: The Construction Dependence of Factor Model Performance

1. Problem Statement

The empirical performance of asset-pricing factor models is traditionally evaluated based on the model's factors and the resulting pricing errors (alphas). However, this paper argues that performance rankings are not invariant to the construction of the test assets used for evaluation. While characteristic-sorted portfolios (e.g., size, value, momentum) are standard for reducing idiosyncratic noise, they often align closely with the specific characteristics used to construct the factors themselves, potentially creating a "home-field bias."

The central question addressed is whether the fit of a factor model generalizes to portfolios formed by different rules. Specifically, the paper investigates how variations in stock selection, initial weighting, portfolio breadth, and post-formation weight management (holding and rebalancing rules) alter the relative performance of competing factor models. The study posits that test-asset construction is not a neutral preprocessing step but a fundamental design choice that determines which dimensions of pricing error are emphasized.

2. Methodology

Data and Universe

The study utilizes a broad CRSP investable universe of common stocks (NYSE, AMEX, NASDAQ) from January 3, 1967, to December 31, 2024. The universe is screened for investability using market equity and trading liquidity (63-day average dollar volume) with hysteresis to prevent frequent entry/exit. The final universe retains approximately 99.7% of aggregate market equity while excluding the most illiquid tail.

Random Portfolio Construction

Instead of characteristic sorting, the paper forms characteristic-unsorted random portfolios. The construction process involves four distinct dimensions:

Stock Selection: Stocks are sampled either with probabilities proportional to market equity (MEP) or uniformly (UNIF). Selection ratios ( $\rho$ ) range from 0.5% to 100%.
Initial Weighting: Selected stocks receive Equal Weighting (EW), Value Weighting (VW), Square-Root Market Equity Weighting (SQME), or Random Weighting (RAND).
Weight Management (Dynamic): The same initial stocks and weights are subjected to two distinct rules:
- Buy-and-Hold (BH): Weights drift endogenously with realized returns.
- Constant Weight (CW): Portfolios are rebalanced daily to restore initial target weights.
Portfolio Breadth: The number of stocks in the portfolio is varied to test the impact of diversification.

Models Evaluated

The analysis compares six factor models:

CAPM
Fama–French 3-Factor (FF3)
Carhart 4-Factor
Fama–French 5-Factor (FF5)
Fama–French 6-Factor (FF6)
Hou–Xue–Zhang 5-Factor (q5)

Evaluation Metrics

Performance is assessed using:

Factor Spanning: Direct tests of whether one model's factors span another (Barillas and Shanken, 2017), measuring the expansion of the mean–variance opportunity set.
Pricing Errors: Mean absolute pricing errors, root-mean-square errors, tail errors, the share of significant alphas, and time-series $R^2$ .
Home-Field Bias: Comparison of model rankings on conventional characteristic-sorted portfolios (Fama–French, global-q, industry) versus random portfolios.

3. Key Results

A. Factor Spanning vs. Portfolio Fit

Spanning tests reveal an asymmetry in the mean–variance opportunity sets. The q5 model offers the broadest sample opportunity set, with its specific factors (particularly Expected Growth, EG) contributing significantly to the maximum Sharpe ratio. However, spanning a larger opportunity set does not imply smaller pricing errors on specific test assets. The paper demonstrates that a model can dominate in spanning tests while leaving large, construction-sensitive pricing errors on random portfolios.

B. Home-Field Bias

On conventional characteristic-sorted portfolios, models perform best on assets aligned with their own research lineage ("home-field bias").

FF models perform best on Fama–French characteristic sorts.
q5 performs best on global-q anomaly portfolios.
Rankings shift significantly when moving to industry portfolios or random portfolios, indicating that strong fit on familiar sorts does not generalize to external construction rules.

C. Construction Dependence on Random Portfolios

The performance of factor models is highly sensitive to the portfolio construction design:

Buy-and-Hold (BH): Under BH and low-frequency rebalancing, FF5 and FF6 generally perform strongly, often outperforming FF3 and q5. The exact leader depends on the interaction between sampling (MEP vs. UNIF) and initial weighting.
Constant Weight (CW): Under daily rebalancing, FF3 emerges as the most stable model, exhibiting the smallest pricing errors in the majority of designs.
The q5 Anomaly: While q5 achieves the highest maximum Sharpe ratio in spanning tests, it leaves comparatively large and construction-sensitive pricing errors on random portfolios, particularly under CW. In many CW designs, q5 produces widespread positive intercepts (underpricing).
Interaction Effects: The marginal contribution of specific factors (e.g., adding ROE or EG to q3) is not invariant. A factor that reduces pricing errors in one design (e.g., industry portfolios) can increase them in another (e.g., Fama–French sorts or CW random portfolios).

D. Impact of Rebalancing Frequency

Varying rebalancing frequency from annual to daily shows that model rankings are not monotonic.

FF6 and Carhart tend to perform better at lower frequencies (annual, quarterly).
FF3 becomes dominant as rebalancing frequency increases (weekly, daily).
q5 generally deteriorates with increased rebalancing frequency, except in specific designs (e.g., MEP-VW), where it performs well under CW.
The divergence between BH and CW results isolates the effect of dynamic weight management, showing that the "return process" being priced changes fundamentally with the rebalancing rule.

E. Robustness

The findings hold across various robustness checks:

NYSE-only samples: The construction dependence remains, though specific rankings shift (e.g., FF5/FF6 weaken in NYSE-only BH).
Microcap exclusion: Removing stocks below the NYSE 20th percentile reduces error magnitudes but does not eliminate the relative performance differences or the sensitivity to construction rules.
Portfolio Breadth: Increasing diversification reduces noise but does not eliminate the systematic pricing errors specific to certain models (e.g., q5 retains large errors even at 100% breadth under CW).

4. Contributions and Significance

The paper makes three primary contributions to the literature on asset pricing:

Systematic Random-Portfolio Design: It develops a framework for sampling alternative tradable directions from the CRSP return space without reproducing the characteristic sorts used to validate candidate models. This allows for the evaluation of models on assets generated by external rules.
Separation of Spanning and Absolute Fit: The study empirically distinguishes between factor spanning (the ability to expand the mean–variance opportunity set) and portfolio-level absolute fit (the magnitude of pricing errors on a prespecified set). It demonstrates that a model can offer a broader opportunity set while simultaneously leaving larger pricing errors on specific portfolio populations.
Dynamic Weight Management as Design: The paper treats dynamic weight management (BH vs. CW) as an integral part of test-asset construction. It shows that changing the holding or rebalancing rule alters the return process being priced, which can change both the magnitude and direction of estimated pricing errors.

Significance:
The paper concludes that there is no "universally superior" factor model. The question "Which model?" cannot be separated from "Which portfolios?" and "How are they managed?" Test-asset construction is not a neutral preprocessing choice but a critical component of the model-evaluation design. Consequently, empirical comparisons should report both direct factor spanning and the level, direction, and stability of pricing errors across alternative portfolio-generation and weight-management rules to provide a complete picture of a model's performance.