Increase Returns, Cut Risk: The Power of Smart Diversification + Simple Rules That Work
In this week’s report:
You can beat the market without taking more risk: the power of smart diversification
Why simple trend-following beats "optimal" strategies in real markets
1. You can beat the market without taking more risk: the power of smart diversification
Can you have it All? Risk Reduction and Outperformance through Smart Diversification (October 21, 2025) - Link to paper
TLDR
Funds that minimize correlations among holdings (not just add more stocks) outperform by 4.1% annually with no additional risk
This "smart diversification" strategy actually performs better during recessions – when you need it most – boosting performance by 50%
The approach combines with concentration strategies: managers can focus on industries where they have an edge while selecting low-correlation stocks within them
What the authors did
Sanchez and Stein analyzed 24 years of US equity mutual fund data (2000-2023) to test whether the debate between diversification and concentration is a false choice.
They introduced "Smart Diversification" (SmartDiv), a new metric that measures portfolio diversification through the lens of correlations, not just stock counts.
The idea is that: You can achieve high diversification (under the SmartDiv measure) with just a handful of stocks that aren’t correlated, whereas a portfolio with hundreds of stocks may be less diversified if the stocks are correlated to each other. This would justify managers focusing on their sectors of expertise without sacrificing diversification.
How they built “Smart Diversification”
SmartDiv is elegantly simple: take every pair of stocks in a portfolio, calculate their return correlations using 12 months of daily data, then average them all. The measure is negated just so it’s easier to interpret: higher values mean better diversification (lower average correlations).
For a 100-stock portfolio, that's 4,950 pairwise correlations. Modern computing makes this trivial to calculate, even for funds holding thousands of securities. The key insight: correlation structure matters more than stock count.
Example from the paper
Warren Buffett's portfolio had an average correlation of 27.5% (0.275) among its four main holdings. Using SmartDiv, that becomes -0.275. The S&P 500's average pairwise correlation was 24% (0.24), which becomes -0.24.
Since -0.24 > -0.27, the S&P 500 is slightly more diversified, even though Buffett's portfolio is highly concentrated by stock count.
What they showed
The results are striking. Funds in the highest SmartDiv decile beat the lowest by 4.1% annually. By contrast, the highest industry-concentrated funds outperform the lowest by only 1.7% – and that difference isn't statistically significant.
When the authors ran both measures together in regressions, both remained significant with higher coefficients, proving they capture different sources of alpha.
The punch line: a skilled manager can concentrate in industries where they have superior information when they select stocks with low mutual correlations.
The recession advantage
During recessions, when stock correlations typically spike and diversification breaks down, SmartDiv delivers even more alpha. The coefficient jumps nearly 50% during NBER-defined recessions. Meanwhile, industry concentration underperforms in downturns.
Smart diversification provides precisely when investors need it most, and exactly what good risk management should do.
Why this works differently than picking winners
The authors prove SmartDiv reflects asset allocation skill, not stock selection. When they examined buy-sell spreads across deciles, industry-concentrated funds in the top decile showed significant trade-level alpha (0.22% monthly). SmartDiv deciles showed no such pattern.
The Buy-Sell Spread Test
The authors tracked every trade funds made (inferred from quarterly portfolio changes) and measured:
Average returns of stocks the fund bought
Average returns of stocks the fund sold
The difference between them
If a fund manager has genuine stock-picking skill, they should:
Buy stocks that subsequently go up
Sell stocks that subsequently go down
Show a positive buy-sell spread
What they found for Industry Concentration
Funds with low industry concentration (decile 1): Buy-sell spread = -0.14% per month (their buys underperformed!)
Funds with high industry concentration (decile 10): Buy-sell spread = +0.22% per month
Difference (10-1): +0.36% monthly, statistically significant
This proves managers with high industry concentration have stock-picking skill – they're making informed bets based on research, private information, or analytical edge.
However, when they applied the same buy-sell spread test to Smart Diversification, there was no meaningful difference across deciles, meaning that funds with higher Smart Diversification didn’t have superior stock-picking skill. Therefore, their superior returns didn’t come from stock-picking but rather from superior diversification.
What they found for Smart Diversification
All deciles: Buy-sell spreads were statistically insignificant
Top decile (10) minus bottom decile (1): No meaningful difference
No systematic pattern across deciles
This proves SmartDiv managers are NOT outperforming through stock selection.
This means SmartDiv doesn't require proprietary information or research advantages – just disciplined correlation management. That makes it accessible to far more investors than strategies requiring superior stock-picking.
2. Why simple trend-following beats "optimal" strategies in real markets
The Fragility of Optimization: How Estimation Risk Undermines Active Investing with Autoregressive Forecasts (October 20, 2025) - Link to paper
TLDR
Optimal capital allocation strategies that maximize Sharpe ratios are highly fragile to estimation error and often underperform simple trend-following in realistic sample sizes
With typical market data, a basic trend-following rule outperforms both buy-and-hold and theoretically superior optimization approaches
The culprit: sophisticated strategies rely on precise forecast magnitudes, while simple rules only need directional accuracy
What the author did
Zakamulin analyzed two active strategies based on autoregressive return forecasts using S&P 500 market parameters:
A simple trend-following rule: If forecast > risk-free rate, invest 100% in stocks. Otherwise, hold 100% cash. It's a binary on/off rule.
Optimal mean-variance capital allocation strategy: Continuously adjust your stock allocation based on both the sign AND magnitude of the forecast. If you forecast +2%, you might hold 120% stocks (using leverage). If you forecast +0.5%, maybe only 60% stocks.
1.Autoregressive return forecasts means using past returns to predict future returns. Specifically, the paper uses an AR(p) model where: Today's return = some weighted average of the last p months' returns + noise.
2.This means that the author set the model inputs to match long-term stock market behavior:
Mean return: μ = 10% per year
Volatility: σ = 20% per year
Risk-free rate: 3% per year
Trend strength: All AR coefficients set to 0.03 (weak persistence)
Model order: p = 10 lags (based on empirical estimates from “Optimal Trend Following Rules in Two-State Regime-Switching Models”, Zakamulin & Giner 2022)
These aren't random numbers—they reflect what you'd actually observe in S&P 500 monthly returns over long periods.
The key idea: he tracked how both strategies degrade when you estimate parameters from finite samples rather than knowing them perfectly.
The theory versus reality gap
Here's where it gets interesting. With perfect parameter knowledge, the optimal capital allocation strategy crushes trend-following, delivering a Sharpe ratio 48% higher than buy-and-hold versus trend-following's 22% improvement.
But introduce estimation risk and everything inverts. The optimal strategy needs to estimate not just forecast direction but also its magnitude and the precise relationship between forecasts and realized returns.
Each parameter estimate carries noise that compounds through the optimization. With 35 years of monthly data – a substantial sample – the optimal strategy requires that full period just to match buy-and-hold performance. Trend-following beats buy-and-hold in half that time (i.e., requires less data to beat buy-and-hold).
The 35+ years refers to the estimation window (how much historical data you use to calibrate the model).
Trend-following wins with less data, not more
Trend-following beats buy-and-hold with just 12 years of monthly data for parameter estimation
Optimal capital allocation needs 35 years of data to match buy-and-hold
Trend-following outperforms optimal allocation for any estimation window under 53 years
So if you're a hedge fund with only 20 years of data on a particular asset, you're in the sweet spot where:
Simple trend-following already delivers positive alpha
"Optimal" strategies are still too starved for data and underperform
Why this matters for realistic hedge fund scenarios
Hedge funds typically face:
New asset classes with limited history (crypto, new sectors, emerging markets)
Regime changes that make old data less relevant (you might only trust post-2008 data)
Strategy capacity constraints that require moving to less-liquid markets with shorter data
In all these cases, trend-following's advantage increases because estimation risk is worse. The optimal strategy degrades faster than the simple rule as data becomes scarcer.
Why magnitude sensitivity kills optimization
The core mechanism is subtle but powerful. Optimal strategies scale position size continuously with forecast strength. When estimation error inflates a mediocre signal, the strategy over-allocates. When noise suppresses a strong signal, it under-allocates. This magnitude dependence creates variance that isn't compensated by higher returns.
Trend-following truncates this error. A false positive just means full exposure (capped risk). A false negative means holding cash. The binary structure – invest or don't – naturally dampens the impact of noisy forecasts. You only need to get direction right, not magnitude.
The paper derives analytical approximations showing the expected forecast-return correlation decays with estimation risk as √(n/T), where n is model complexity and T is sample size. That correlation drives both strategies' performance, but the optimal strategy's continuous sizing amplifies the penalty from any correlation degradation.
The empirical proof
Testing on 12 industry portfolios and 10 size-sorted portfolios confirms the theory.
Trend-following delivered Sharpe ratios averaging 0.465 versus 0.413 for buy-and-hold – a statistically significant 12.5% improvement. The optimal capital allocation strategy averaged just 0.407, underperforming the passive benchmark.
The pattern held across portfolios: simplicity won. Even the sophisticated hedge fund manager with 60+ years of data faces meaningful estimation risk that favors robust rules over precise optimization.
The bottom line
Stop optimizing on noisy signals. When return predictability is weak – as it typically is – estimation error makes theoretically optimal strategies practically inferior. A basic trend-following rule that just tracks forecast direction delivers more reliable outperformance than mean-variance optimization.
Disclaimer
This publication is for informational and educational purposes only. It is not investment, legal, tax, or accounting advice, and it is not an offer to buy or sell any security. Investing involves risk, including loss of principal. Past performance does not guarantee future results. Data and opinions are based on sources believed to be reliable, but accuracy and completeness are not guaranteed. You are responsible for your own investment decisions. If you need advice for your situation, consult a qualified professional.
