Robustness checks — does the edge degrade gracefully?

The robustness mindset

Everything in D4 so far measured a strategy at one point: one parameter set, one period, one universe, one cost assumption. The last discipline is to poke it. A real edge degrades gracefully under perturbation; an overfit one collapses. Real effects are properties of markets, so they should survive small changes to everything that was your choice rather than the market's.

The four standard checks

1. Parameter perturbation. Vary each parameter ±20–50% around your choice and recompute Sharpe. Plot the surface. A plateau — broadly similar performance across the neighborhood — supports a real effect. A cliff — a sharp peak that collapses one step away — is the signature of a parameter fit to noise.

2. Subperiod analysis. Split the history into eras (for example thirds, or pre/post 2015, or crisis vs calm) and evaluate each separately. A real edge should be positive in at least 2 of 3 eras. A strategy whose entire P&L comes from one regime is a bet that the regime returns.

3. Universe perturbation. Randomly drop 20% of the assets and rerun, several times. If the result depends on a handful of specific names, you found those names' histories, not a strategy.

4. Cost stress. Double your cost assumption from D4-03 and rerun. An edge that dies at 2× assumed costs is a bet on your cost model being exactly right — and D4-03 showed how wide that uncertainty is.

The plateau principle

When you must choose a parameter, pick the middle of a stable region, never the peak. The peak is partly selection bias (D4-05, within a single strategy); the plateau's center is the point most likely to still work when the noise re-rolls. Quantify stability crudely as the range of neighboring Sharpes relative to the best: near 0 is a plateau, near 1 is a cliff. You'll implement exactly this below.

The full D4 pre-deployment checklist

D4-01's checklist, extended by everything since:

• Signals lagged ≥1 period; universe point-in-time; dividends/splits handled (D4-01, D4-02)
• Equity curve, max drawdown, and turnover computed and reported (D4-02)
• Costs documented, justified, and shown as a sensitivity table (D4-03)
• Walk-forward with purging/embargo; final holdout touched exactly once (D4-04)
• Research log kept; trial count known; DSR or equivalent deflation applied (D4-05)
• Parameter plateau, not peak; works in ≥2/3 of subperiods; survives universe drops and 2× costs (D4-06)
• Paper-traded before live capital, with divergence from backtest monitored (D4-04)

The honest close

The goal of backtesting is not to make a strategy look good. It is to give the strategy every chance to reveal itself as noise — and to deploy only what refuses to. Most ideas will die on this checklist. That is the checklist working. The few that pass are the only ones worth your capital, and even they get paper-traded first.

Try it

Teach it

Your generative activity: a friend shows you a backtest — Sharpe 1.4, lookback 20, tested on 2015–2025 tech stocks, 5 bps assumed costs. Teach them, out loud or in writing, the four robustness checks as a review of their strategy: name each check, say exactly what you would rerun, and state what result would make you trust or reject the edge. Finish by explaining the plateau principle in your own words.

Predict before the next lesson

Track D4 is complete — you now own the methodology. Track D5 begins the payoff: the classic strategy archetypes — momentum, mean reversion, carry, value, and friends. Predict:

• Momentum (buy what went up) and mean reversion (buy what went down) sound like opposites. On what timescales does each tend to appear?
• For each archetype, who is on the other side of the trade — and why might they knowingly pay you?