A strategy can be tuned to fit any past perfectly — that proves nothing. The real question is whether it works on data it has never seen. Out-of-sample testing is how you answer it, and it is the dividing line between a genuine edge and an elaborate description of history.
Train and test (walk-forward)
The method: split your history into a 'train' (in-sample) period where you build and tune the strategy, and a separate 'test' (out-of-sample) period it has never touched. If the edge holds on the unseen test data, it has a chance of being real. If it evaporates — as overfitted strategies reliably do — you have learned the 'edge' was just curve-fitting. Walk-forward repeats this across many train/test splits for robustness.
Why one good window isn't enough
A single out-of-sample win can still be luck. The strongest standard is robustness across many disjoint windows and regimes — does the edge survive a trending month and a ranging one, this quarter and last? This project's own research is the cautionary tale: a strategy looked positive across two windows, then dissolved to nothing under five disjoint windows. Two windows was luck; five revealed the truth. Demand multi-window evidence before believing an edge.
Paper and live forward testing
Beyond historical out-of-sample, the final proof is forward testing: running the strategy on live data going forward (on paper first, then small real size) — which surfaces real-world frictions a backtest misses (slippage, latency, your own discipline). The honest sequence is: rules → costed backtest → multi-window out-of-sample → paper-forward → small live. Skip steps and you are gambling, not trading. 'Edge first, prove it, then size' is the whole discipline.