Validation · 8 min
Backtesting and paper trading prove different things
A backtest tests a hypothesis about the market. Paper trading tests your software against reality. Skipping either one means shipping on an untested assumption.
Blog
Proving that an edge exists before risking money on it — honestly.
A backtest tests a hypothesis about the market. Paper trading tests your software against reality. Skipping either one means shipping on an untested assumption.
Every leaked bit of future information makes a backtest look better. That is why leaks are so hard to notice — the evidence of the bug is a result you were hoping for.
Latency only matters in proportion to how fast your edge decays. Most strategies are not latency-sensitive and spend money as though they were.
Standard gradient boosting treats rows as exchangeable. Market data is not. We wrote a library where purged validation is the default and look-ahead is a lint error.
Assuming you fill at the touch is the most common way a backtest lies. A crude but measured slippage model is worth more than a sophisticated guess.
Splitting time-ordered data at random produces beautiful, meaningless results. The correct procedure is not much harder and it is the difference between a result and a story.
We write about what we build. If you need it built, get in touch — scoping is free.