Synthetic market data: useful for stress testing, dangerous for training

Financial history is short and it happened once. Every backtest is a sample of size one from the distribution of possible histories, which is why a strategy can look excellent and be an artefact of one particular path.

Synthetic data is the obvious response, and it is genuinely useful — for exactly one of the two things people use it for.

The reasonable uses

Robustness under resampling. Block bootstrap the historical return series: sample contiguous blocks with replacement to build alternative paths that preserve short-range autocorrelation. Run the strategy on a few hundred of them and look at the distribution of outcomes rather than the single number.

This answers a question the single backtest cannot: how much of the result was luck. A strategy whose Sharpe ranges from 0.2 to 2.4 across resampled paths is not a 1.3 Sharpe strategy — it is a coin flip with a nice mean.

Stress scenarios. Construct conditions that did not occur but plausibly could: a 30% gap with no intermediate prints, a venue outage mid-position, book depth at a tenth of normal, funding at ten times baseline. You are not predicting these; you are checking that the system degrades rather than detonates.

Filling gaps for engineering tests. Generating a synthetic order book to test that your parser handles a crossed book, a zero-size level or a sequence gap. No claim about realism is needed — you are testing code paths.

The dangerous use

Training a signal model on synthetic data. The problem is structural rather than a matter of generator quality.

A generator embeds assumptions about how markets behave. A model trained on its output learns those assumptions, not the market. It will perform beautifully on held-out synthetic data — because that data comes from the same generator — and the validation gives you confidence in a circular result.

The subtler failure

Standard generators — GBM, GARCH, most GAN variants trained on returns — reproduce the statistical properties they were designed around and miss the ones that determine whether a strategy survives:

The missing column is where strategies die. A model trained on synthetic data has specifically never seen the conditions that matter most.

A workable protocol

1. Validate on real out-of-sample data with purge and embargo. This is the result.
2. Block bootstrap to get a distribution of outcomes. Report the range, not the point.
3. Construct explicit stress scenarios and check for graceful degradation.
4. Use synthetic data for engineering tests without any claim about realism.
5. Do not train the signal on synthetic data. If you have too little real data, that is information about the strategy, not a problem to engineer around.

That last point is the honest one. “Not enough data to validate” is a conclusion. Generating more of it does not change the conclusion; it just makes it harder to see.