Latency: what it actually costs, and when it is worth paying for

Latency is the most over-discussed and least measured property of a trading system. The useful question is never “how fast is it” but “how much does the edge decay in the time it takes to act”.

Where the time goes

Between an event happening and your order resting on the book:

Note where the variance lives. Network segments are large but stable; the strategy segment is the one that occasionally takes a hundred times longer than usual, and it does so precisely when the market is busy.

Tail latency is the number that matters

Average latency is close to useless. Your system's behaviour is determined by its p99 — and the p99 arrives during exactly the volatile moment when the decision was worth the most.

The usual causes of a bad tail are within your control: garbage collection, a blocking log write, a lock held across an I/O call, a dataframe operation on the hot path. Our microstructure engine keeps features in preallocated RAM ring buffers with no pandas anywhere in the realtime loop, for exactly this reason. Pandas is excellent for research and it allocates unpredictably, which is a different requirement.

Slippage is the price of latency

Latency itself costs nothing. What costs money is the price moving between your decision and your fill. That is measurable directly: record the mid price at decision time, the fill price, and the difference.

# Log this on every fill. It is the only honest slippage number you will get.
slip_bps = (fill_price - decision_mid) / decision_mid * 10_000
if side == "sell":
    slip_bps = -slip_bps

metrics.observe("slippage_bps", slip_bps,
                tags={"symbol": symbol, "urgency": order_type})

Collected over a few thousand fills, that distribution tells you what latency is actually costing. Compare it against the assumption in your backtest. If the backtest assumed 1 basis point and reality is 4, the strategy you validated is not the strategy you are running.

Decide with a budget, not a preference

Work out the edge decay rate for your signal — how much of the expected move is gone after 10 ms, 100 ms, one second — and compare it to what an improvement costs. Concretely:

• Signals on closed candles, minutes apart: a hundred milliseconds is noise. Spending on colocation here buys nothing.
• Order-book imbalance and flow signals: decay is measured in the tens of milliseconds. Network location starts to matter.
• Market making: you are exposed to adverse selection continuously, and cancel latency directly determines how often you are picked off.
• Cross-venue arbitrage: the spread you are capturing exists for as long as it takes someone faster to take it.

Most strategies live in the first two rows. Most latency spending is justified by the last two.

The free improvements first

1. Reuse connections. TLS handshakes on every request are the most common self-inflicted wound.
2. Never block the event loop on logging, disk or a chat API.
3. Preallocate. Do not allocate in the hot path.
4. Move the host to the same region as the venue before considering anything exotic.
5. Prioritise cancels above quote refreshes in the request queue.
6. Only then discuss faster hardware.

In our experience the first five recover more milliseconds than the sixth, at a fraction of the cost — and unlike the sixth, they also reduce the tail rather than just the mean.

Test the sensitivity, then decide

Before spending anything, run the backtest across a latency grid. If profitability is flat from 10 ms to 200 ms, latency is not your problem and you have just saved a budget. If it collapses, you have learned that the strategy is a bet on infrastructure — which is a legitimate bet, but it should be a conscious one.