Building a trading strategy on AI: what that actually involves

“Can you build me an AI trading strategy?” is a question we get weekly. The answer is yes, and the useful part of the answer is what that sentence actually contains — because the model is maybe ten percent of it.

Here is the whole pipeline as we build it, in order, with the failure mode at each step.

1. Decide what you are predicting

Before any data is collected, the target has to be a specific, measurable, tradeable quantity. “Will it go up” is not one. Real targets look like:

• Direction of the mid price over the next N seconds, given that the move exceeds costs.
• Maximum favourable excursion within a horizon — how far it goes your way before it goes against you.
• Maximum adverse excursion — used to size the stop, not the entry.
• Probability that a move has exhausted, which is a different question from direction.

Our microstructure engine predicts three of these with separate model heads: direction, MFE and MAE. Predicting direction alone tells you where to go and nothing about how much to risk, which is half a strategy.

2. Collect data before you need it

This is the step people skip and then cannot recover from. Market microstructure data is not available retroactively at the resolution that matters. If you want L2 book snapshots, taker flow and liquidations at event resolution, you have to have been recording them.

Our first phase on any research project is collect — a process that does nothing but record order book, trades and liquidations to disk. It is boring, it produces no results for weeks, and every project that skipped it ended up limited by data quality rather than by modelling.

3. Features, and the honesty problem

Feature engineering is where domain knowledge lives, and where look-ahead bias gets in. Every feature must be computable at decision time from information that existed at decision time — which sounds obvious and is violated constantly.

Our pump-fade model uses 59 features across four families: episode structure (how many legs the move has run, how long, deepest pullback), higher-timeframe context, the BTC background, and instrument liquidity. An automated audit recomputes each one on a truncated series and found three genuine leaks, all of them alignment bugs in how funding and open interest were joined to the bar series.

# Every feature gets this treatment before it is allowed into a model.
for name, fn in FEATURES.items():
    full    = fn(df)                      # normal pipeline
    partial = fn(df.iloc[:t + 1])         # future physically absent
    if not close_enough(full.iloc[t], partial.iloc[-1]):
        raise LeakError(name)             # not a warning — a build failure

4. Validation that does not flatter

Random cross-validation on time-ordered data produces beautiful, meaningless numbers. The honest version is walk-forward with purge and embargo: training samples whose label windows overlap the validation set are removed, plus a margin after it for autocorrelation.

We made this the default rather than an option in TideGBM, because the honest path being harder than the convenient one is how shortcuts win under deadline pressure.

5. The expected-value gate

This is the step that separates a model from a strategy. A prediction is not a trade. The model outputs a number; the strategy decides whether acting on it has positive expected value after fees, slippage and latency.

def should_trade(pred, book, cfg):
    edge = pred.direction_conf * pred.expected_mfe      # what we expect to capture
    cost = (cfg.fee_in + cfg.fee_out
            + slippage_estimate(book, size)
            + expected_funding(cfg.hold_periods))
    if edge <= cost * cfg.margin:
        return None                                     # most predictions end here
    size = kelly_fraction(pred, cfg) * cfg.equity
    return Intent(size=size, stop=pred.expected_mae * cfg.stop_k)

In practice this rejects the large majority of predictions, and that is correct. The rule we write into the design: if honest out-of-sample EV is not positive after fees, slippage and latency stress, the system does not trade. It lives in code, not in a document.

6. Deployment is a different discipline

A model that works in a notebook and a model that works in a live loop are not the same artefact. The realtime path has requirements research code does not:

• Bounded, predictable inference latency — no allocation surprises on the hot path.
• Feature computation identical to training. A subtle difference between the research and production feature code is the single most common cause of a model that “worked in backtest”.
• Versioning, so you know which model produced which trade.
• A rollback path that does not require a deploy.
• Monitoring on the prediction distribution, not just on P&L.

Our engines run the same feature code in backtest, paper and live, with a mode flag rather than separate implementations — precisely so that this class of bug cannot exist.

What we will and will not claim

We will build this pipeline for you, and we will be honest at every stage about whether the result justifies going live. We have told clients that the answer was no more than once.

What we will not do is sell a model as an edge. Machine learning is a flexible function approximator applied to features you chose, validated by a procedure you can audit. It finds interactions a human would not write by hand. It does not know anything you did not give it, and it fails confidently — which is why the risk engine matters more, not less, when a model is involved.

If you have data and a hypothesis, the honest first project is not a bot. It is the collect phase and a purged backtest — the cheapest possible way to find out whether the bot is worth building.