Backtest vs forward test: why optimizer numbers differ from live

Key takeaways

1. 1Backtests replay history, walk-forward tests replay tuning, forward tests run live on new data.
2. 2Only forward results on paper, over hundreds of trades, show whether a setup works now.
3. 3Optimizer wins and forward results can disagree; trust forward results.

Three kinds of test

Each answers a different question. A backtest asks "would this have worked?". A walk-forward test asks "would tuning have worked, if I had done it at the time?". A forward test asks "does it work now?". Only the last one is not a prediction.

backtest

Run the strategy over past data with fixed settings

walk-forward test

Tune on one period, test on the next, repeat across history

forward test

Run it live, on paper or testnet, on data that did not exist when it was tuned

Why backtests flatter

1. 1Overfitting: with enough settings, some combination always fits the past.
2. 2Selection: when you test many strategies, the best-looking one is partly luck.
3. 3Costs: backtests often understate fees, slippage and funding.
4. 4Fills: simulated fills happen at prices real orders may not get.
5. 5Regime shift: the past may simply not look like the next month.

How TensorTrader’s optimizer guards against it

TT-Autotune scores each candidate on an anchored walk-forward with four consecutive test windows, where each window's model only sees bars up to its own end. The score is the mean Sortino across windows minus half their spread, which rewards consistency. Costs are included (5 bps fee, 2 bps slippage). Champions must be positive in their regime, beat the default settings on both Sortino and ROI, have enough trades spread across windows, sit on a plateau rather than a spike, and survive a replay of the full six-regime pack the way Pine actually runs it.

We are also explicit about what is not in the publish path: combinatorial purged cross-validation and block-permutation tests run in the research bench, not as publish gates today.

The forward record

Optimizer output says 91% of regime cells beat the default settings. Our forward testnet books, where those champions trade on real testnet matching engines, have not yet shown an edge after fees: the Binance testnet book is slightly negative across more than 4,300 closes, and the Hyperliquid testnet book lost 11.9% over 105 trades. Both statements are true at once, and the gap between them is exactly what this article is about.

Statistics worth reading

Sortino ratio

Return per unit of downside volatility; penalizes losses, not upside swings

Max drawdown

Worst peak-to-trough fall; what you must be able to sit through

Profit factor

Gross wins / gross losses; above 1 means winners outweigh losers

Win rate

Share of winning trades; meaningless without average win and loss size

Trade count

Too few trades and every other statistic is noise

How many trades is enough

There is no magic number, but a few dozen trades cannot distinguish skill from luck. Aim for hundreds across more than one regime before trusting a win rate or profit factor. TensorTrader's portfolio ranking follows the same idea: live results only influence a cell's rank after 40 live trades, and even that is a floor, not proof.

Frequently asked questions

How long should I forward test?

Until you have enough trades to trust the numbers, typically hundreds, across more than one regime.

Is walk-forward testing enough?

It is much better than a plain backtest, but forward results are still the ones to trust.

Why do TradingView backtests look better than my live results?

Simulated fills, commission settings and repainting can all flatter a backtest; live orders pay real fees and slippage.

What is the minimum I should check?

Net ROI after fees, max drawdown, trade count and whether the result held in more than one regime.

Keep reading

• Crypto trading automation, paper first: the complete guide

  How to automate crypto trading without losing your shirt: key safety, testnets, market regimes, DCA, leverage, fees, backtests versus forward tests, choosing a platform, and our own honest results.

• How TT-Autotune works: one alert, six regime champions

  TT-Autotune tunes the Lorentzian Classification strategy per token, per timeframe and per market regime, then syncs the winning settings into a single TradingView alert. Here is how the optimizer, the regime switch and the extension fit together, plus a live testnet ROI chart.

• How fees and slippage destroy trading bot ROI, with real numbers

  Why a strategy that looks profitable before costs can lose badly after them: round-trip costs, compounding over thousands of trades, and fill-based accounting from TensorTrader’s own books.

• Lorentzian Classification explained: the TradingView ML indicator, in plain English

  How jdehorty’s Machine Learning: Lorentzian Classification works: nearest neighbours, Lorentzian distance, its five features, filters and kernel regression, plus the default settings.