Backtest looked great, live results flat: look-ahead bias from using the candle close

This is an engineering post about simulation correctness, not about whether any strategy works. We built a small backtesting pipeline for an internal research project and tested a simple crossover rule on hourly candles. The backtest showed a smooth equity curve. Two weeks of paper trading with the same rule showed roughly flat results, with fills consistently worse than the backtest assumed.

Reproducing it

The backtest code, simplified:

df["fast"] = df["close"].rolling(10).mean()
df["slow"] = df["close"].rolling(50).mean()
df["signal"] = (df["fast"] > df["slow"]).astype(int)
df["position"] = df["signal"]                        # position on the same bar
df["ret"] = df["close"].pct_change()
df["strategy_ret"] = df["position"] * df["ret"]      # earns the bar's return

On bar t, the signal uses the close of bar t. The position is then credited with bar t's return, from the close of t-1 to the close of t. The strategy effectively bought before seeing the price it used to decide.

What we ruled out

• Fees and slippage alone. Adding realistic fees reduced returns but did not explain the gap between backtest and paper results.
• Data mismatch between backtest and live. Candles from the same source, aligned in UTC.

Root cause

Look-ahead bias. The signal for bar t is only known after bar t closes. The earliest you can act is at the open of bar t+1 (or later, depending on latency). Crediting the position with bar t's return uses information from the future relative to the decision. For momentum-style rules, this is especially flattering: the bar that triggered the crossover usually moved in the signal's direction, and the backtest captured that move.

What is happening under the hood

Every row in a backtest must respect a timeline: what was known when the decision was made, and at what price the trade could actually execute. Common sources of look-ahead:

• Using the current bar's close (or high, low) to decide and fill on the same bar.
• Indicators computed with centred windows or future data, such as normalising by statistics of the whole series.
• Resampling that labels a bar by its start time while including data up to its end.
• Joining external data (funding rates, on-chain metrics) by date without accounting for when it was published.

The fix is usually a one-line shift, and the hard part is noticing it.

Environment

Minimal reproduction

import numpy as np, pandas as pd
rng = np.random.default_rng(0)
close = pd.Series(100 * np.exp(np.cumsum(rng.normal(0, 0.01, 5000))))
ret = close.pct_change()
same_bar = (ret > 0).astype(int) * ret                 # "signal" = the bar's own direction
print(same_bar.sum())                                   # large positive on pure random walk
print(((ret > 0).astype(int).shift(1) * ret).sum())     # around zero, as it should be

On a random walk, any same-bar signal makes money. That alone is a good test.

The fix

1. Execute on the next bar. The position decided at the close of bar t earns returns starting from bar t+1:

df["position"] = df["signal"].shift(1).fillna(0)

For more realism, we model the fill at the next bar's open plus a slippage estimate, and charge fees on every change of position:

df["fill_price"] = df["open"].shift(-1)                          # next bar open
df["position"] = df["signal"].shift(1).fillna(0)
df["trade"] = df["position"].diff().abs().fillna(0)
df["strategy_ret"] = (df["position"] * df["close"].pct_change()
                      - df["trade"] * (FEE_RATE + SLIPPAGE_RATE))

2. An event-driven check. Vectorised backtests are fast and easy to get wrong. We added a slower event-driven simulator that processes bars one by one, only giving the strategy data up to the current bar, and compare its results with the vectorised version for every new rule.

3. Leakage tests. Two automated tests now run for every strategy:

• Random walk test: run the strategy on synthetic random-walk data many times; average returns before costs must be close to zero.
• Truncation test: compute signals on data up to time T, then on data up to T + k, and assert that signals before T do not change. If adding future data changes past signals, something uses the future.

def test_signals_do_not_use_future(strategy, candles):
    early = strategy.signals(candles.iloc[:3000])
    late = strategy.signals(candles.iloc[:3500]).iloc[:3000]
    pd.testing.assert_series_equal(early, late)

Alternatives and trade-offs

• Use a backtesting framework that enforces next-bar execution. Reduces mistakes; you still need to check custom indicators and data joins.
• Tick or order book level simulation. More realistic fills and slippage, much more data and complexity.

How to test the fix

After the shift, the random walk test returned a mean close to zero. The corrected backtest for the crossover rule matched the paper trading period's behaviour within the noise of two weeks.

Confirming in production

• Paper trading and backtest results over the same period now agree within a small tolerance once fees are included.
• Two other research rules that looked promising were found to depend on the same bias and were dropped before any capital was involved.

Regression guard

• Random walk and truncation tests are mandatory for every strategy in the research repository.
• Code review checklist item: "at what time is each input known, and at what price can the trade execute?"

Security implications

None in the usual sense. A biased backtest is a financial risk: decisions based on it allocate capital to something that never worked. Treat backtest code with the same review and testing discipline as production code.

Prevention checklist

• Decide on bar t, execute on bar t+1 at the earliest.
• Model fills at realistic prices with fees and slippage.
• Run a random walk test; same-bar profits on noise mean leakage.
• Check that adding future data never changes past signals.
• Cross-check vectorised backtests with an event-driven simulator.

References

• pandas Series.shift — https://pandas.pydata.org/docs/reference/api/pandas.Series.shift.html
• Bailey, Borwein, Lopez de Prado, Zhu: The Probability of Backtest Overfitting — https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2326253
• scikit-learn, data leakage pitfalls — https://scikit-learn.org/stable/common_pitfalls.html#data-leakage