3 Things to Do When a Backtest Looks Too Good to Be True
Every quantitative researcher wants to discover a strategy with exceptional historical performance. Ironically, the better a backtest looks, the more aggressively it should be challenged.
Every quantitative researcher wants to discover a strategy with exceptional historical performance. Ironically, the better a backtest looks, the more aggressively it should be challenged.
1. Look for Information the Strategy Could Not Have Known
The first question is whether the backtest accidentally used future information. Look-ahead bias can enter a model in subtle ways. A signal may use the final value of a candle before that candle has closed. An indicator may rely on data that becomes available later. A backtest may enter at a price that occurred before the condition triggering the trade was known.
At every decision point, reconstruct exactly what information would have been available at that moment. If the model needs anything from the future to make the decision, the result is invalid regardless of how impressive the equity curve appears.
2. Make the Assumptions Worse
A robust strategy should not require perfect conditions. Increase assumed slippage. Move the entry slightly later. Make the exit slightly less favorable. Change a parameter by a reasonable amount. Test neighboring time windows. Include commissions and realistic contract behavior.
The objective is not to intentionally destroy the strategy. It is to determine whether modest deterioration in assumptions causes the entire edge to disappear. If the strategy remains economically interesting after conservative assumptions are introduced, the evidence becomes more compelling.
3. Try to Break the Strategy
Once a strategy looks promising, the researcher's job should change from proving it works to trying to prove it does not. Test different historical periods. Remove the best trades. Examine the worst months. Separate development and validation data. Look for parameter cliffs where a tiny change transforms a profitable model into an unprofitable one.
A strategy that survives serious attempts to break it is more interesting than one that simply produces the highest number on an optimization table.
The Quantivix Perspective
At Quantivix, quantitative research is designed to turn market observations into explicit, testable rules. Ideas are evaluated through historical testing, sensitivity analysis, realistic execution assumptions, and ongoing monitoring. Artificial intelligence and computational tools can accelerate that process, but speed does not replace validation.
The objective is not to eliminate uncertainty. Markets make that impossible. The objective is to understand the evidence well enough to define when a strategy should act, how much risk it should take, and what conditions deserve additional scrutiny.
This material is provided for informational and educational purposes only and does not constitute investment advice, an offer to sell, or a solicitation of an offer to purchase any security, commodity interest, or investment product. Futures involve substantial risk and are not suitable for all investors. Past or hypothetical results do not guarantee future performance.
