Backtest limitations
The complete list of what a Stretus backtest does not model, does not know, and cannot tell you. Read it once properly; it is the difference between using a backtest and being used by one.
Execution mechanics
| Not modelled | What it means for your result |
|---|---|
| Partial fills | An order fills entirely or not at all. In reality a large order in a thin book fills in pieces at worsening prices |
| Queue position | No modelling of where your order sits in the book. On a limit order at a busy level, position determines whether you fill at all |
| Latency | Zero delay between signal and fill. Live there is network time, broker processing and venue matching |
| Spread dynamics | Slippage is a constant 5 bps by default. Real spreads widen exactly when you most want to exit |
| Market impact of size | The slippage allowance does not scale with your order. A position large relative to volume moves the price against itself |
| Order rejection | Every order fills or does not fill on price. Live orders are also rejected for margin, permissions, rate limits and venue state |
The intra-candle path
The most consequential mechanical limitation, and the one to understand precisely.
The simulator sees each candle's open, high, low and close, not the order in which the price visited them. When both your stop and your target lie inside one candle's range, it cannot know which was hit first.
The stop is assumed to fill first. That is the conservative reading and it is the right default. But it means a strategy with a tight stop and a nearby target may show worse results than reality, and one that genuinely got stopped shows the same thing. You cannot distinguish the two cases from the result.
On short timeframes with tight brackets, this is a material effect.
Data
| Limitation | Consequence |
|---|---|
| Window floor: 1 January 2024 | Every backtest tests one era's market conditions. See Choosing a test period |
| Survivorship bias | Available symbols are symbols that exist now. Delisted equities and dead tokens are absent from every universe test |
| Candle-boundary evaluation | Conditions are evaluated at candle close. A condition true mid-candle and false at close never fires |
| Resampling from 1-minute data | Arbitrary timeframes are built by resampling. Boundary alignment is the resampler's, which may differ from your broker's chart |
| Warm-up near the floor | A stated window starting close to 1 Jan 2024 cannot get full indicator warm-up padding, the padding is clamped to the floor |
| Corporate actions | A corporate-actions service exists and handles splits, dividends, mergers, spin-offs and symbol changes. Whether a given historical series is fully adjusted for every event is a data-quality question, not a modelling one, check surprising results on equities around known corporate action dates |
Cost modelling
Costs are modelled. See Fees & charges. What remains:
| Limitation | Consequence |
|---|---|
| Statutory rates are as of a date | F&O rates are as of 1 Oct 2024. Rates change by notification. cost_model_version records which model produced a stored result |
| Flat fees become a bps approximation | ₹20/order has no price-space representation, so it is folded into commission against allocated capital. Accurate in aggregate, approximate per trade |
| Broker-specific schedules are not modelled | The commission default is a representative rate, not your broker's tariff |
| Financing and funding are not modelled | Crypto perpetual funding rates and equity margin interest are absent |
Market mechanics
| Not modelled | Consequence |
|---|---|
| Circuit limits | An Indian equity locked at a circuit limit cannot be traded. The simulation trades it |
| Trading halts | Same |
| Auction sessions | Pre-open and closing auctions are not distinguished from continuous trading |
| Liquidity at your price | The simulation assumes a fill is available |
| Overnight gap risk beyond the price series | The gap appears in the data; its untradeability across it does not |
Strategy-level
Overfitting
A backtest cannot detect that you chose the parameters after seeing the result. If you ran twenty variations and kept the best Sharpe, the grade reflects the best configuration of the past and may generalise to nothing.
The improvement loop makes this easy to do at industrial scale, 25 optimisation rounds toward one objective will find a configuration that fits the tested window. See Improving a strategy.
The remedy is procedural, not analytical: run the first version, accept the result, and iterate against the reason for weakness ("drawdown too high, so tighten the stop") rather than optimising the metric directly.
Period selection
Choosing the window that produces the best number is the same error as choosing the parameters that do. It is harder to notice because it feels like diligence.
Single-path dependence
One backtest is one realisation. It does not tell you the distribution of outcomes the strategy could have produced under slightly different conditions.
Regime dependence
A strategy tested through a directional run tells you nothing about consolidation. The market alignment label in the assessment partially discloses this. See Strategy grades.
What a backtest genuinely does tell you
Being fair to the tool, because the list above is long:
| It does establish | Why that is worth having |
|---|---|
| The rules are internally coherent and executable | The validator's job, confirmed by the run |
| The rules produce trades on real data | A strategy that never fires is a strategy you would have found out about live |
| Roughly how often it trades, and how long it holds | Directly checkable against your intent |
| The order of magnitude of its drawdown | Not a ceiling, but a scale |
| Whether the costs consume the edge | The gross-versus-net gap is real and is the honest verdict on an over-trading strategy |
| Whether the win/loss profile matches what you designed | The Trades tab, per trade, with reasons |
The costs point is the strongest. A strategy whose gross return is 12% and whose net return is 1.5% has been given a clear and reliable answer: it trades too much. That conclusion does not depend on any of the limitations above.
Multi-symbol runs are not portfolio backtests
Running the same strategy on several symbols runs each independently, each with the full starting capital. It does not allocate across them, does not net exposure, and does not produce a combined equity curve.
Two independent 20% returns are not a 20% portfolio return. See Example strategies
For one strategy trading a selected set of instruments, use a Dynamic strategy, a genuinely different mechanism.
The one-line version
A backtest tells you whether your rules were coherent, how often they fired, and whether costs ate the edge. It does not tell you whether the edge exists, whether it persists, or what you would actually have been filled at.
Next
- Reading results honestly, what to do with all this
- Fees & charges
- Paper trading, the next-best evidence