Validation system

Who it’s for

Users who hit a validation message and want to know what it caught. And why it matters

Assumes

You have read Strategy generation process

Validation is the single gate that decides whether a strategy is allowed to exist. It is fail-closed: if it cannot run its checks, it errors rather than passing the strategy through.

The strategies it catches are not malformed. They are strategies that would run, produce a plausible-looking backtest, and be silently wrong. That is what makes this the most important component you will never interact with directly.

Why a validator, when the shape is already correct​

By the time a strategy reaches the validator, its structure is already guaranteed, types, enumerations and value ranges were enforced when the object was constructed. The validator adds the checks that need the live quant engine, and it exists because of a specific class of failure:

A strategy whose entry condition is false on every single bar produces a backtest that succeeds with zero trades and reports no error anywhere.

There are at least four distinct ways to arrive at that outcome, and every one of them looks like a working strategy right up to the results screen. Each of the check families below closes one of them.

The seven check families​

1. Engine availability: fail-closed​

If the quant engine is unreachable, validation hard-errors. It never passes a strategy on the grounds that it could not check it.

This is the property that makes everything else meaningful. A validator that degrades to "allow" under failure is a validator that is absent exactly when you need it.

2. Condition compilation​

Every formula string is parsed by the engine's own parser. Not a lookalike. If the engine would reject it, validation rejects it.

Plus a bare-identifier walk, which exists because of an asymmetry in the engine: it rejects an unknown function loudly, but silently resolves an unknown identifier to NaN. A condition referencing an indicator that does not exist would therefore compile, evaluate to NaN on every bar, and produce a strategy that never trades, with no error. The identifier walk backstops that.

3. Benchmark binding​

A condition that reads a reference series (REF_* terms, or a relative-strength RS call) must name its reference symbol. Without it the engine loads no reference series and those terms are NaN on every bar.

Same silent-zero-trade failure, different cause. This check makes it loud.

4. Satisfiability​

A condition whose AND-ed clauses bound one term into an empty interval is false on every bar by construction:

RSI(14) >= 70  AND  RSI(14) <= 30

There is no value of RSI that satisfies both. This is a genuinely easy mistake to make when iterating on a strategy, add one clause too many and the entry becomes unreachable. The backtest would succeed, report zero trades, and say nothing about why.

5. Gates fit the market​

A clock window has to mean something on the instrument it gates. Two cases:

• An exchange session transplanted onto a 24/7 symbol (a 09:15-15:30 IST window on BTC/USDT is not wrong so much as meaningless, and probably not what you intended.
• A window disjoint from the instrument's own session) a 16:00-18:00 IST window on an NSE equity gates the strategy to hours the market is closed. It would never trade.

6. Safety and coherence​

The family with the most direct consequence for your capital:

7. Enumerations and risk vocabulary​

Timeframe and market are supported; stop-loss types, trailing types and anchors are legal in the engine's own vocabulary.

Worth noting how the rejection message is worded here: it states the supported range, not a preset list. An earlier version rendered a list of eight presets that omitted 4h. So a rejection told the reader 4h was unsupported when the check one line above accepted it. A range states both bounds, which a preset list cannot do.

What was removed, and why​

There used to be a behavioural probe: run the strategy against 800 synthetic bars and check whether it ever fires. It was removed, and the reasoning is worth reading because it shows what this validator is for.

• It was redundant with the compile and satisfiability checks, which catch the real never-fires causes deterministically.
• It was frequently false-positive on legitimately selective strategies. A strategy designed to trade four times a year should not fail validation for not firing in 800 synthetic bars.
• The real backtest is the ground truth on whether a strategy trades.

A check that fires on correct strategies trains users to ignore checks.

Blocking versus non-blocking​

Not every finding stops a strategy.

The non-blocking channel is mostly assumed-value reductions, places where the platform supplied or narrowed a value. The rule is absolute: nothing is ever silently substituted. A value the platform chose is a value you are told about.

Findings carry a disposition, not just a verdict​

The important design decision: a finding says what should happen, not merely that something is wrong.

Why this matters: with only two outcomes (ship, or tell the user it failed) every new check shipped a dead end by default. Six consecutive fixes to strategy generation each narrowed one check's predicate and left every check's disposition untouched, so each fix converted a single case into a pass and handed the next case the identical fate.

An unregistered finding falls back to BLOCK. Safe and loud, rather than silently permissive.

Full behaviour: Repair system.

Messages you see are not messages the model sees​

A validation finding carries two texts, and only one of them is for you.

• The model-facing message is addressed to the repair model and is correct for that job. It looks like: Re-emit take_profit with the number the user gave (type='percent' and value='').
• The user-facing message is separately authored English you can act on.

The renderer default-denies: only text explicitly written for humans reaches a person. A finding with no human text gets a generic line rather than its machine text. Someone was once shown the example above, which is why the rule is denial by default rather than a best-effort translation.

Failure modes this system does not catch​

Being honest about the boundary:

Validation guarantees your strategy means something the engine can execute faithfully. It guarantees nothing about whether the meaning is a good one.

Next​

• Repair system, what happens when a finding is actionable
• AI limitations, what cannot be expressed at all
• Reading results honestly