Curriculum·R411 Building, Validating, and Operating Your Own Automation·about 32 min
Backtesting automation honestly
By the end of this lesson you can
- →Name the four places look-ahead enters through a data pipeline rather than through logic
- →Run a walk-forward validation and state what each fold can conclude
- →Compute the trial count an automated search actually performs
- →Measure the size of your own look-ahead by shifting the data by one bar
Senior · enrolled learners
This lesson opens with The Ariane 5 conversion, read as a validation boundary.
- What happened
- The inertial reference software that destroyed Ariane 5 Flight 501 on 4 June 1996 had flown successfully on Ariane 4 for years. It converted a 64-bit floating point horizontal bias value into a 16-bit signed integer, which can represent values only up to 32,767. Ariane 4's flight profile never produced a horizontal velocity large enough to exceed that. Ariane 5 accelerated faster on a different trajectory, the value overflowed within 37 seconds of ignition, and the resulting exception was not handled. Both redundant inertial units ran the same software and failed within milliseconds of each other, so the redundancy provided no protection at all. The inquiry board found the software had not been revalidated against Ariane 5's trajectory.
- The decision point
- The code had a validation record covering years of successful flights, and every one of those flights was inside an envelope nobody had written down. A backtest is the same document: a record of correct behavior across a range of inputs, which says nothing about behavior outside that range and does not state where the range ends. Per part two the honest version of a backtest reports the envelope rather than the return.
- Recorded loss
- $370,000,000
What you will be able to answer
- →Where does look-ahead enter?
- →What does a walk-forward fold conclude?
- →What is an automated search's trial count?
- →How do you measure your own look-ahead?
Orientation and Year One are open: anyone can read them without an account. From Year Two onward the lessons are for enrolled learners, because progress through the later years only means anything if it is tracked against a record.
It is free. We do not sell the list and there is nothing to buy at the end of it.
Sources and review
- https://www.esa.int/Newsroom/Press_Releases/Ariane_501_-_Presentation_of_Inquiry_Board_report
- http://sunnyday.mit.edu/nasa-class/Ariane5-report.html
- https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2308659
- https://global-q.org/uploads/1/2/2/6/122679606/houxuezhang2020rfs.pdf
Confidence high·Volatility low·Reviewed 2026-08-07·Owner unassigned
Contested
The course autopsy and this lesson use the same incident for two arguments: at course level, that software correct in one envelope is not validated for another and identical redundancy is not redundancy; here, that a validation record describes an envelope nobody wrote down. The split is declared. The $370M figure for the launcher and payload is widely cited and varies by source.
R405-03 owns the seven sins of backtesting in general, including the multiple testing arithmetic reproduced in part four. This lesson owns the ways automation specifically introduces those sins through a data pipeline rather than through judgment. Keep the split and do not re-derive the sins here.
Walk-forward validation is one scheme among several, including combinatorial purged cross-validation, and it has known weaknesses where the folds are not independent. It is presented here because it is implementable by hand, which per R411-L is what the lab requires.