Curriculum·Graduate·G905
Backtesting and Validation
Victor Niederhoffer's fund, 1997. Niederhoffer was a celebrated trader and an early user of computers to predict short-term moves, whose strategy included selling out-of-the-money options, collecting steady premiums that had worked for years and looked excellent in the record.
Lesson 1 opens on
Victor Niederhoffer's fund, 1997
Victor Niederhoffer was a celebrated trader, a pioneer of using computers to predict short-term moves in stocks and commodities, whose yearly returns had reached 30 percent and whose strategy included selling out-of-the-money options, which collects a steady premium in almost every period. Selling those options is a bet that a large move will not happen, so the strategy earns a little again and again while quietly carrying the risk of a rare, enormous loss.2 percent on 27 October; his broker called for margin he could not produce and the fund was wiped out, and a decade later, in 2007, his Matador fund closed down more than 75 percent after another sell-off.
The lessons
- G905-01The backtest that hid the tailExplain that a smooth backtest can hide a strategy that is short the tailabout 34 min · opens on Victor Niederhoffer's fund, 1997
- G905-02Overfitting: a pattern that was only noiseExplain that a rule tuned to fit past data captures noise, not signal, and fails out-of-sampleabout 34 min · opens on The Foolish Four, late 1990s
- G905-03The regime your data never containedExplain that a model built on a stable regime is blind to the regime changingabout 34 min · opens on Everest Capital and the Swiss franc, 15 January 2015
- G905-04Chasing a signal that means nothingExplain that a signal with no real information is noise, and acting on it is acting on nothingabout 34 min · opens on Long Island Iced Tea to Long Blockchain, December 2017