Curriculum·Senior·R405
Strategy Development and Validation
The quant quake, week of 6 August 2007. A number of high-profile quantitative long/short equity market-neutral funds recorded unprecedented losses.
Lesson 1 opens on
JPMorgan's Chief Investment Office, 2012
JPMorgan's Chief Investment Office held a large synthetic credit portfolio managed in London. In January 2012 a new Value at Risk model was introduced for the office, replacing the previous one and roughly halving the reported risk figure, which allowed the position to grow without breaching limits. The model was implemented as a series of Excel spreadsheets completed manually, with data copied and pasted between them.
The lessons
- R405-01The pipelineName the six stages a strategy passes through and what each one is allowed to concludeabout 33 min · opens on JPMorgan's Chief Investment Office, 2012
- R405-02Writing rules that are actually testableState the reproduction test, and apply it to a strategy descriptionabout 33 min · opens on Markopolos on Madoff, 1999 to 2008
- R405-03The seven sins of backtestingName the seven sins and the specific artifact each one produces in a resultabout 33 min · opens on Backtest overfitting, documented 2014
- R405-04Statistical honestyCompute the number of trades needed to distinguish your expectancy from zeroabout 43 min · opens on Replicating Anomalies, published 2020
- R405-05The journal as a data pipelineSpecify the journal as a schema rather than as a diary, and name the required fieldsabout 34 min · opens on Growth in a Time of Debt, and its replication, 2010 to 2013
Labs
R405-L
Two parts. First, one of your own strategies: specify it to R405-02's reproduction test, write the acceptance rule and the rejection rule before you run anything, report every configuration tried, include costs, run the test, and defend the verdict the evidence supports, accept, reject or inconclusive. A rejection is the usual outcome and is not the requirement. Second, a supplied strategy with a reported backtest that is known to be overfit: name the errors that account for its result and show, with R405-03's and R405-04's arithmetic, why the reported figure is not evidence. That half is where rejecting is the skill.