Curriculum·R411 Building, Validating, and Operating Your Own Automation·about 33 min
Reviewing automated performance
By the end of this lesson you can
- →Review at component level rather than on the total, and say why the total hides a failure
- →Compute the expected detection time for a silent component failure at a given review cadence
- →Compare live results against the backtest envelope rather than against the backtest return
- →Set the review cadence, the attribution fields and what a review may change
Senior · enrolled learners
This lesson opens with AXA Rosenberg's model error, 2007 to 2011.
- What happened
- The SEC charged three AXA Rosenberg entities with securities fraud for concealing a significant error in the computer code of the quantitative model used to manage client assets. A material error that disabled one of the model's key risk management components was introduced in April 2007 and was not discovered until June 2009, a period of about twenty-six months. Rather than disclosing and correcting it, a senior official directed others to keep quiet and declined to fix it at that time. In February 2011 the firms agreed to pay $217M to cover investor losses and a $25M civil penalty, being $242M in total. Barr Rosenberg, the co-founder, settled separately for a $2.5M penalty and a lifetime bar from the securities industry.
- The decision point
- A component that manages risk does not announce its absence in the return. The portfolios kept trading, the model kept producing positions, and for twenty-six months the only visible symptom was performance that differed from what the model was supposed to produce, which is indistinguishable from a market that behaved differently. A review that looks at the total return cannot find this. A review that reconstructs the contribution of each component can.
- Recorded loss
- $242,000,000
What you will be able to answer
- →Why review at component level?
- →How long can a silent failure run?
- →What do you compare live results against?
- →What may a review change?
Orientation and Year One are open: anyone can read them without an account. From Year Two onward the lessons are for enrolled learners, because progress through the later years only means anything if it is tracked against a record.
It is free. We do not sell the list and there is nothing to buy at the end of it.
Sources and review
- https://www.sec.gov/news/press/2011/2011-37.htm
- https://www.sec.gov/enforcement-litigation/litigation-releases/lr-22298
- https://www.ai-cio.com/news/axa-rosenberg-settles-with-sec-paying-242-million-for-hiding-computer-glitch/
- https://www.lexology.com/library/detail.aspx?g=6e9d580d-5e0a-433b-94f7-a38c57842a93
Confidence high·Volatility low·Reviewed 2026-08-07·Owner unassigned
Contested
The concealment after June 2009 is a separate matter from the twenty-six months during which the error went undetected, and this lesson relies on the second. The detection failure is the reviewable one and the concealment is a governance failure of the kind R405-01's independence requirement addresses.
The detection arithmetic in part two uses a stated per-review detection probability, which nobody can measure in advance. It is on the page so it can be replaced. The conclusion, that expected detection time is the reciprocal of the per-review probability and therefore that cadence and thoroughness multiply, holds at any value.
R405-05 owns the journal schema and S211-05 owns the reporting stack. This lesson owns the review of an automated system specifically, being attribution and envelope comparison. Keep the splits.