Stress Testing AI Models: A Practical Nightmare
- James W.
- 3 days ago
- 1 min read
Regulators require stress testing. Traditional approach: shock the model inputs (rates +200bps, values -20%), see how risk weights respond.
This works for statistical models. It breaks for deep learning models.
Why? Because you're asking the model to predict in conditions it's never seen. If trained on 2013-2023 data (when rates ranged 0.1% to 5%), asking it to predict at 8% rates is extrapolation into the unknown.
The model produces outputs. With confidence. But that confidence is based on pattern matching from a domain it never experienced.
Is your stress test right? Wrong? Unknown.
ACRGA-STRESS provides systematic methodology: scenario retraining, data distribution stress, sensitivity analysis, governance review.
Not perfect. But defensible.