Week 16 · lab
Lab — Break the System Around the Model
Open the AI System Pipeline Simulator.
The model is only one stage. Use the fault controls to break different parts of the pipeline while watching which downstream components are affected.
Run at least four cases:
- degraded input;
- corrupt input contract;
- model unavailable or excessive latency;
- human override unavailable.
Notice that disabling the override does not necessarily change the model score. It changes the system's authority boundary. That is still a serious failure even though model inference may look normal.
Evidence to save
For each case, record:
- failed component;
- first observable symptom;
- downstream effect;
- fallback or response;
- evidence you would want logged;
- one claim the model's accuracy metric cannot support.