Week 16 · lab

Lab — Break the System Around the Model

Open the AI System Pipeline Simulator.

The model is only one stage. Use the fault controls to break different parts of the pipeline while watching which downstream components are affected.

Run at least four cases:

  1. degraded input;
  2. corrupt input contract;
  3. model unavailable or excessive latency;
  4. human override unavailable.

Notice that disabling the override does not necessarily change the model score. It changes the system's authority boundary. That is still a serious failure even though model inference may look normal.

Evidence to save

For each case, record:

  • failed component;
  • first observable symptom;
  • downstream effect;
  • fallback or response;
  • evidence you would want logged;
  • one claim the model's accuracy metric cannot support.