Lab — Train, Break, and Defend a Classifier

The chapter so far used a threshold because it is simple enough to inspect. Now use the same idea as an experiment rather than a diagram.

Open the Classification Playground.

Start with the default dataset and answer three questions before changing anything:

1. What threshold is currently being used?
2. Which points are classified incorrectly?
3. What evidence would justify moving the boundary?

Then move the threshold slowly. Watch accuracy, precision, recall, and total errors. Find two different thresholds that produce different error patterns.

Next, use Corrupt two labels. Do not immediately repair the model. First inspect what changed in the evidence. A learning algorithm cannot know that a human label is wrong just because the resulting model is inconvenient.

A defensible lab note might say:

With the original labels, threshold 31 cm produced one error on this dataset. After two labels were flipped, the same threshold produced a different error pattern. The experiment shows that fitted behavior depends on the labels supplied; it does not show which labels are correct in a real robot environment.

Evidence to save

Record:

• one threshold you tested;
• the metric values you observed;
• what changed after label corruption;
• one claim supported by the simulation;
• one claim the simulation cannot support.

The playground is deterministic and fictional. It teaches model behavior; it is not a safety controller.