Review & Self Study — Which Error Matters?

The confusion matrix is the evidence table. Metrics summarize different parts of that table. Thresholds change which cells receive new cases.

If you remember only one idea from this week, remember that model quality depends on the error being measured and the decision that follows the prediction.

Review questions

1. What are the four confusion-matrix outcomes?
2. What question does accuracy answer?
3. What question does precision answer?
4. What question does recall answer?
5. Why can a highly imbalanced dataset make accuracy misleading?
6. How can threshold changes alter precision and recall without retraining the model?

A useful evaluation statement is specific:

At threshold 0.62 on this held-out dataset, recall was higher than at 0.78, but the lower threshold also produced more false positives. We selected 0.62 because missed positives were defined as the higher-cost error for this fictional decision.

That statement exposes the tradeoff instead of hiding it behind one score.

Self study

• Build a 2×2 confusion matrix for ten predictions of your own.
• Find an example where false positives and false negatives have clearly different consequences.
• Return to the lab and find two thresholds with the same accuracy but different error distributions, if the fixed cases allow it.
• Explain when a system should refuse automatic action even though the model produced a numeric score.