Week 14: Detection Is a Testable Hypothesis

An alert is not proof of an incident. It is a rule or model saying, “this observation matches a condition we decided to notice.”

This week turns detection into engineering. Students will define signal, condition, expected false positives/negatives, supporting evidence, tuning, and health checks for a small set of fictional detections.

This week

1. An Alert Is a Claim About Evidence — separate raw event, detection condition, alert, and analyst conclusion.
2. Thresholds Trade Sensitivity for Noise — tune a deterministic rule and measure false positives/negatives in a labeled synthetic dataset.
3. Detections Need Tests and Maintenance — write positive/negative/regression cases, monitor pipeline health, and document blind spots.

Evidence artifact

Create a Detection Engineering Record with rule purpose, required telemetry, logic, labeled test set, confusion counts, tuning decision, regression cases, blind spots, and monitoring plan.