An Alert Is a Claim About Evidence

A detection system receives evidence and applies logic.

If the evidence matches the rule, it produces an alert.

That alert still requires interpretation.

Separate four layers

RAW EVENT
   ↓
DETECTION CONDITION
   ↓
ALERT
   ↓
ANALYST / SYSTEM DECISION

Example raw events:

login account=alice result=failed
login account=alice result=failed
login account=alice result=failed
login account=alice result=success

Example rule:

Alert when an account has 3 or more failed logins within 5 minutes.

Example alert:

alert=login-failure-threshold account=alice count=3 window=5m

What does the alert prove?

It proves the detection logic matched the available events represented in the dataset.

It does not prove why the failures happened.

Possible explanations include:

• mistyped password;
• old saved credential;
• automated service using stale secret;
• training simulation;
• unauthorized guessing attempt;
• duplicated log records.

Detection narrows attention. Investigation determines mechanism.

Detection rules have requirements

The threshold rule depends on:

• failed-login events being generated;
• account identity field being present/consistent;
• timestamps being usable;
• collection being sufficiently complete;
• duplicate handling; and
• rule engine functioning.

If the telemetry pipeline fails, the rule may become blind.

This is why Week 13 comes first.

Write the detection purpose before the syntax

A useful detection specification:

Purpose: identify unusual bursts of failed authentication for review.
Required telemetry: authentication result, account, timestamp.
Condition: at least 3 failed results for same account in 5 minutes.
Alert fields: account, count, window start/end, source.
Known benign causes: forgotten password, stale service credential.
Blind spots: missing logs, distributed activity across identities, inaccurate time.
Response: review correlated authentication/session/system evidence; do not automatically declare compromise.

That specification is much more valuable than a screenshot of a red alert.

High-signal context can improve a rule

Suppose a privileged-role change event exists:

role_change account=alice old=viewer new=admin approver=none

A detection may alert whenever privileged role is granted without required approval field.

That rule is closer to a concrete policy violation than a generic “three failed logins” rule.

Still, you must test data quality and legitimate exception cases.

Activity: classify alert conclusions

For each statement, mark supported, possible, or unsupported given only the threshold alert:

1. Three failed-login records exist for Alice in the detection dataset. — supported by alert logic if functioning as documented.
2. Alice's password was guessed by an attacker. — possible, not proven.
3. Alice's account was compromised. — unsupported by the alert alone.
4. The detection should be investigated. — reasonable operational decision.
5. No other failed logins happened. — unsupported unless dataset completeness is established.

Detection logic should be reproducible

Use a small deterministic dataset:

14:00 alice fail
14:01 alice fail
14:02 alice fail
14:03 bob fail
14:07 alice fail

With a 5-minute, 3-failure rule:

• Alice should alert on the first three.
• Bob should not.
• The later Alice failure may or may not be inside a particular implementation's rolling window; define the window semantics explicitly.

Ambiguous rule language creates ambiguous outcomes.

Begin your Detection Engineering Record

Choose one fictional detection:

• failed-login burst;
• privileged-role change without approval;
• critical service health missing for 3 intervals;
• unexpected service binding change;
• backup job failure.

Document:

purpose
required telemetry
rule condition
window/state semantics
alert fields
known benign causes
blind spots
what the alert does not prove

Lesson 2 tests threshold tradeoffs.