Test Normal, Boundary, Failure, and Recovery States

The capstone is not graded by how many green boxes you produce.

A failure found and explained honestly is stronger evidence than a test quietly removed because it failed.

Execute the matrix without moving the goalposts

For each test, record:

test ID
requirement
starting state
input/action
expected result
observed result
evidence IDs
pass/fail
interpretation
limitation

Keep the expected result written before the observation.

Example normal test

T-01
Requirement: viewer can read status
Expected: allow + HTTP 200 + expected state field
Observed: allow + HTTP 200 + state=ok
Evidence: E-05, E-09
Result: pass

The test supports the required viewer function for the represented case.

Example boundary test

T-02
Requirement: viewer cannot update match note
Expected: authorization deny
Observed: deny reason=role
Evidence: E-10
Result: pass

This is an important security test because it checks a prohibited relationship.

Example failure test

Selected failure:

log-forwarder queue fills and drops events

Expected behavior:

• forwarding-health alert fires;
• collector gap is visible;
• incident timeline marks dataset incomplete;
• application required function continues if architecture allows it.

Suppose observation is:

queue_full recorded
48 events dropped
collector freshness alert did NOT fire

That is a capstone failure.

Do not hide it.

It reveals a monitoring gap.

Root the correction in the failure mechanism

Diagnosis:

forwarder has local queue alert
collector has no source-freshness rule

Correction in fictional model:

add source freshness/canary check for privileged telemetry stream

Retest:

safe synthetic canary stops arriving
freshness alert fires after defined interval
stream resumes
alert clears after healthy intervals

Now your architecture improved because the test failed.

Recovery test should restore more than one indicator

For any repaired failure, verify:

• original failed condition corrected;
• required function restored/preserved;
• related defensive control healthy;
• evidence pipeline healthy;
• no new regression in adjacent requirement.

This is why the course has emphasized regression testing since Week 4.

Use an evidence table for the complete run

A mixed table is normal.

Distinguish test failure from system compromise

If a negative test unexpectedly allows an action, the evidence proves the control failed in the represented test.

It does not automatically prove a real attacker exploited it.

Keep the claim narrow.

Do not overfit the system to the test

If you add a control only to make one exact synthetic input fail while leaving the underlying mechanism unchanged, the improvement may be fake.

Ask:

Did the correction change the boundary/mechanism, or only the example string?

For example, a role check should enforce the authorization relationship, not block one username.

Complete the final test run

Your dossier needs at least:

• 3 normal tests;
• 3 boundary/negative tests;
• 2 failure tests;
• 2 recovery tests;
• 1 monitoring/control-health test;
• 1 backup/recovery evidence test.

Tests may overlap requirements, but each must answer a clear question.

Update risk and architecture after the run

If a test fails:

• add/update risk register;
• assign owner;
• record treatment;
• update architecture/control map if design changed;
• preserve original failure evidence;
• add regression test.

Do not erase the failure from history after fixing it.

Lesson 2 exit criteria

Before final defense:

• all tests have expected + observed results;
• failures remain visible;
• at least one failure has mechanism + correction + recovery evidence;
• risk register reflects discoveries;
• evidence IDs are referenced consistently;
• limitations are written.

Lesson 3 turns this engineering record into a defensible report and presentation.