Freeze the System Model and Test Plan

Testing becomes meaningless when the system changes every time the result is inconvenient.

Before the capstone begins, freeze the configuration you intend to evaluate.

Define the capstone system

Use the fictional Northstar architecture or another teacher-approved fictional system with equivalent boundaries.

Record:

system version/configuration date
components and versions where represented
required identities/roles
service bindings
network relationships
schema/data contracts
logging sources
backup/recovery mechanism
active controls
known exceptions

This is your configuration baseline for the defense.

Define requirements as testable statements

Weak requirement:

Users are secure.

Testable requirements:

R1 viewer can read status
R2 viewer cannot update match notes
R3 operator can update match notes
R4 operator cannot change roles
R5 privileged role grant without approval is denied
R6 maintenance service is not directly reachable from represented remote client
R7 required privileged events reach centralized telemetry
R8 backup restore reproduces the supplied required dataset state

Each requirement should map to one or more tests.

Build a requirements-to-evidence matrix

This prevents the final report from becoming a pile of screenshots with no reason.

Freeze assumptions too

Examples:

• all identities are fictional;
• IP addresses use documentation networks;
• logs are supplied synthetic evidence;
• no public/production systems are tested;
• physical safety is outside scope;
• software vulnerabilities not represented in the scenario are not evaluated;
• test results apply only to the frozen capstone version.

Scope makes the final claims defensible.

Define four test classes

Concept flow

A defensible security test plan uses four test classes

The capstone should prove required behavior, boundary control, controlled failure, and recovery without changing the system every time the result is inconvenient.

1. NORMALrequired function under expected conditions
   then test
2. BOUNDARYdenied action, invalid input, or disallowed path
   then inject
3. FAILUREcontrolled misconfiguration or supplied fault
   then restore
4. RECOVERYcorrect the fault and verify function plus control
   record
5. EVIDENCElink each claim to an evidence ID

Use all four classes deliberately:

• Normal: required function under expected conditions.
• Boundary / negative: action, input, or path just outside the allowed boundary.
• Failure: controlled misconfiguration or supplied failure condition.
• Recovery: restore or correct the failure and verify function plus control.

Every major control should appear in at least one meaningful test class.

Choose one failure to defend deeply

Examples:

• stale privileged session after role change;
• unintended service binding;
• malformed telemetry message;
• broken log forwarding;
• expired/mismatched training certificate;
• failed backup restore test;
• detection schema regression.

Do not try to simulate every possible failure. One deeply evidenced failure/recovery case is more valuable than ten shallow demonstrations.

Pre-register expected outcomes

Before running the tests, write what you expect.

Example:

Test T-05
Condition: viewer attempts update_match_note
Expected: authorization deny
Evidence: authorization event decision=deny reason=role

If the result differs, do not rewrite the expectation afterward. Record the failure and investigate it.

That prevents confirmation bias.

Build the capstone evidence registry

Assign IDs:

E-01 architecture diagram
E-02 role matrix
E-03 socket baseline
E-04 route/path evidence
E-05 authorization log
E-06 telemetry health
E-07 backup restore output
E-08 risk register

Every major claim in the final report should reference one or more evidence IDs.

Capstone readiness check

Before Lesson 2, you should have:

• frozen architecture/configuration;
• at least eight testable requirements;
• assumptions/scope;
• test matrix with expected outcomes;
• one selected failure/recovery scenario;
• evidence registry;
• rollback/recovery conditions.

Do not begin the final test run until those exist.

Lesson 2 executes the matrix and records failures without hiding them.