Week 18 · lesson
Audit Every Claim Against Evidence
Before presenting the capstone, read every sentence that makes a technical claim and ask: what evidence supports this?
Useful categories are:
- demonstrated — directly observed in a recorded test;
- calculated — derived from stated data or measurements;
- designed — part of the architecture but not yet exercised;
- assumed — required for the design but not verified;
- not tested — explicitly outside the experiment.
This vocabulary prevents a common problem: architecture diagrams and plans quietly turning into claims about real behavior.
Compare:
The system handles model-service failure safely.
with:
In the simulated service-unavailable test, the request was rejected and the defined fallback state appeared. Recovery after repeated service failures was not tested.
The second statement is narrower and therefore stronger.
Evidence should survive questions
A reviewer should be able to ask:
- Which version was tested?
- What input caused this result?
- Where is the output or log?
- Was the same model used in the demonstration?
- What failed?
- What changed after the failure?
- What remains unknown?
If the answer is “we think it would,” label that item as design or assumption rather than demonstrated behavior.
Before the lab
Audit three claims from your capstone draft. Mark each as demonstrated, calculated, designed, assumed, or not tested.