Test Perception at Its Boundaries

A perception demo in perfect lighting tells you almost nothing about robustness.

Boundary testing changes one condition at a time and watches where performance changes.

Conditions worth testing

Depending on the system:

• bright/dim lighting;
• backlighting;
• target distance;
• target rotation;
• partial occlusion;
• motion blur;
• background similarity;
• camera vibration;
• multiple similar objects.

The goal is not to make the detector fail for entertainment. It is to discover the operating envelope.

Example test matrix

Now the statement "vision works" can be replaced with something useful:

Detection was reliable in the tested normal-light cases to 2 m, but performance degraded in the dim-light trial.

Separate model failure from pipeline failure

A missing detection can come from:

• camera not producing frames;
• exposure problem;
• image transport delay;
• preprocessing error;
• model inference;
• threshold logic;
• coordinate transform.

Do not blame "AI" until you know which layer failed.

Build a boundary report

Use supplied outputs, a simulator, a safe camera test, or a clearly labeled hypothetical dataset.

Change at least three conditions.

Record:

• expected behavior;
• observed output;
• failure point;
• whether the robot should continue, degrade, request help, or stop.

End with an operating-boundary statement for the perception system.

Build a boundary test matrix

A perception demo under perfect lighting proves very little about operating limits.

Change one condition at a time:

The table is valuable even when you use a supplied dataset rather than a live camera.

Find the first boundary

If Trial B fails, do not immediately conclude "the model is bad." Repeat or narrow the condition:

100% light → pass
70% light  → pass
50% light  → pass
40% light  → intermittent
30% light  → fail

Now you have an approximate operating boundary to investigate.

Design the system around uncertainty

A robust robot can respond to low-confidence or missing perception by:

• reducing speed;
• requesting another observation;
• switching sensor sources;
• stopping;
• handing authority to a person.

Perception quality and robot policy belong in the same safety conversation. The model can be uncertain while the system response remains deliberate.