Agent Execution, Review & Recovery

The agent says:

Done. Feature implemented. All tests pass.

That is the beginning of your final review, not the end of the task.

First compare the repository with the permission boundary

Start with:

git status
git diff --stat
git diff

Classify every changed file:

REQUESTED
NECESSARY SUPPORTING CHANGE
UNEXPECTED
OUT OF SCOPE
UNKNOWN - INVESTIGATE

A file does not become legitimate merely because the agent touched it.

Read the diff before running repair commands

Suppose the task allowed:

reports.py
tests/test_reports.py

But git status shows:

modified: reports.py
modified: tests/test_reports.py
modified: requirements.txt
modified: storage.py

Do not immediately run tests and hope the green result explains the extra changes.

Inspect requirements.txt and storage.py first.

The permission boundary already produced a finding.

Verify tests independently

Run the command yourself:

python -m pytest

Record what actually happened:

collected tests
passed
failed
skipped
warnings/errors

Then inspect test changes.

Ask:

Were expected values altered?
Were tests removed?
Were markers added to skip tests?
Did a test stop calling the relevant code?
Did the agent add only happy-path cases?

A passing suite is stronger evidence after you know what the suite still claims.

Run an adversarial case the task did not advertise

Suppose the feature searches players by rank.

Required cases already include:

match
case-insensitive match
no match

Try another meaningful input:

empty player list

or inspect whether whitespace handling was specified at all.

The purpose is not to invent endless random tests. It is to challenge assumptions at nearby boundaries.

Compare behavior with the Definition of Done

Return to the task card.

Example:

[ ] only allowed files changed
[ ] required behavior works
[ ] protected tests unchanged
[ ] complete suite passes
[ ] no dependency changes
[ ] implementation explainable

If five boxes pass and one fails, the task is not “mostly done” unless you deliberately revise the definition.

Recovery has several levels

Do not treat every bad agent result as a reason to nuke the whole repository.

Reject a small line/change

If one unnecessary comment or helper was added, remove that piece and retest.

Revert a file to the baseline

If an out-of-scope file changed and none of those edits are needed, restore that file using the approved Git/workspace workflow.

Reject the entire candidate

If the patch is structurally wrong, too large, or too hard to explain, returning to the known baseline can be cheaper and safer than manually untangling it.

Tighten the task and retry

A failed attempt may reveal a missing requirement or permission rule.

Record that learning before another attempt.

Never use destructive Git commands you do not understand

Commands that reset, clean, force, or rewrite history can destroy unrelated work.

This course does not teach “copy this rollback command from the AI.”

Instead, identify:

what state you want to recover
which files/commits contain it
what uncommitted work must be preserved

Then use the recovery method approved for your workspace.

Git is a precision tool. Recovery should be intentional.

An agent can create plausible repository theater

Watch for claims such as:

I updated documentation.
I added comprehensive tests.
I improved error handling.
I refactored for maintainability.

Those sentences sound positive and contain almost no evidence.

Translate them:

Which documentation file and which line?
Which new cases?
Which failure now has defined behavior?
What responsibility moved where?
What measurable complexity was removed?

The review should operate on repository facts.

Example review

Task:

Add find_by_id(records, record_id).
Allowed: records.py, tests/test_records.py.
No dependencies.

Agent changes:

records.py                         expected
tests/test_records.py              expected
requirements.txt                   unexpected

Diff shows:

+ rapidfuzz==...

The implementation uses fuzzy matching for IDs.

Review:

requirement says exact ID lookup
new dependency not permitted
algorithm adds behavior not requested

Decision:

REJECT candidate

Even if the tests pass.

Build a claim-vs-evidence table

For your task, record:

This table feeds directly into the Unit evidence check.

Finish with an explicit decision

Do not end the review with:

Looks good.

Use one of:

ACCEPT
ACCEPT AFTER HUMAN REVISION
REJECT AND RETURN TO BASELINE
REJECT AND RESPECIFY
NEEDS MORE EVIDENCE

Then state why.

That is how a tool-using workflow remains a development workflow instead of becoming automated trust.