Verification After Fast Generation

Fast generation creates an uncomfortable illusion: the project can look almost finished before you have established that the new code belongs there.

The response is not “review every character with equal intensity.”

Use a layered verification pass that follows risk and evidence.

Pass 1: Establish the change boundary

Start with repository state.

git status
git diff --stat
git diff

Your first questions are architectural:

What changed?
Why did each changed file need to change?
Did dependencies change?
Did public interfaces move?
Did tests change?
Did documentation or configuration change?

If the task expected two files and the diff contains ten, do not begin by debugging a function deep inside file nine.

First explain the blast radius.

Pass 2: Read high-consequence code first

Not every line has equal risk.

Prioritize changes involving:

• file deletion or overwriting;
• authentication/authorization;
• network requests;
• shell/subprocess execution;
• parsing untrusted input;
• persistence/database writes;
• dependency installation;
• configuration/secrets;
• tests that define expected behavior; and
• public interfaces used by other modules.

For a classroom console app, the stakes may be low. The reasoning habit is still useful.

Generated code that touches a dangerous boundary deserves more scrutiny than a renamed local variable.

Pass 3: Explain the control and data flow

Pick the changed behavior and trace one case.

For a save feature:

caller
  -> save_fleet(robots, path)
  -> serialize defined fields
  -> open destination
  -> write JSON
  -> close file
  -> caller continues

Then trace a failure case:

invalid/unwritable path
  -> file operation fails
  -> what exception/fallback occurs?
  -> what does caller observe?

If you cannot explain where data enters, changes form, crosses a boundary, and exits, the implementation is not ready to accept.

Pass 4: Inspect tests as code changes

Do not let test files hide in the “good news” category.

For every changed test, ask:

What requirement does this encode?
Did an expected value change?
Was a failing test deleted?
Did a broad assertion become weaker?
Does the test call the real behavior?
Is the setup realistic enough for the claim?

A candidate implementation can make its own suite green by weakening the suite.

Example:

Before:

assert result == {"name": "Nova", "score": 10}

After:

assert result is not None

That change may be syntactically valid and dramatically less useful.

Pass 5: Run the right tests yourself

Use the project's actual environment.

python -m pytest

When the suite is large, start with targeted tests and then expand.

Record:

command
number collected
failures
relevant output

Do not rely on a generated summary saying “all tests pass.”

The tool may have run a different command, stale subset, or no test at all.

Pass 6: Exercise runtime behavior

Automated tests are not the only evidence.

Run the feature in the application when appropriate.

Use at least:

one normal path
one boundary or failure path

For a JSON load feature:

normal file
missing file

For a search:

match exists
no match

For an API:

valid supplied response/fixture
failure response or malformed data

The runtime check can expose wiring problems the unit tests never reached.

Pass 7: Return to the specification

This is where many reviews stop too early.

A change can be readable, tested, and still solve the wrong problem.

Use a requirement matrix:

This keeps architectural and non-functional constraints visible alongside behavior.

Pass 8: Decide what to remove

Generated output often contains extra material:

• unused helper functions;
• comments narrating obvious syntax;
• duplicated validation;
• unnecessary abstractions;
• dead imports;
• extra configuration;
• speculative future features.

Deletion is part of review.

Do not keep code because the tool spent effort generating it.

The project has no obligation to preserve unnecessary output.

Verification can reveal that the specification was incomplete

Suppose your tests reveal ambiguity:

What should happen when the JSON file exists but contains an empty list?

If the specification never decided, do not invent a hidden behavior during review.

Update the requirement/acceptance criterion deliberately, then implement/test it.

Verification is allowed to send you back to the specification.

That loop is healthier than pretending every ambiguity belongs to the generator.

Review debt has a stop point

Imagine a candidate produces 900 lines across 14 files.

You understand only the first three modules, the test suite is unfamiliar, and several dependencies appeared.

The correct next step may be:

REJECT OR SPLIT THE CHANGE

not “review faster.”

A smaller implementation that you can explain and verify is often the better engineering result.

Create a verification receipt

For one generated or supplied slice, save:

TASK / REQUIREMENT:
BASELINE COMMIT OR STATE:
FILES CHANGED:
DEPENDENCIES CHANGED:
TEST COMMAND + RESULT:
MANUAL CASE:
BOUNDARY / FAILURE CASE:
SPECIFICATION RESULT:
CODE REMOVED OR REVISED:
FINAL DECISION:

That receipt becomes the evidence for the Unit 16 wrap-up.

The rule to keep

Fast generation does not justify shallow review.

It requires better boundaries, smaller changes, and stronger evidence because the cost of producing more code has fallen while the cost of understanding incorrect code has not.