Week 06 · lesson

Safe Automation Is Repeatable and Reversible

The best automation is not the script with the most features. It is the one whose behavior remains understandable when run twice, when something fails halfway through, and when a human needs to recover.

Two useful ideas are idempotence and rollback/recovery.

Idempotence means repeated execution converges on the same intended state

Suppose a script ensures a directory exists:

from pathlib import Path

Path("lab06/archive").mkdir(parents=True, exist_ok=True)

Run it once: directory exists.

Run it again: directory still exists.

The second run does not create archive2, duplicate content, or fail merely because the desired state already exists.

That is an idempotent-style behavior.

Not every operation can be perfectly idempotent, but the question is valuable:

What happens if this runs again?

Repeating a non-idempotent change can amplify damage

Imagine:

append firewall rule
append firewall rule
append firewall rule

or:

add 10 to timeout
add 10 to timeout
add 10 to timeout

If the operator reruns the script after an uncertain failure, the state may drift farther from the intended target.

A stronger pattern is often declare the desired state, inspect the current state, then change only what differs.

DESIRED

OBSERVE CURRENT

DIFFERENT?
├─ no → no change
└─ yes → bounded change

        verify

Recovery should be designed before the failure

Rollback does not always mean “undo everything.”

Depending on the system, recovery may mean:

  • restore a known-good configuration;
  • move a file back;
  • restart from a clean temporary directory;
  • stop and require human review;
  • revert only the last committed state; or
  • rebuild the disposable lab environment.

The key is that the recovery path is known before the risky change.

Lab: controlled failure in a temporary model

Use a disposable folder or paper simulation.

Desired task:

Copy three supplied .log files into lab06/archive without overwriting an existing filename.

Baseline:

incoming/
  alpha.log
  beta.log
  gamma.log

archive/
  beta.log

The destination conflict for beta.log is intentional.

Design the automation so it:

  1. checks the root paths;
  2. identifies candidate files;
  3. reports the conflict before overwrite;
  4. copies only allowed non-conflicting files;
  5. returns a failure or partial-failure status according to your documented policy;
  6. leaves the existing beta.log unchanged; and
  7. records a bounded operation log.

You may implement this locally in a temporary classroom directory or complete it as pseudocode plus supplied traces.

Inject one failure

Choose only one:

  • destination directory temporarily absent;
  • simulated permission-denied result supplied by the lesson;
  • one destination conflict;
  • one malformed filename; or
  • one intentionally invalid configuration value.

Do not test on a real shared or production directory.

Record:

expected behavior
observed behavior
exit status
side effects that occurred
side effects that did not occur
recovery action

Retest three properties

1. Required function

Does the task still perform the allowed operation?

2. Preserved boundary

Does it still avoid overwrite, external paths, or other prohibited side effects?

3. Repeatability

What happens on a second run after the desired state already exists?

A strong automation artifact answers all three.

Avoid the “worked once” trap

A successful demonstration is weak evidence when:

  • the starting state was undocumented;
  • no negative case was tested;
  • the run changed more than one variable;
  • the script cannot distinguish partial failure;
  • logs are missing; or
  • rerunning produces a different uncontrolled state.

A defensive tool needs predictable behavior under failure, not only under ideal conditions.

Finish the Automation Safety Record

Include:

  1. task contract and allowed root;
  2. preconditions;
  3. dry-run or planned-change evidence;
  4. outcome/exit semantics;
  5. bounded log fields;
  6. one controlled failure;
  7. observed side effects;
  8. recovery/rollback path;
  9. repeatability retest; and
  10. limitation.

A strong final claim:

In the disposable Week 6 directory model, the automation copied the allowed non-conflicting files, preserved the existing conflicting file, reported the conflict, and converged on the same intended state when rerun. This does not establish behavior on production filesystems or with permissions not represented in the lab.

That is operational evidence, not confidence theater.