Week 17 · lesson

Lesson 3: Correct, Roll Back, and Verify

Finding the strongest cause is not the end of troubleshooting.

It is the point where you finally earn the right to change something.

That change still has to be authorized, narrow, recoverable when practical, and tied to a verification plan.

A repair that fixes one symptom and breaks two other systems is not a good repair. It is a new ticket with better marketing.

Every change creates a new system state

Before the change, you have a baseline.

After the change, the system is different.

That means the change itself becomes part of the evidence.

Use this model:

baseline

approved change

new observed state

compare to expected result

keep / roll back / escalate

The cleaner the change, the easier that comparison becomes.

System process animation

Correction is not completion

A command succeeding does not prove the original service requirement returned.

Technician question: Why is a zero exit status insufficient?

One controlled change is better than five hopeful changes

Suppose a user reports:

The portal stopped working.

You change:

  • DNS;
  • browser cache;
  • firewall;
  • proxy;
  • network adapter settings.

The portal works afterward.

What fixed it?

You do not know.

Worse, one of the other changes may have created a problem you have not discovered yet.

Now compare a controlled approach:

remote IP works
DNS lookup returns wrong address
approved DNS record/configuration identified
one correction made
hostname retested
portal retested

The result is explainable.

That matters.

The correction should match the failed boundary

Use the narrowest action that addresses the evidence.

Examples:

EvidenceNarrow correctionOversized response
stale mapped drivecorrect mappingreinstall Windows
wrong VM network modecorrect virtual networkreboot physical router
one bad plugin updateroll back pluginreimage workstation
one firewall rule blocks appcorrect ruledisable firewall
one user missing group membershiprestore intended groupgrant Everyone Full Control
one RAM module poorly seatedreseat/test modulereplace motherboard

The oversized response may make the symptom disappear.

That is not the same thing as good troubleshooting.

Rollback belongs in the plan before the change

Rollback means returning to a known acceptable state if the new change fails.

The rollback method depends on the layer.

Hardware

Possible rollback:

  • reinstall known-good component;
  • remove the newly added part;
  • return firmware setting to documented baseline.

Drivers and applications

Possible rollback:

  • restore previous approved driver;
  • restore previous plugin/package;
  • revert configuration;
  • uninstall the new version under the approved procedure.

Network and security

Possible rollback:

  • restore previous firewall rule;
  • restore previous VLAN/policy mapping;
  • remove a newly created port-forward rule;
  • restore approved network profile.

Virtualization

Possible rollback:

  • restore prior virtual configuration;
  • use a snapshot only when it is an appropriate recovery point;
  • restore VM data from independent backup when the failure domain requires it.

Data

Possible rollback:

  • restore from a verified backup;
  • recover a previous version;
  • restore from an approved export.

Not every change is reversible.

That is exactly when backup, approval, and escalation matter most.

A snapshot is not a universal rollback answer

A snapshot can be useful before a risky VM change.

But ask:

  • Is the snapshot consistent?
  • Does the application support snapshot rollback safely?
  • Is important data stored elsewhere?
  • Does the snapshot live on the same failed storage?
  • Will restoring it undo newer legitimate work?

Rollback tools have failure boundaries too.

Use the recovery mechanism that matches the problem.

Verification must return to the user's real requirement

Weak verification:

The warning disappeared.

Stronger verification:

The approved graphics application launches, renders the test project, detects the expected GPU, and the display remains stable at the required resolution and refresh rate.

The user's requirement is the destination of the troubleshooting path.

Do not stop at the first intermediate success.

Positive verification is only half of some security fixes

Security changes often need an allowed test and a denied test.

Example requirement:

guest Wi-Fi → internet: allowed
guest Wi-Fi → staff file server: denied

If guest internet works, the network is not necessarily secure.

If the staff server is still reachable, the isolation requirement failed.

The denial is part of success.

Same with permissions:

Finance user → Finance folder: allowed
Student user → Finance folder: denied

Security verification checks the boundary in both directions.

Worked case: display driver regression

Evidence:

display works during firmware/boot
Safe Mode: display works
normal Windows: external display loses output
problem began immediately after approved driver update
cable/monitor: known-good

Strong theory:

The normal Windows graphics-driver path regressed after the update.

Controlled action:

Use the approved driver rollback or known-good package.

Rollback plan:

If the prior driver does not restore required operation, return to the documented baseline or recovery procedure.

Verification:

  • normal Windows startup;
  • expected displays detected;
  • correct resolution/refresh;
  • Device Manager shows no relevant warning;
  • user's required graphics workflow works.

Notice what we did not do.

We did not replace the monitor or reinstall Windows.

The evidence never earned those actions.

Worked case: plugin update breaks one application

Evidence:

OS: stable
other apps: stable
affected app: crashes at launch
recent change: plugin update
application log: plugin load failure
previous approved plugin: available

Controlled correction:

Roll back the implicated plugin.

Verification must include:

  1. application launches;
  2. plugin-dependent function works;
  3. application remains stable during the test workflow.

A successful launch alone would be incomplete if the plugin feature is the reason the application exists.

Worked case: firewall rule blocks an approved service

Evidence:

IP path: healthy
DNS: healthy
server: reachable
service: running
failure began after firewall rule change

Bad proposal:

Turn off the firewall.

Better:

Inspect and correct the specific rule under policy.

Verification:

  • required app/service communication succeeds;
  • firewall remains enabled;
  • unrelated denied traffic remains denied where the supplied policy requires it.

The repair preserves the control.

Worked case: mobile policy disables synchronization

Evidence:

device online
account sign-in succeeds
sync worked before policy update
managed profile now disables required sync
user asks technician to remove MDM

The device is not yours to unmanaged because one policy is wrong.

Correct response:

  • verify intended policy;
  • document the mismatch;
  • use approved MDM/policy path or escalate to the owner;
  • retest required synchronization;
  • verify the device remains managed.

The smallest change respects authority as well as technology.

Worked case: remote support session

Evidence:

authorized ticket: yes
approved remote tool: available
user awareness/consent required: yes
technician task: inspect one application setting
file transfer: not needed

A professional session includes:

verify requester/device
→ authenticate technician
→ establish approved session
→ perform bounded inspection/change
→ verify result with user
→ close session
→ document actions/outcome

Do not enable file transfer because the tool offers it.

Capabilities you do not need are extra exposure.

Failed correction is still useful evidence

Suppose your theory predicts:

Reverting the plugin will restore the application.

You revert it.

The application still crashes.

That result matters.

The theory just became weaker.

Do not hide the failed test and keep changing things until something works.

Record it and update the model.

Troubleshooting is allowed to revise itself.

Build three controlled-change records

For three supplied incidents, document:

baseline:
strongest theory:
authorized correction:
why this change is proportional:
what could be affected:
rollback/recovery path:
expected result:
actual result:
verification of user requirement:
negative/security verification if relevant:
next action if verification fails:
user-facing explanation:
documentation/escalation:

The important line is not change performed.

It is the relationship between the evidence, the change, and the verification.

Before you move on

A complete repair is not:

I changed something and the error went away.

It is:

I had a bounded theory
→ I made the smallest justified change
→ I preserved a recovery path
→ I compared the new state to the expected result
→ I verified the actual user requirement
→ I documented what changed and what remains uncertain

Next, the category labels disappear. You will use that process across hardware, networking, software, security, and operations in the same queue.

Read it. Prove it.

Lesson knowledge checks

Answer from the lesson you just completed. Results stay in this browser and are not submitted.
Knowledge check 1

Why should rollback be planned before a configuration correction?

Knowledge check 2

What should be verified after correcting a suspected root cause?