An installer can print “done” while leaving the machine one reboot away from failure. The interesting engineering problem is the transition between installing files and running the intended system.

This article began as a CentOS 6 and OpenVZ installation shortcut. That old recipe is retired. We’ll use its underlying problem to learn how to build an installer that can resume, detect failure, and prove its result.

Think in states, not lines of shell

Imagine four states: packages installed, machine restarted, service started, and service verified. Each transition needs evidence. A package manager returning successfully proves something different from an HTTP request reaching the service.

A postcondition is the condition that should be true after a step. Checking it before running the next step keeps a partial installation from looking complete.

Trace the diagram from installation to verification. A reboot sits in the middle because it can change which kernel, configuration, and startup mechanism actually run.

Installation is only step oneEach installation stage needs its own postcondition. Verify the running system and a representative application operation after the restart boundary. Installation is only step one FOLLOW THE ARROWSInstallDeclare required stateRebootResume the next stageInspectCheck running stateVerifyExercise the service
Each installation stage needs its own postcondition. Verify the running system and a representative application operation after the restart boundary.
View full-size diagram (opens in a new tab)

Work through a missing-file failure

Here’s a toy installer. Create an empty directory and save this as install_demo.py inside it:

from pathlib import Path

root = Path("demo-state")
root.mkdir(exist_ok=True)
config = root / "service.conf"
expected = "mode=demo\n"

if not config.exists() or config.read_text() != expected:
    temporary = root / "service.conf.tmp"
    temporary.write_text(expected)
    temporary.replace(config)
    print("repaired configuration")
else:
    print("configuration already correct")

assert config.read_text() == expected
print("verified configuration")

Run python3 install_demo.py twice. The first run creates the configuration. The second reports that it is already correct. Both verify the actual file.

That is idempotence in this narrow example: repeating the operation leads to the same intended state. It doesn’t mean every installer action is harmless to repeat. Creating a user, migrating a database, and rotating a key each need their own retry design.

Pause and predict

Suppose the installer wrote a done.txt marker, but service.conf was later deleted. Is the marker enough to skip installation?

Need a hint?

Which file is evidence about the current postcondition?

Show the reasoning
No. The marker records that a step once ran. Inspecting service.conf establishes whether the required state exists now. A marker may help track a migration, but it cannot replace every check on the resulting system.

Add the reboot boundary on paper

A real reboot ends the installer process. Variables in that process disappear. A resumable installer therefore needs a durable description of the remaining work and an explicit mechanism to run it after startup.

Write this small plan before choosing a startup tool:

Stage Evidence before advancing
Install Required package and configuration exist
Restart Running system reports the expected version
Start service Service starts using the intended configuration
Verify A bounded request returns the expected result

A service manager or configuration manager can implement those stages. Avoid replacing a shared startup file to sneak in the next command: it creates another piece of state to restore if the process fails.

Try it yourself

Delete only demo-state/service.conf, then rerun the example. It should recreate the file. Next, change its contents to mode=wrong and rerun. It should repair the value. Remove the disposable directory when finished.

Apply the idea

A service reports active after reboot, but its only useful request fails. Which stage is complete, and which is still incomplete?

Need a hint?

Separate the process lifecycle from the application result.

Show the reasoning
Startup may be complete. Application verification is not. Keep the installation incomplete until the representative request succeeds, and preserve its failure output for diagnosis. Repeating the startup command alone may never repair the underlying problem.

You can move on when you can name a postcondition for each stage and explain what happens if execution stops between two stages. For the next exercise, apply that reasoning to a repair that must survive replacement (lesson in preparation).