All pages
Checkpoints, forks and templates
A checkpoint captures a machine as it is at that instant, memory included. A restore puts the moment back: the same processes, the same open files, the same half finished thing you were in the middle of.
Take one before the risky step
pilot machine checkpoint scratch
pilot machine checkpoints scratch
The capture pauses the guest briefly, writes the memory and the disk as content-addressed blocks, and returns an id. Blocks that already exist are not written again, so a second checkpoint of the same machine stores the difference rather than the whole thing.
Put it back
pilot machine restore ck-3a3077f4d51c9b0e7a624f18
A restore is IN PLACE. The machine keeps its id, its name and its URL, and what was running at the moment of the checkpoint is running again when it finishes. That is also why it is destructive: everything the machine did after the checkpoint is gone. Confirm it with whoever owns the machine first, and an agent driving the API must ask before calling it.
Branch instead of overwriting
A fork makes a NEW machine from a machine or from one of its checkpoints, with its own id and its own URL. Use it to try two answers from one starting point, or to give every run of a test suite the same warm state.
pilot machine fork scratch
pilot machine fork ck-3a3077f4d51c9b0e7a624f18
pilot machine fork golden --idle-timeout 1h --label project=p_1
pilot machine fork golden --disk 8GiB
pilot machine fork golden --no-wait # answers at once with the name; poll pilot machine info <name>
A fork keeps its source's lifecycle policy, so a machine set to wait an hour
before suspending forks into machines that wait an hour too, and it keeps the source's
labels. The one thing it does not inherit is cron schedules, because ten forks of a machine
with a cron would otherwise run it eleven times. --idle-timeout changes the
policy for every fork, and --label replaces the source's labels. Over the API
the same are knobs, a partial policy merged onto the inherited one, and
labels, where {} forks with none.
A fork's root disk is its source's size unless --disk
(disk_gib over the API) asks for more. It may be larger than its source's, never
smaller, and the forked filesystem already fills it when the fork returns. A template made
from a machine with a larger disk passes that size on to the machines made from it
(a larger root disk).
A fork shares its source's memory, but not what has to be unique to one
machine. The kernel reseeds its random number generator when any snapshot is restored, so
two forks read different bytes from /dev/urandom, and every new machine gets its
own /etc/machine-id, its own name as its hostname and, if the image has them,
new SSH host keys. What no restore can reset is randomness a process already drew before
the snapshot: a key generated in memory, or a generator seeded once at startup. Generate
those after the fork, or restart the process that holds them.
Keep a base that outlives its machine
A checkpoint belongs to its machine and goes when the machine is destroyed. A template is a machine's or checkpoint's moment kept on its own: make it once, destroy the source, and make machines from it for as long as it exists. Each is the same restore a fork is, with the source's processes already running.
pilot template create base --from golden
pilot machine destroy golden
pilot machine create p-42 --template base --env PROJECT=42
pilot template fork base --count 5
Nothing runs for a template. Its memory and disk are the content-addressed blocks the checkpoint already wrote, kept in object storage rather than copied. Each host keeps the templates it uses most on local disk ahead of time, so the first machine made from one there is as fast as the hundredth, and any host can make one from object storage when it holds none of it. A machine made from a template costs only what it changes.
A machine made from a template starts with the template's size, its
lifecycle policy (the source's, without cron schedules) and its URL gate. A create or fork
can change the policy, the labels and the gate. --env is merged over the
source's environment and written where exec and the next start of the
application read it. Processes already running in the image keep the environment they
started with. Deleting a template leaves the machines made from it running. A machine with
a volume cannot be a template.
POST /v1/templates {name, from_machine | from_checkpoint, labels?}
GET /v1/templates ?label=k=v
GET /v1/templates/{id|name}
DELETE /v1/templates/{id|name}
POST /v1/templates/{id|name}/fork {name?, count?, knobs?, labels?, env?, secret_env?, url_auth?, disk_gib?}
POST /v1/machines {name, template, knobs?, labels?, env?, url_auth?, disk_gib?}
What a capture stores
A machine's disk is a template plus one per-machine difference, and the depth of that chain is exactly 2. A disk chain any deeper is refused when the header is parsed rather than failing later at fault time.
Memory is layered instead. Each checkpoint or suspend stores only the pages that changed since the previous capture and points every other page at the earlier build that already holds it, one step and never a chain, so a 2 GiB machine that changed little stores 13-24 MB. A memory build may reference at most 16 earlier builds, and the smallest are folded in well before that.
What a restore promises
A restore never fails because a memory image is gone. Memory snapshots carry raw processor identification, so an image is portable within a vendor pool and never across one. When no host of that pool is live, the machine cold-boots from its own disk instead.
| Survives a cold boot | Does not |
|---|---|
| The id, the name and the URL | The processes that were running |
| The volume and every byte on disk | The memory they held |
| The machine’s own agent token |