Services and releases

A service is one or more machines behind a URL that never changes. Everything that happens to it afterwards, a deploy, a rollback, a scale, is a change of which machines sit behind that address.

What is running

pilot service ls
pilot service info web
pilot service releases web

service info gives the health check, the environment KEYS, the domain and the replica ids. It never returns an environment VALUE, which is deliberate and is not a gap to be worked around (secrets says why).

A release, and the gate in front of it

A deploy builds a root filesystem, creates a release, starts a replica from it, and lets that release take traffic only once the replica has answered its health check. Until then the previous release is still serving, and if the new replica never answers, the previous release simply stays.

That is the whole reason a deploy takes as long as your application takes to answer for the first time. A health_gate_failed answer carries the service, the replica, the release, the grace it was given and the last thing the probe saw. When the probe saw a refused connection it means one of two things and almost nothing else: the app is listening on the wrong port, or it bound the loopback address instead of all interfaces.

Rolling back

pilot service rollback web

Rolling back puts the previous release's machines back in front of the URL. It changes what is live for everyone, so confirm it with whoever owns the service first, and an agent must ask before calling it. pilot service releases lists what there is to go back to, newest first, with whether each was healthy and which build it came from.

Replicas and scaling

pilot service scale web --replicas 3
pilot service scale web --vcpus 2 --mem 2048

A SIZE change replaces the replicas one at a time at the same release: a replica comes up at the new size, passes the same gate a deploy passes, and only then is an old one retired. Nothing is rebuilt and no request is dropped. A volume-backed service is the exception, because the replacement cannot mount the volume until the old machine has let go, and requests arriving in that window are held rather than refused.

• A replica with no traffic suspends after about half a minute of quiet, and the next request wakes it. That is the default, and it costs nothing while it sleeps.
• min_machines_running in the compose file keeps replicas resident. A queue consumer or a scheduler with nothing connected to it needs one, because nothing is going to wake it.
• soft_limit, also a compose key, is how many concurrent requests a replica takes before another starts, 20 by default.
• A volume-backed service runs exactly ONE replica and asking for more is refused (volumes).

Services with no URL

A service that has nothing to serve, a worker or a database, asks for no address with private: true. Its peers still reach it at <name>.internal, and nothing outside the app can.

Reading a replica

pilot logs web
pilot logs web --tail 50

pilot logs prefixes every line with the replica it came from, so one command covers a service however many machines it has. It is the console, so it needs no shell in the image and no logging agent inside it. Do not exec a replica to tail a file.

Answers worth knowing