All pages
Errors
Every failure answers in the same shape, and the useful field is the one naming what to do next. Branch on the code, never on the wording: the sentence is written for a person and it will be reworded.
{
"error": "a sentence for a person",
"code": "health_gate_failed",
"next": "the one thing to do about it",
"details": { },
"request_id": "the name the host logged this request under"
}
code is a closed list. next names the command or the
call to make now. details is typed per code and absent otherwise.
request_id is the same value as the X-Request-Id header on every
answer, and it is what to quote when you report one. The CLI prints the error and then
next on its own line, and --json prints the server's body
unchanged.
Every code
| Code | Status | What it means |
|---|---|---|
bad_request
| 400 | The request is malformed or missing a field. The answer names which. |
unauthorized
| 401 | No key, or a key that does not resolve. Sign in, or set the key variable. |
scope_required
| 403 | The key resolves but lacks the scope, or is held to a name prefix and reached past it. Mint one that has the scope, or act on what the prefix covers. |
self_only
| 403 | A machine’s own token reached past the machine it was minted for. Act on that machine, or use an API key for anything wider. |
repo_not_connected
| 403 | This organization has no claim on the repository named. Connect it with an admin-scoped key, or send a tar of the directory instead. |
billing_hold
| 402 |
The team has used its free allowance for this month, or its bill reached the budget it set. Its machines are asleep and nothing was deleted. Run pilot billing to add a card or change the budget, and the machines wake as usual once the hold lifts.
|
not_found
| 404 | No such object, or this key belongs to another organization. |
app_required
| 400 |
A service with no app was given an environment that names a .internal address. Those names resolve only inside one app, so set app to the app of the service it connects to.
|
conflict
| 409 | An operation is already running on it. Wait for it to finish. |
rollout_in_progress
| 409 |
Another deploy, rollback or resize of this service is still rolling. Wait for it: pilot service deploys <service> shows it.
|
release_unrestorable
| 409 |
A rollback or redeploy named a version whose image is no longer stored. Deploy the service from source with pilot deploy.
|
checkpoint_unrestorable
| 409 | A fork or restore named a checkpoint whose saved state is no longer fully stored. Take a new checkpoint, or fork the machine itself. |
state_lost
| 409 | A sleeping machine was woken whose saved disk is no longer fully stored, so it cannot come back. Create it again, or redeploy its service. |
volume_in_use
| 409 | A volume is attached or mounted elsewhere. The answer names what holds it. |
payload_too_large
| 400 | A field is too big to live in a replicated row. Put the value in a volume or in object storage and reference it. |
no_capacity
| 507 | The request was fine and the fleet has no room for the machine right now. This is the fleet's capacity, never a limit on your account: no team is held to a ceiling. Retry shortly or ask for a smaller one. |
busy
| 503 |
A host had no room for the create or fork this moment, held it in its line for up to 20 s, and found none. Wait the time in the Retry-After header and send the same request again. The CLI does this by itself.
|
not_configured
| 400, 501, 503 | This host or fleet was built without the piece. The answer names it. |
not_implemented
| 501 | The route is not built yet. |
unavailable
| 503 | Something this needs did not answer: the host that writes this object, or a service's volume. When it is the volume, the deploy did not start and the service stays on its release. Retry in a minute. |
internal
| 500 | The host broke. Retry once, and say so if it repeats. |
plan_unsupported
| 400 | The compose file asks for something pilots does not do. |
compose_invalid
| 400 | The compose file does not parse. The message names the file. |
unknown_framework
| 400 | No compose file, no Dockerfile, no recipe. |
plan_multi_service
| 400 | A push tried to deploy a repository holding more than one app. |
build_failed
| on the log line | The build stopped. The line carrying the failure says where. |
health_gate_failed
| 422 | The replica never answered its health check. |
The ones that carry details worth reading
health_gate_failed carries the service, the replica, the
release, the grace it was given, and the last thing the probe saw: a status and a body
when the replica answered, and an error when it did not. It carries NO address, because
the probe target is the host's own view of the replica and is not reachable from where
you are reading the error. A refused connection there means one of two things and almost
nothing else: the app is listening on the wrong port, or it bound the loopback address
instead of all interfaces.
It also carries console: the replica's last 40 console lines, with
every value of the service's environment and anything shaped like a credential replaced by
[redacted]. That is usually the answer outright, a missing module or a crash on
boot. The replica is parked rather than destroyed, so pilot machine logs on it
still has the whole console. A build that was asked to deploy ends its log on a line
carrying the same code, next and details, replica and
console included.
scope_required for a prefix carries name_prefix,
what the key is held to, and name, the thing it reached for. It is a refusal
and never a not_found, on purpose: a client that reads "not found" as "the
machine is gone" would make a fresh one over a refusal.
unknown_framework carries the directory, what was looked for,
the listing, the manifests, any workspaces, and the rules a Dockerfile must obey. That is
enough to write one without opening the repository again
(deploy).
busy carries retry_after_ms, position and
queue_length. Each host keeps its own line for starts that do not fit yet,
because machines sleep seconds after going quiet and free their memory. The position is
your place in the line of the host that answered, and it is approximate: there is no line
across the fleet, and a retry may start on another host. The SDKs raise it as a typed
error with the wait and the place, and pilot waits and tries again for up to
three minutes, printing the wait and the place while it does.
What to do
| You got | Do |
|---|---|
| Any error |
Read next and do that.
|
health_gate_failed
| Diagnose the replica named in the answer, then fix the app. Do not redeploy first. |
build_failed
| Read the log line carrying the failure, fix it, build again. |
unknown_framework
| Write the Dockerfile from the details, build with it, and deploy that build under a name. |
internal
| Retry once. If it repeats, say so rather than working around it. |
busy
|
Wait details.retry_after_ms and send the same request again.
|
A 503 about a machine whose owner is gone
| Retry. The host that would bring it back is still joining the fleet and claims nothing until its replica has caught up. A host says so about itself on its health route, and it clears itself within seconds. |
Three habits to avoid
- Parsing the sentence. It is for a person and it will be reworded.
- Retrying a failed health gate by deploying again. Nothing changed, so nothing will.
- Treating a
not_foundas proof the object does not exist. A key scoped to another organization gets the same answer, on purpose.