Health endpoint & deploy/rollback health gate

DeployMill keys every "is this deploy good?" decision off one app-owned health endpoint. deploy, rollback, get_app (its health block), and auto-rollback all probe the same path and apply the same rule:

200 means everything's good. Anything else (a non-200 status, a connection refused, or a timeout) means the deploy is bad → roll back.

There is one function and one rule. Put your app's real readiness checks in that handler and return 200 only when they all pass.

The contract

• Every web app exposes one health endpoint. The default path is /healthz (the starter templates already serve a real 200 there).
• The handler should assert whatever "ready" means for your app (DB reachable, migrations ran, a required file exists, an upstream API answers) and return 200 only when all of them pass. Return a non-2xx (e.g. 503) when any check fails.
• An agent debugging a runtime issue can extend this handler to assert whatever it needs; the gate then enforces it on every deploy.

Probe semantics

The probe HEADs the resolved health path on each attached domain.

• Strict mode (a real health path like /healthz): 200 = healthy. Any other status, a connection error, or a timeout = unhealthy. A 500 or 503 from the app counts as a failure (unlike the legacy gateway-only rule).
• Lenient mode (the bare root /, the opt-out): only 502/503/504 or "can't connect" count as unhealthy. Any 2xx–4xx proves the edge routes. This is the pre-DET-120 behavior.
• Fail N times → unhealthy. The probe does not short-circuit on the first healthy response (during a rolling update the old container may still be answering). It requires N consecutive healthy probes to confirm health and declares failure only after N consecutive failures (including timeouts), spaced by intervalMs. This is the "if it fails n times it rolls back" behavior.
• 404 fallback. If a strict health path returns 404 (the endpoint simply isn't there, e.g. an older app that never added /healthz), the probe falls back to a lenient root / probe instead of treating the missing endpoint as a hard failure. Add the /healthz handler to get the strict gate.

Declarative config

Add a health block to .deploymill/project.json:

{
  "health": { "path": "/healthz", "retries": 3, "intervalMs": 3000, "timeoutMs": 5000 }
}

reconcile_project mirrors the resolved block into the app's metadata so deploy (a primitive that never reads project.json) can read it. reconcile's plan.health reports { current, desired, action, orchestratorGate }.

Defaults / back-compat. Omitting the block keeps existing apps working: the probe still defaults to /healthz but falls back to lenient / on a 404, and no orchestrator-level health gate is wired (no platform-layer behavior change for apps that didn't ask for it). New web apps are scaffolded with the block declared.

Orchestrator-level gate (declaring health opts in)

When a web app declares a health block, reconcile_project also wires an orchestrator-level health gate with a health-gated rolling update. That means the orchestrator:

• won't shift traffic to the new container until its port is accepting connections, and
• rolls back the update if the new container never starts listening within the monitor window (and restarts a container whose port later goes dead).

This gate is a TCP port-readiness probe, not an in-container HTTP check: the kubelet just opens a connection to the app's port, so it needs no probe binary (curl/wget/python) inside the image and works for any stack or BYO Dockerfile. The strict HTTP /healthz → 200 judgment lives in DeployMill's post-deploy edge probe, which runs from outside the container and drives auto-rollback. Splitting them this way means declaring a health block can never wedge your rollout on a missing in-image probe tool. The gate that sequences the rollout is dumb-but-robust, and the strict check happens at the edge where DeployMill controls the tooling.

Notes:

• The gate is stack-independent. It wires for every web app with a container port, including images that ship no shell utilities.
• Wiring is best-effort: a backend rejection is surfaced as a warning and never fails the reconcile; the post-deploy edge probe is the reliable strict layer regardless.

Reading health on demand

get_app({ applicationId }) returns the app's config/detail PLUS a live health block (status, edge probe, image drift, hibernation; pass diagnostics:true for pod-level reasons like CrashLoopBackOff / OOMKilled / ImagePullBackOff and storage warnings). By default it runs a live edge probe. Pass probeEdge:false for a cheap config-only read. When an app is actually misbehaving, prefer debug_app({ applicationId }), which bundles this health snapshot together with filtered build + runtime logs, env-key sources, recent health-gate verdicts, and recent audit entries in one read-only call, so you debug without 5+ round-trips. See deploymill://guides/logs.

Auto-rollback keys off this gate

With "rollback": "auto", a deploy whose health endpoint doesn't return 200 within retries attempts is automatically reverted, and it reverts to the most recent earlier deploy that was recorded healthy (not merely the previous image, which may also be broken). See deploymill://guides/rollback for the full auto-rollback flow. deploy records each deploy's health verdict so this "last healthy" target selection works.

Writing a real health handler

Node (Hono):

app.get("/healthz", async (c) => {
  try {
    await pool.query("SELECT 1"); // DB reachable + migrations ran?
    return c.json({ ok: true });
  } catch {
    return c.json({ ok: false }, 503); // not ready → deploy stays on the old image
  }
});

Python (FastAPI):

@app.get("/healthz")
def healthz(response: Response) -> dict:
    try:
        engine.connect().close()
        return {"ok": True}
    except Exception:
        response.status_code = 503
        return {"ok": False}

What NOT to do

• Don't make /healthz heavy. It's hit repeatedly by the post-deploy edge probe (and on each deploy). Keep checks fast (a SELECT 1, a file stat), not a full integration test.
• Don't return 200 unconditionally if you care about readiness. A handler that always returns 200 defeats the gate. The deploy will look healthy even when the DB is down.
• Don't put liveness-only logic here and expect readiness gating. One endpoint, readiness semantics: 200 iff the app can actually serve.
• Don't probe a path that needs auth or a body. The probe is an unauthenticated HEAD. Keep /healthz open and HEAD-able.