Most apps an agent deploys are not busy. They're demos, side projects, the thing you spun up to show someone, the preview from three days ago you forgot about. They get a handful of requests a week and otherwise sit there holding memory and CPU hostage. Multiply that across every app on the platform and you're paying to run a graveyard.
So apps that go idle should sleep. They release their resources and wake up when someone actually shows up. Easy to say. The hard part isn't the sleeping. It's knowing when an app is actually idle.
"Idle" is harder to define than it sounds
The naive version: track the last time the app got a request, and if it's been quiet for a while, put it to sleep. Fine. But where does "the app got a request" come from? The app itself can't be the source of truth. It might be the thing that's asleep, and a sleeping app reports nothing. So the signal has to come from the layer in front of the app: the ingress that every request passes through whether the app is awake or not.
That turned into a small pipeline. The ingress streams a lightweight access signal (just enough to say "host X was hit at time T," not request bodies or anything sensitive) into an in-memory collector. A periodic sweep reads that collector and advances each app's last_activity_at. Only then does the hibernation logic get to ask "has this been quiet long enough to sleep?" against a number it can actually trust.
The rule that keeps a busy app awake
There's one invariant in that sweep that I want to call out because it's the kind of thing that's obvious in hindsight and ruinous if you miss it: activity time only ever moves forward. The sweep advances last_activity_at to the greater of what's stored and what it just observed, never just overwriting it.
Why does that matter? Because the signals arrive out of order, from different windows, with different lag. If a later sweep could move the timestamp backwards, because its particular batch happened to see older data, you'd get an app that's serving live traffic and getting put to sleep anyway, because some stale observation regressed its activity clock. A busy app must never be auto-slept. Monotonic-forward is what guarantees it.
Waking up should feel like nothing happened
A sleeping app costs nothing and that's the point, but only if waking it is cheap and invisible. A request comes in for a sleeping app, the platform brings it back, and the request completes. From the visitor's side it's a slightly slower first load and then it's just an app. The dashboard shows you which apps are currently asleep so the state is never a mystery, and there's a wake link when you want to nudge one yourself.
Why this is a neutrality story too, quietly
The reason this could be built cleanly is that "where do requests get observed" already lived behind a provider-neutral seam, the domain/ingress interface. Adding "report recent activity for these hosts" was one more method on an interface that already existed, not a special case bolted onto a specific vendor's load balancer. The same discipline that lets the compute or database backend be swapped is what let a feature like hibernation slot in without reaching around the abstraction. That's the quiet dividend of taking the seams seriously: the features you didn't plan for are cheaper when they arrive.
- Ryan