Skip to content
Odysseus

Health checks and process reaping

The problem

Section titled “The problem”

A health declaration that does nothing is worse than none at all, because two other mechanisms read it and believe it.

A rolling update needs to know whether a new container is ready before it takes traffic from the old one; with no probe, readiness is unknowable. Startup ordering has the same dependency: waiting for something to become healthy is only meaningful if health is observed. Both of those checks used to be satisfied by the mere presence of a health block — and a health block that produces no probe satisfied them exactly as well as one that works. The result was a deployment that passed every gate and was never actually checked.

How the platform solves it

Section titled “How the platform solves it”

Only a probe the platform can actually run is accepted. A declaration that names a URL or a port, and would require the platform to invent a command to test it, is refused when you apply it rather than stored and quietly ignored. The accepted form is in the deployment reference, and the refusal names the field and the alternative.

That refusal is a measured decision, not a preference. When the platform did synthesise probes, it overrode health checks that images already carried with commands those images could not run — the tool the invented probe called was not in the image, so a healthy container reported failure.

One rule rewrites what you stored, and it says so. A probe timeout below the platform floor is raised to the floor rather than left in place, because a timed-out probe is killed, and it is the kill path that leaks the process described below. The change is announced with its own code in the response and the log, so a value you did not send never changes in silence. Every such announcement is listed in rejections and alterations.

Two other values follow the platform rather than being stamped into your description: they are applied when the container is created and your stored document keeps saying nothing. The difference matters — a default written into your record pins you to whatever the platform default was on the day you last touched it, and nobody decided that.

Reaping

Section titled “Reaping”

A probe that times out is killed, and anything it started is orphaned. An orphaned process is adopted by whatever runs as the first process in the container, and most images run the application there — an application that never collects the dead children it has just inherited. They accumulate for the life of the container.

The remedy is a tiny first process whose only job is to collect them. It is not reserved for containers that declare a probe: an image can carry its own health check, and an application that starts subprocesses leaks with no probe at all.

Not this

This is not the load-balancer probe. The routing edge has its own health check with its own path and interval, it decides whether traffic is sent, and the two are never inferred from one another.

This page also does not carry the floor values, the defaults or the accepted commands. They are columns in the generated reference, and a number repeated in prose is a number that can disagree with the one the platform enforces.

Where to see it

Section titled “Where to see it”