Podman healthchecks are surprisingly more about controlling downstream dependencies than the container itself.
Let’s see what a healthy and unhealthy container looks like from the host’s perspective.
# Start a simple Nginx container with a healthcheck
podman run -d --name nginx-test \
--health-cmd 'wget -q --spider localhost:80' \
--health-interval 10s \
--health-timeout 5s \
--health-retries 3 \
nginx:latest
# Check its status - initially it will be 'starting'
podman healthcheck inspect nginx-test
# After a few seconds, it should become 'healthy'
podman healthcheck inspect nginx-test
# Now, let's break it. We'll block port 80.
podman exec nginx-test iptables -I INPUT -p tcp --dport 80 -j DROP
# Wait for the healthcheck to fail (it takes a few retries)
sleep 40s
# Inspect again - it should now be 'unhealthy'
podman healthcheck inspect nginx-test
# Clean up
podman stop nginx-test
podman rm nginx-test
This health-cmd is the core. Podman executes this command inside the container at regular intervals (health-interval). If the command exits with status code 0, the container is considered healthy. If it exits with a non-zero status code, it’s unhealthy. If it doesn’t exit within the health-timeout, it’s also considered unhealthy. Podman will retry the check health-retries times before marking the container as unhealthy.
The real power comes from how Podman uses this information. When a container is marked unhealthy, Podman’s default behavior is to restart it. This means if your application inside the container crashes and its healthcheck fails, Podman will attempt to bring it back up automatically.
Beyond simple restarts, this health status is crucial for orchestrators like Kubernetes or even simpler service management tools. If a container reports unhealthy, these systems can take action: stop sending traffic to it, reschedule it on a different node, or trigger alerts.
The health-cmd itself can be anything executable within the container’s environment. For a web service, wget or curl to a dedicated health endpoint (e.g., /healthz) is common. For a database, you might try to connect to it and run a simple query.
The health-interval, health-timeout, and health-retries are tuning knobs.
health-interval: How often to run the check. Too frequent can be a burden on the application. Too infrequent means you might not detect failures quickly.health-timeout: How long to wait for the check to complete. This needs to be long enough for a quick check to run but short enough to detect unresponsive services.health-retries: How many consecutive failures before declaring the container unhealthy. This helps avoid flapping (declaring unhealthy due to transient network blips).
The --health-cmd is executed as the container’s user. If your healthcheck command requires root privileges, you might need to adjust the container’s user or the command itself.
A common pitfall is making the healthcheck too complex or too slow. If the healthcheck itself is resource-intensive, it can exacerbate the problems it’s trying to detect. Aim for a minimal, fast check that verifies the core functionality of your service. For instance, checking if a web server is listening on port 80 is good, but a healthcheck that renders a full HTML page is likely too much.
The --health-start-period is another useful flag. It defines a grace period after the container starts during which failures of the healthcheck are ignored. This is invaluable for applications that need time to initialize, like databases or services that perform background tasks on startup. Without it, a container that’s still starting up might be immediately marked unhealthy and restarted repeatedly.
You can also configure Podman to not restart unhealthy containers using --no-health-restart. This is useful when you want to manually intervene or when another system (like Kubernetes) is managing the restart policy.
The --health-status flag on podman inspect is your primary tool for seeing the current state. It will show starting, healthy, unhealthy, or none if no healthcheck is configured.
The actual command that Podman runs to perform the healthcheck is executed within the container’s network namespace. This means if your healthcheck needs to access services running on the host or other containers, it will use the container’s view of networking, which might require specific configuration or using --network host.
The iptables rule we used (iptables -I INPUT -p tcp --dport 80 -j DROP) is a direct way to simulate a service becoming unresponsive. This rule is added to the container’s own iptables chain, effectively blocking incoming traffic on port 80 for that container.
When a container is marked unhealthy, Podman’s default restart-policy (if set to on-failure or always) will trigger a restart. However, the healthcheck failure itself is the trigger, not the cause of the restart. The restart policy dictates if a restart happens based on the container’s exit code, and Podman’s healthcheck mechanism provides a specific exit code (non-zero) for unhealthy states.
The next common issue you’ll face is managing healthchecks for microservices that depend on each other. You’ll need to ensure your healthcheck commands are aware of these dependencies, or that your orchestration layer handles dependency ordering.