Inspect runtime failures

A container stops when its main process exits. When that happens, the worst thing you can do is restart it right away. The stopped container holds the evidence. Read it first.

Start with the state

Here’s the sequence I follow:

docker ps -a
docker inspect notes --format "{{.State.ExitCode}} {{.State.Error}}"
docker logs --tail 100 notes
docker stats --no-stream notes
docker exec -it notes sh

docker ps -a lists every container, including stopped ones, and the STATUS column already says a lot: Exited (1) 3 seconds ago versus Exited (137) 2 minutes ago versus Up 5 minutes.

The inspect line pulls out two fields. The exit code is what the process returned. 0 is a clean exit, 1 is usually an application error, 137 means it was killed with SIGKILL, often by the out-of-memory killer, 139 is a segfault. .State.Error is different: Docker fills it when the container failed before the process even ran. A typo in CMD gives you executable file not found in $PATH here, with no logs at all, because there was never a process to write them.

docker logs --tail 100 shows the last 100 lines the process wrote to standard output and standard error. This is why your app should log there and not to a file inside the container: Docker collects it, and it survives the process.

docker stats --no-stream prints one snapshot of CPU, memory, and network usage. Useful when a container is alive but slow.

docker exec -it notes sh opens a shell inside the container. It only works while the container runs. If it already exited, you’re back to logs and inspect.

Ask more specific questions

One inspect line answers most “why did it die” questions:

docker inspect notes --format '{{.State.Status}} exit={{.State.ExitCode}} oom={{.State.OOMKilled}} restarts={{.RestartCount}}'

oom=true means the kernel killed it for exceeding its memory limit, and no amount of code reading will find the bug. restarts=14 means a restart policy has been looping it, hiding a crash behind a “running” status. And the FinishedAt timestamp in .State tells you when, which you can line up with the logs.

Network failures

If the process runs but can’t reach the database, test from inside the same network, not from your laptop. Your laptop has different DNS and different routes. docker compose exec api sh, then a connection attempt to database:5432, tells you the truth.

Method

Keep the failing container until you’ve collected the state, the exit code, and the logs. Change one thing. Rerun the smallest command that can prove your theory wrong. Repeat.

Try this: start the API with a wrong command, like docker run --name notes notes-api:dev node serverr.js, read the exit code and the logs, then remove it and start it with the right command.