Operate and troubleshoot

Diagnose a production request

Trace one failing request through DNS, listener, server selection, location, files or upstream, TLS, and response logs.

8 minute lesson

~~~

A 502, a timeout, a TLS error, and a wrong virtual host are different failures. Each one points at a different layer, so the job is to find the first boundary that does not match expectations — then stop and look there.

Learn the two status codes that get confused constantly. 502 Bad Gateway means Nginx reached for the upstream and got a broken answer: connection refused, connection reset, or garbage instead of HTTP. The process is down or crashing. 504 Gateway Time-out means the upstream accepted the connection but didn’t respond within proxy_read_timeout. The process is up but stuck or slow. Same symptom for the user, opposite investigations for you.

Work the chain in order, from the outside in.

Reproduce with the exact hostname, because server selection depends on it:

curl -sv https://api.example.com/orders -o /dev/null

The verbose output shows DNS resolution, the TLS handshake with the certificate names, and the final status. Half your diagnosis is in this one command: wrong IP means DNS, handshake failure means certificates, wrong content means a server-selection problem.

Then confirm the layers on the server itself:

sudo ss -lntp | grep nginx                    # is the listener up on 443?
sudo nginx -T | grep -n "api.example.com"     # which server block owns this name?
sudo tail -n 20 /var/log/nginx/error.log      # what does Nginx say happened?

nginx -T prints the effective configuration, which beats reading files that might not even be included. The error log names the failing upstream and the reason: connect() failed (111: Connection refused) reads as 502, upstream timed out (110: Connection timed out) reads as 504.

Then step past Nginx and test the upstream directly:

curl -i http://127.0.0.1:3000/orders

If this fails the same way, Nginx is innocent and the problem is the application. If this works while the proxied request fails, the problem lives in the proxy configuration between them — headers, timeouts, or the proxy_pass URI.

The discipline that makes this fast: change one layer only after evidence points there. Restarting the app, editing the config, and flushing DNS all at once tells you nothing about which one mattered.

Practice on a disposable failure. Stop the upstream service, watch the 502 appear, and record the client result, the Nginx error line, the direct upstream test, the repair, and the final health check. Ten minutes of practice buys you calm at 3am.

Lesson completed

Take this course offline

Get every free book and course as PDF and EPUB files.

Get the download library →