Services and logs

Read the system journal

Filter journal messages by service, boot, time, and severity instead of scrolling through unrelated output.

Systemd services normally send standard output, errors, and lifecycle messages to the journal. journalctl reads those structured records.

Running journalctl without filters can produce thousands of unrelated lines. On a busy server that dump is useless. Start with the service and the time of the failure instead.

If a web request returns 502, do not scroll the whole journal. Ask for nginx messages from the last twenty minutes first.

Filter one service

sudo journalctl -u nginx --since '20 minutes ago' --no-pager

Add -b to limit the query to the current boot:

sudo journalctl -u nginx -b --no-pager

Show the latest 50 lines with useful explanations:

sudo journalctl -u nginx -n 50 -e --no-pager

Follow new messages while reproducing a problem:

sudo journalctl -u nginx -f

Press Ctrl-C when you have captured the failure.

Compare boots and severity

List known boots:

journalctl --list-boots

Read kernel messages from the previous boot:

sudo journalctl -k -b -1 --no-pager

Filter warnings and more severe messages for the current boot:

sudo journalctl -b -p warning --no-pager

Severity is a clue, not a verdict. A service can log an important failure as ordinary text, while an old warning may be unrelated. I use -p warning to scan for noise, then drop back to the full unit log once I spot the right boot.

Read the event in context

Find the first relevant error, then read the lines before and after it. A final connection refused message may be the result of an earlier configuration or permission error.

Keep timestamps, unit name, and exact message when reporting a problem. Redact tokens, email addresses, and customer data before sharing logs.

Try this on a test machine: restart a harmless test service, then use one journal query to show only that service’s messages from the current boot and last five minutes.

Lesson completed