# The Linux commands I actually needed to run a server this year > The Linux commands that mattered when my Ubuntu servers broke in 2026: df, du, journalctl, free, ss, ufw, fail2ban, apt, with real output and what I did next. Author: [Flavio Copes](https://flaviocopes.com/about/) | Published: 2026-09-24 | Topics: [CLI](https://flaviocopes.com/tags/cli/) | Canonical: https://flaviocopes.com/linux-server-commands/ I run two small Ubuntu servers on DigitalOcean. One runs [Sendy](https://sendy.co/), the app I use to send my newsletter. The other runs a self-hosted [Plausible Analytics](https://flaviocopes.com/how-to-self-host-plausible-analytics/) instance for this site. I connect to both from my Mac over SSH. In 2026 they gave me a run of incidents: a process that kept dying, a disk at 99%, a newsletter that took eight hours to send, a security pass on a rebuilt machine, and a Plausible upgrade I had postponed for years. I wrote a post about each one. This post collects the commands that mattered across all of them, with the exact invocation, the real output where a post quotes it, and what I decided because of it. Most of these were typed by a coding agent (Codex on my Mac, working over SSH) while I watched. During the full disk morning I typed them myself and pasted the output back. Either way I had to read the output and make the call. I log in as root on these boxes, so the original sessions often skip `sudo`. I added it here where a normal user needs it. ## 1. `ssh`: get in Everything below runs on the server. This is the one command that runs on the Mac: ```bash ssh root@ -i ~/.ssh/digitalocean ``` `-i` picks the private key. DigitalOcean installed the matching public key when the droplet was created, so there is no password step. That exact line is also what I paste to the agent when I want it to work on a server. If you type it often, put the host in `~/.ssh/config` and use an alias, which I covered in [SSH for developers](https://flaviocopes.com/ssh-for-developers/). And for long work by hand, start it inside [tmux](https://flaviocopes.com/tmux/): during the Plausible upgrade the SSH session reset in the middle of a 2.2 GB backup and the agent had to reconnect. ## 2. `df -h`: how full is the disk `df` reports free space per filesystem. `-h` prints sizes as G and M instead of blocks. ```bash df -h ``` One morning the Sendy server stopped responding and I couldn't even SSH in. After a reboot from the DigitalOcean panel, this is the line `df -h` printed for the root filesystem: ```text /dev/vda1 48G 47G 951M 99% / ``` 951 MB left on a 48 GB disk, which is why a reader had emailed me that morning to say the unsubscribe link didn't work. `df` tells you *that* the disk is full and nothing about what fills it. ## 3. `du -xhd1 | sort -h`: what is eating it `du` measures how much space a directory tree uses. These flags make it useful on a server: ```bash sudo du -xhd1 / | sort -h ``` `-x` stays on one filesystem, so it skips `/proc` and mounted volumes. `-h` gives human sizes. `-d1` goes one level deep. Piping into `sort -h` sorts those human sizes correctly, so the biggest directory ends up last. On the full disk it printed: ```text 43G /var 47G / ``` The same command on `/var` gave `42G /var/lib`, and on `/var/lib`: ```text 42G /var/lib/mysql 42G /var/lib ``` Three runs, each one level deeper, and I had the directory. ## 4. `ls -lhS`: which files, exactly Once `du` points at a directory, `ls` sorted by size tells you what's inside: ```bash sudo ls -lhS /var/lib/mysql | head -40 ``` `-S` sorts by size, biggest first. This is what came back, over and over: ```text -rw-r----- 1 mysql mysql 111M May 5 09:47 binlog.000300 -rw-r----- 1 mysql mysql 110M May 5 09:47 binlog.000301 -rw-r----- 1 mysql mysql 105M May 5 17:11 binlog.000408 ``` Over 400 files, about 100 MB each. MySQL binary logs, a journal of every change made to the database. The Sendy database itself was 528 MB. The logs were 41 GB. The decision this led to was *not* `rm`. MySQL keeps an index of its binary logs and can refuse to start if they vanish, so the fix went through MySQL: ```bash sudo mysql -e "PURGE BINARY LOGS BEFORE DATE_SUB(NOW(), INTERVAL 1 DAY);" ``` Then `df -h` again: ```text /dev/vda1 48G 20G 29G 41% / ``` From 951 MB free to 29 GB. The permanent fix was `binlog_expire_logs_seconds=86400` under `[mysqld]` in `/etc/mysql/mysql.conf.d/mysqld.cnf`. Sendy writes a row for every send, open and click, and MySQL 8 turns binary logging on by default, so the logs had grown at the speed of my newsletter. ## 5. `journalctl --vacuum-time` and the other cheap cleanups Before digging, free the easy space. The systemd journal first: ```bash journalctl --disk-usage sudo journalctl --vacuum-time=3d ``` The first prints the journal's size, the second deletes archived journal files older than three days. `sudo apt clean` and `sudo apt autoremove --purge` do the same for the package cache. Neither was my problem that day, but they were the first two items in the triage plan Codex gave me. If a log file *is* the problem, empty it instead of deleting it: ```bash sudo truncate -s 0 /var/log/apache2/access.log ``` If you `rm` a log that a running process has open, the process keeps writing to the deleted file and the space is not freed until you restart it. On a Docker host, `docker system df` shows what images, containers and volumes take. After the Plausible upgrade, `docker image prune -a` recovered about 2.57 GB of old images without touching the volumes. ## 6. `journalctl -k`: why did my process die In April 2026 I installed Claude Code on the Sendy server and it got killed a few seconds after starting, every time, with no error. I pointed Codex at the server from my Mac, and this is the command it ran: ```bash journalctl -k --since "2 hours ago" --no-pager | grep -Ei "killed process|out of memory|oom|claude|node" | tail -n 80 ``` `-k` shows only kernel messages, `--since` limits the window, `--no-pager` prints everything instead of opening `less`. The `grep` filters for the words Linux uses when it kills a process for using too much memory. It found this: ```text Out of memory: Killed process ... (claude) anon-rss: ~864MB ``` That's the **OOM killer**, the part of the kernel that picks a process and kills it when RAM runs out and there is no swap. The droplet had 2 GB of RAM, MySQL was using about 786 MB, and `claude` grew to about 860 MB. The whole session is in [Debugging a process killed on my server with Codex](https://flaviocopes.com/server-process-killed/). One gotcha: `-k` implies `-b`, the current boot only. If the machine rebooted since the kill, add `-b -1` to read the previous boot. `sudo dmesg -T` reads the same kernel buffer. ## 7. `free -h` and `ps --sort=-rss`: where the memory went `free` shows total, used, free and available memory, plus swap. `-h` scales the numbers: ```bash free -h ``` On the Sendy droplet, during the OOM session, it reported 1.9 GiB total and `0B` swap. With zero swap a tight machine has nothing to fall back on, so the kernel starts killing. The column to read is `available`, not `free`, because Linux uses spare RAM for disk cache and gives it back when a process asks. `free` says memory is tight. `ps` says who has it: ```bash ps -eo pid,ppid,comm,rss,%mem,%cpu,args --sort=-rss | head -n 15 ``` `-e` lists every process, `-o` picks the columns, `--sort=-rss` orders by resident memory, biggest first. On the Sendy server the list showed MySQL at roughly 786 MB, several Apache workers, and `claude` on its way to 860 MB. Two processes wanted more than the box had. ## 8. `swapon`: give the kernel a buffer **Swap** is disk space the kernel uses as overflow when RAM runs out. It's slow, but it gives the system a buffer instead of a hard wall. Both my servers had none. Both got a 2 GB swap file during this series, added by the agent: ```bash sudo fallocate -l 2G /swapfile sudo chmod 600 /swapfile sudo mkswap /swapfile sudo swapon /swapfile ``` `fallocate` reserves the space, `chmod 600` keeps other users out of it, `mkswap` formats it, `swapon` activates it. To keep it after a reboot, add `/swapfile none swap sw 0 0` to `/etc/fstab`. `swapon --show` lists the active swap, and nothing printed means none. Swap was the cheaper fix for the OOM problem, and Codex listed it first. I resized the droplet instead, which turned into [a much bigger mistake](https://flaviocopes.com/devops-mistakes-ai/). ## 9. `systemctl status` and `journalctl -u`: is the service running systemd manages the services on Ubuntu, and `systemctl` is how you ask about them: ```bash systemctl status mysql ``` It prints whether the unit is active, since when, its main PID, memory use and the last few log lines. After the binary log purge, the new expiry setting needed `sudo systemctl restart mysql` to apply. The security pass also ran a check I had never thought of: ```bash systemctl --failed ``` This lists units in the failed state, and an empty list is what you want. After the Sendy server's upgrade and reboot, it confirmed Apache, MySQL, SSH, Fail2ban and cron were all active again. Service logs live in the journal, and `-u` picks one unit: ```bash sudo journalctl -u ssh --since "10 minutes ago" ``` On Ubuntu the SSH daemon's unit is `ssh`, not `sshd`. This shows every connection attempt in the window, including the ones the server rejected and why. `ssh -v` on the client shows what your computer offered; this shows why the server said no. ## 10. `docker compose ps` and `logs`: the same questions on the Docker box Plausible runs as three containers, so "is it running" becomes: ```bash docker compose ps ``` Run it inside the directory with the `compose.yml` (mine is `/opt/plausible-ce-current`). It lists each service with its status and ports. After the upgrade all three showed healthy or running: Plausible CE v3.2.0, PostgreSQL 16, ClickHouse 24.12. The first HTTP check after the upgrade got a 502. The logs explained it: ```bash docker compose logs -f plausible ``` The app was still running database migrations. Once the logs showed normal startup, the check returned 200. A 502 right after a restart often means "not ready yet". One snag from that session: `docker compose exec -T` hung after the command inside had finished. Plain `docker exec -i ` worked. ## 11. `curl -I`: does it answer `curl -I` sends a HEAD request and prints only the response headers. It works from the Mac as well as from the server: ```bash curl -I https://b.flaviocopes.com ``` The answers I was after: `HTTP/2 200` after the Plausible upgrade, `200 OK` after every hardening step on the Sendy server (a security pass that takes the site down is a classic), and `HTTP/1.1 101 Switching Protocols` on `/live/websocket` once the Nginx websocket headers were fixed, which is what proved the [Plausible upgrade](https://flaviocopes.com/updating-plausible-with-ai/) was done. The variant worth knowing is `--resolve`. During the Sendy migration the new server had to be tested before DNS pointed at it, and `--resolve` makes curl use a specific IP for a hostname: ```bash curl -I --resolve list.flaviocopes.com:443: https://list.flaviocopes.com/ ``` TLS still validates against the real hostname. ## 12. `dig +short`: what does DNS say `dig` asks DNS a question. `+short` prints just the answer: ```bash dig +short list.flaviocopes.com ``` Right after the migration I got `ERR_CONNECTION_REFUSED` in the browser. The new server was fine. My browser had the old IP cached, and the old Apache had just been stopped. `dig +short` printing the new IP was the sign DNS had caught up. When the [newsletter took eight hours to send](https://flaviocopes.com/sendy-slow-emails-ai/) and the open rate dropped from 24% to 16%, the open rate part traced back to a record I had set to "proxied" in Cloudflare. With the proxy on, `dig` returns Cloudflare's IPs, not your server, and AWS SNS webhooks were not reaching Apache. A public resolver shows what the world sees: ```bash dig +short list.flaviocopes.com @1.1.1.1 ``` Switching the record to DNS-only fixed the webhooks. ## 13. `ss -tulpn`: what is listening `ss` lists sockets. The flags that matter on a server: ```bash sudo ss -tulpn ``` `-t` TCP, `-u` UDP, `-l` listening only, `-p` the process that owns each socket, `-n` numeric ports instead of service names. You get one line per listener: local address and port, plus the program behind it. On the Sendy box, the security audit confirmed MySQL listened only on `127.0.0.1` and Apache exposed only 80 and 443. On the Plausible box, the app listens on `127.0.0.1:8000` behind Nginx. Anything listening on `0.0.0.0` or `*` is reachable from the internet unless the firewall says otherwise. A service bound to localhost is safe even with the firewall off, so read this list before the firewall rules. ## 14. `ufw status verbose`: is the door locked UFW is Ubuntu's firewall front end: ```bash sudo ufw status verbose ``` It prints `Status: active` or `inactive`, the default policies for incoming and outgoing traffic, and one line per rule. On the freshly migrated Sendy server it said inactive, so every port on the machine was reachable from the internet. When I asked the agent what the biggest issue had been, this was its answer. The fix was to allow 22, 80 and 443 and turn it on. The whole audit is in [Securing a server using AI](https://flaviocopes.com/securing-a-server-with-ai/). On the Plausible box it was active, but with two extra rules: public `2375/tcp` and `2376/tcp`, the Docker API ports. Nothing was listening there, and they got deleted. ## 15. `fail2ban-client status sshd`: who is knocking Fail2ban watches the logs and bans IPs that fail to log in too many times. To see it working: ```bash sudo fail2ban-client status sshd ``` It prints the filter side (currently failed, total failed, which log it reads) and the action side (currently banned, total banned, the banned IP list). On a public server the total climbs steadily. The security pass confirmed the jail was active on the Sendy server. SSH there rejects passwords, and root can log in with a key only (`PermitRootLogin prohibit-password`), so the bans are mostly noise reduction. If you change the SSH configuration, run `sudo sshd -t` before restarting, and keep your session open until a second login works. ## 16. `apt update && apt upgrade`: keep it patched Ubuntu prints pending updates in the login banner: 124 security updates on the rebuilt Sendy server, 127 lines on the Plausible box that had gone untouched for years. Applying them: ```bash sudo apt update && sudo apt upgrade ``` `update` refreshes the package lists. `upgrade` installs newer versions of what you have without removing anything. `apt list --upgradable` shows what's pending first, and after the Plausible upgrade only `ubuntu-advantage-tools` was left on that list. Kernel and libc updates need a reboot, and Ubuntu leaves a marker when that's the case: ```bash cat /var/run/reboot-required ``` Both servers needed it. After the reboot, `uname -r` tells you which kernel is running. The Sendy server came back on `6.8.0-110-generic`, the Plausible one on `5.15.0-176-generic` (more on `uname` in [Linux commands: uname](https://flaviocopes.com/linux-command-uname/)). The agent rebooted on purpose, to check that the firewall and the SSH policy survived a restart. Ubuntu can also install security updates on its own, through the `unattended-upgrades` package. To see what it would do right now, without doing it: ```bash sudo unattended-upgrade --dry-run -d ``` The security pass enabled this on the Sendy server: refresh the package lists daily, install security updates automatically, clean the cache weekly. The one thing it can't do is the kernel reboot, which I can schedule at a quiet hour when `/var/run/reboot-required` shows up. The last check the agent ran after hardening was `sudo certbot renew --dry-run`, a rehearsal of the certificate renewal against Let's Encrypt's staging server, because copied certificates are only useful if renewal works on the new box. ## 17. `crontab -l`: what runs on a schedule Sendy sends scheduled campaigns from cron. During the migration the agent disabled that cron job on the new server until cutover, because two live servers would have sent 150,000 people every email twice. I hadn't thought of that. ```bash crontab -l systemctl list-timers ``` `crontab -l` lists the current user's cron jobs. `systemctl list-timers` shows the systemd timers, where newer software puts its scheduled work. Read both before cloning a server. ## What I'd check first now I still have close to zero Linux server skills, and an agent typed most of what you just read. What changed over these months is that I recognize the output: `99%` from `df -h`, `0B` in the swap row of `free -h`, `Killed process` in the kernel log, `Status: inactive` from `ufw`. Each one tells me which command to run next. Setting up a server tomorrow, I would run five of these before installing anything: `free -h` to check for swap, `ufw status verbose`, `ss -tulpn`, `apt list --upgradable`, and `df -h` with a reminder to run it again in a month. If the command line itself is new to you, the free [Shell Commands course](https://flaviocopes.com/courses/terminal/) covers the basics, and the [Linux Server Troubleshooting course](https://flaviocopes.com/courses/linux-troubleshooting/) turns a list like this one into a method.