Operate the server

Recover or retire the Droplet

Use the recovery console carefully, rebuild compromised systems, and remove every billable resource when the server is no longer needed.

Two things happen to every server eventually. One day you can’t SSH into it. One day you don’t need it anymore. Both go better with a plan written in advance.

When SSH is gone

The DigitalOcean recovery console gives you a terminal on the Droplet through the control panel, without going over the network. It’s the tool for a server that’s running but unreachable. Use it instead of turning password login back on.

Once you’re in, collect evidence before changing anything:

df -h
sudo systemctl --failed
sudo systemctl status ssh --no-pager
sudo sshd -t
sudo ufw status numbered
sudo journalctl -u ssh -b --no-pager

Each line tests a different cause. df -h at 100% means a full disk. systemctl --failed shows services that died. sshd -t prints the bad line if your last config change broke it. ufw status shows whether the SSH rule is gone. The journal shows why sshd refused connections.

Fix only the broken layer. If a drop-in file caused it, move it aside, run sshd -t, reload SSH, and test a real remote login before closing the console.

When the server was compromised

A compromise is not a configuration bug. If you find a miner, a strange cron job or a user you didn’t create, assume there’s more you haven’t found.

Preserve the logs and write down the timeline. Cut the server off the network if you can. From a computer you trust, rotate every credential the server had: SSH keys, database passwords, API tokens. Then rebuild from a fresh image and a verified backup.

Retiring a server cleanly

Before you destroy a Droplet that served real traffic:

  1. move DNS or traffic to the tested replacement
  2. keep the old server up until the old DNS TTL has expired
  3. take a final data backup and restore it somewhere to prove it works
  4. copy the logs your retention policy requires
  5. rotate credentials the server used
  6. check that no cron job, webhook or integration still points at the old IP

Then destroy the Droplet from the control panel. Powering it off is not enough. A powered-off Droplet keeps billing.

Now look through the project for leftovers with their own bills: volumes, snapshots, backups, reserved IPs, load balancers, firewalls. Delete only what you’ve positively identified as unused. A snapshot might be there on purpose. An unattached volume still costs money and may still hold sensitive data.

There is no undo for Destroy. Before clicking it, confirm the Droplet name, project, IP, the health of the replacement and where the final backup is.

Write two short runbooks now: “SSH is unavailable” and “retire this Droplet”. Include how to reach the recovery console, the commands above, where the backups are, who controls DNS, and what evidence you need before destroying anything. Future you will thank you.

Lesson completed