Diagnose memory pressure

Linux uses unused memory for cache and can reclaim much of it. That’s why almost every “the server is out of RAM” report from someone reading the used column is a false alarm. Focus on pressure and behavior, not the used column alone.

Read free correctly

free -h

               total        used        free      shared  buff/cache   available
Mem:           7.8Gi       5.1Gi       210Mi       120Mi       2.5Gi       2.3Gi
Swap:          2.0Gi       1.4Gi       600Mi

The column that matters is available: the kernel’s estimate of memory that can be handed to new work, including reclaimable cache. Here free is a scary 210Mi, but available is 2.3Gi. Healthy. The number that should worry you in this output is swap: 1.4Gi used means the system was under real pressure at some point.

Check whether pressure is happening now

Swap used is history. Swap activity is the present. Watch vmstat:

vmstat 5 3

procs -----------memory---------- ---swap-- ...
 r  b   swpd   free   buff  cache   si   so
 2  0 1468k 21540  84120 2519k     0    0
 3  1 1471k  98212  84120 2380k   840  1204

The si and so columns are pages swapped in and out per second. Zeros mean the old swap usage is stale. Repeated swap-in and swap-out — sustained nonzero si/so — means the working set doesn’t fit and the system is thrashing right now.

Then find who’s holding the memory, by process RSS:

ps -eo pid,user,rss,cmd --sort=-rss | head -4
# 3721 app   4183212 node /srv/app/server.js

If the service runs in a cgroup (any systemd unit with MemoryMax), check its limit too — a container can be OOM-killed while the host has gigabytes free.

When the OOM killer fires

The OOM killer records which process it selected and why, in the kernel log:

journalctl -k | grep -i 'out of memory'
# kernel: Out of memory: Killed process 3721 (node) total-vm:5124404kB, anon-rss:4183212kB

The trap: assuming the killed process was the guilty one. The OOM killer picks a victim by score, which favors big processes — but a small, fast leaker can push the system over while the database, being the largest RSS, takes the bullet. Collect memory evidence before and during a controlled workload, and decide from the trend whether the system is healthy, reclaiming cache, swapping, or killing work.