CPU, memory, and storage

Interpret load and CPU

Separate runnable demand, blocked work, CPU utilization, per-core saturation, and one busy process.

8 minute lesson

~~~

Load average is not a CPU percentage. It counts runnable tasks and tasks in uninterruptible sleep, averaged over 1, 5, and 15 minutes. That last part is why Linux load numbers confuse people: a machine doing zero CPU work can show a load of 20 if twenty processes are stuck waiting on a dead disk.

Read load against the core count

uptime
# 15:04:33 up 12 days,  3:12,  1 user,  load average: 6.42, 5.90, 2.15
nproc
# 4

Compare load with CPU count. Load 6.4 on 4 cores means demand exceeds capacity: on average, 2+ tasks are waiting at any moment. The three numbers also give you direction — here the 15-minute average is 2.15, so this started recently and is getting worse.

Find out what kind of demand it is

top splits CPU time into categories on its %Cpu(s) line:

%Cpu(s):  8.1 us,  2.3 sy,  0.0 ni, 41.2 id, 48.0 wa,  0.0 hi,  0.4 si

The fields that matter: us is user code, sy is kernel work, id is idle, and wa is I/O wait — CPU sitting idle while tasks wait for storage. This sample is the classic pattern: load is high, but 41% of CPU is idle and 48% is wa. The bottleneck is the disk, not the processor. Chasing a “CPU problem” here wastes the afternoon.

Then check per-core numbers, because averages hide saturation:

mpstat -P ALL 5 1
CPU    %usr   %sys  %iowait   %idle
all   26.10   3.02     0.51   70.37
  0   99.20   0.60     0.00    0.20
  1    2.10   3.80     0.70   93.40

One saturated core can limit a single-threaded application while total CPU still looks moderate — 26% overall, but core 0 is pinned at 99% and that’s exactly where your Node process lives. pidstat 5 (or pressing 1 inside top) tells you which process owns the busy core.

Capture five timed samples during a controlled load rather than staring at one refresh. A single snapshot catches noise; five samples show whether demand is user CPU, system CPU, I/O wait, or one constrained process.

The trap is the one this lesson opened with: high load with idle CPUs. Load counts D-state tasks too, so slow storage or a hung NFS mount inflates it. When load looks scary, check wa and process states before you blame the CPU.

Lesson completed

Take this course offline

Get every free book and course as PDF and EPUB files.

Get the download library →