Timers and operations
Handle missed and randomized runs
Choose what happens after downtime and avoid making every server start maintenance at the same instant.
8 minute lesson
A cron job that was scheduled while the machine was off just never happens. Timers let you choose the behavior instead of inheriting it.
Catching up with Persistent=
Persistent=true lets a calendar timer catch up after the machine was off. systemd records the last trigger time on disk; if the machine boots and a run was missed, the timer fires once, immediately.
[Timer]
OnCalendar=*-*-* 03:00:00
Persistent=true
RandomizedDelaySec=30min
That behavior is useful only when a late run is safe. A missed backup should absolutely catch up. A billing job that charges customers should probably not fire as a surprise the moment a machine comes back — especially during incident recovery, when a boot triggering a pile of catch-up jobs is the last thing you want.
This is why idempotent jobs are safer: retries or catch-up runs do not duplicate damage. A backup that overwrites today’s file is idempotent. A job that appends charges is not, and it needs its own run-once guard before you let a timer catch it up.
Spreading the load
RandomizedDelaySec= spreads work across a window. With the 30 minute value above, each server picks a random delay after 03:00, so a fleet of a hundred machines does not hammer the backup target at the same instant.
AccuracySec= controls scheduling precision and defaults to one minute — systemd batches timer wakeups to save power. If a job genuinely must fire at 03:00:00, set AccuracySec=1s. Most jobs do not care, and the default is fine.
Verify the resulting schedule instead of trusting your reading of it:
systemctl list-timers backup.timer
NEXT LEFT LAST PASSED UNIT ACTIVATES
Tue 2026-08-04 03:11:42 UTC 9h left Mon 2026-08-03 03:24:17 UTC 14h ago backup.timer backup.service
The randomized offset is visible right in the NEXT column.
Decide per job
Work through three common jobs: a temp-file cleanup, a database backup, and a monthly billing run. For each one, decide whether a missed run should catch up, and give it a persistence and randomization policy. Cleanup and backup usually get Persistent=true. Billing usually gets Persistent=false and an alert when a run is missed, so a human decides.
Lesson completed