Write the incident review

A useful review explains how the system allowed the incident and how the response unfolded. Not who typed the bad command — how the system made that command possible, easy, and invisible until it hurt. Reviews that hunt for a culprit produce quiet engineers and repeat incidents; reviews that hunt for conditions produce fixes.

Write it within a few days, while the timeline you built and the evidence you preserved still make sense to you.

The structure

One page covers it. Include detection, impact, mitigation, root cause, contributing factors, what worked, and specific follow-up owners:

# Incident review: API outage 2026-08-03

**Impact**: /api/orders returned 500 for all users, 14:05–14:38 UTC (33 min).
**Detection**: user report in Slack at 14:33. No alert fired.

## Timeline (UTC)
- 14:02 deploy of release 2026.31 finished
- 14:05 error rate reached 100% (found later in logs)
- 14:33 first user report
- 14:38 rollback completed, external check green

## Root cause
Release 2026.31 read DB_POOL_SIZE as a string; the pool
initialized with 0 connections. [confirmed in staging]

## Contributing factors
- config values not validated at startup [fact]
- no alert on error rate; detection relied on users [fact]
- deploy finished at peak hours [inference: worsened impact]

## What worked
Rollback took 5 minutes; evidence capture made root cause quick.

## Follow-ups
- [ ] validate config at startup, fail fast — Ana, by Aug 14
- [ ] alert: 5xx rate > 5% for 5 min — Marco, by Aug 10
- [ ] deploy checklist: verify one real request — Flavio, by Aug 7

Separate confirmed facts from inference, exactly like in the timeline lesson. “The pool initialized with 0 connections” was reproduced in staging: fact. “Peak hours worsened impact”: plausible inference, marked as such. A review that presents guesses as findings sends the next responder down the wrong path with extra confidence.

Follow-ups that actually change something

Prefer changes that remove failure modes over changes that ask humans to try harder. “Be more careful with config” fixes nothing — the same person under the same pressure makes the same mistake. “The app refuses to start with invalid config” makes the mistake impossible. And notice the detection gap deserves a follow-up of its own: 28 of the 33 minutes were spent not knowing.

Every follow-up needs an owner and a date, or it’s decoration.

The trap: writing the review and never checking the follow-ups landed. A pile of reviews with open checkboxes is how the same incident happens twice, with a better-documented second act. Write a one-page review for a simulated incident, and end with one prevention change, one detection change, and one response improvement.