Secure, test, and operate
Load-test connections and fanout
Measure concurrent connections, event fanout, queue growth, CPU, memory, and reconnect storms.
A single fast laptop client tells you almost nothing about ten thousand quiet viewers and one incident broadcast.
Live systems usually fail on connection count and fanout shape, not on ordinary HTTP requests per second.
Two different pressures
Case A: 1 incident -> 10 000 SSE viewers (mostly idle)
Case B: 50 operators sending commands every second
Case A stresses memory per connection and broadcast loops. Case B stresses CPU per message and authorization checks. Benchmark both separately instead of blending them into one vanity number.
What to record
Stage the test and write down the first limit you hit:
connections open: 8000
broadcast latency p99: 120 ms
heap used: 1.4 GB
CPU: 78%
first failure: outbound queue cap on slow client profile
Tools like autocannon help for HTTP snapshots. For WebSockets you often need a small script that opens N connections, listens, and logs reconnect timing. The exact tool matters less than holding connection lifetime, message size, and fanout constant while you change one variable.
Include recovery events
Load tests that only measure steady state miss the scary part. After the soak phase, kill the server and measure synchronized reconnects. That is when backoff and snapshot recovery earn their keep.
Run a staged test, note the first resource that saturates, and stop chasing a headline connection count that hides queue growth.
Slow client profile
Include at least one client that sleeps fifty milliseconds before reading each message. That profile triggers bufferedAmount growth long before average viewers notice trouble.
Record reconnect storm separately after the soak. A graph with two humps tells you whether backoff worked or everyone retried in the same second.
Try this on your own project: soak ten thousand idle SSE clients, broadcast one incident update, and record p99 delivery time plus heap usage.
Lesson completed