Reliability and CI

Add a k6 smoke test

Send a small controlled workload to the Books API and define thresholds that turn latency and error expectations into pass or fail.

All the tests so far ask “is the answer right?”. A load test asks “does it stay right when many requests arrive at once?”. We’ll start with the smallest possible version, a smoke test: a tiny amount of traffic against a server we own, to check nothing falls over.

The tool is k6. You write the scenario in JavaScript and k6 runs it with many virtual users.

The script

Save this as smoke.js:

import http from 'k6/http'
import { check } from 'k6'

export const options = {
  vus: 2,
  duration: '10s',
  thresholds: {
    checks: ['rate>0.99'],
    http_req_failed: ['rate<0.01'],
    http_req_duration: ['p(95)<300']
  }
}

export default function () {
  const response = http.get('http://localhost:3000/books')
  check(response, { 'status is 200': r => r.status === 200 })
}

Two virtual users (vus) call GET /books in a loop for ten seconds. Run it with the Books API up:

k6 run smoke.js

You get a summary like this:

✓ status is 200

checks.........................: 100.00% ✓ 412  ✗ 0
http_req_duration..............: avg=4.8ms p(95)=9.2ms
http_req_failed................: 0.00%   ✓ 0    ✗ 412

Checks versus thresholds

A check records whether one response looked right. A failed check shows up in the summary, but on its own it doesn’t make k6 exit with an error.

A threshold is the pass/fail contract for the whole run. That’s why the checks threshold is there: it says over 99% of checks must pass, and connects the checks to your CI exit code. The other two say fewer than 1% of requests may fail, and 95% of them must finish under 300 milliseconds.

Why the 95th percentile

Averages hide the slow tail. Nine requests at 10ms and one at 2 seconds average out to about 200ms, which looks fine. The tenth user waited two seconds.

p(95) says “95% of requests were faster than this”. Pick the number from a real expectation or a measured baseline. Not from whatever makes today’s run green.

What a smoke test is not

Two users for ten seconds tells you the system survives a small controlled load and answers correctly. It tells you nothing about capacity. Finding the maximum throughput needs realistic traffic shapes, isolated test data, observability on the server, and a conversation with whoever owns that environment.

Run against infrastructure you own, locally or in a dedicated environment. Never point k6 at a shared staging server without asking.

One more detail. A fast 200 with an error page in the body still passes the status check. Add a second check on a small stable body property, like the response being a JSON array. And keep credentials and unbounded data creation out of the script.

Try this: run the smoke test and note the request count, p95, error rate, and check rate. Then break the response body while keeping the 200, and separately add a 500ms delay to the route. The two changes should fail two different thresholds.

Lesson completed