Finish the testing strategy

A test suite is a product. It has users, it needs maintenance, and if you neglect it people stop using it. Slow, duplicated, ownerless tests become tests everyone ignores.

Let’s turn what we built for the Books API into a strategy someone else can pick up.

Write the risk matrix

Don’t describe the suite as folders. Describe it as risks. One row per risk, with the observable evidence, the boundary that must be real, where the test lives, how long it takes, who owns it, and any known gap.

Two rows from ours:

Risk	Evidence	Boundary	Test	Runtime
Duplicate ISBN overwrites a book	Insert rejected, one row remains	Database constraint	books.db.test.js	2s
Reader can add a book from the page	New list item visible	Browser + API + storage	add-book.spec.ts	8s

The matrix shows two things at once. Tests that cross the same boundary to prove the same thing, which you can delete. And risks no row covers, which you need to write.

Map changes to feedback

Not every change needs the whole suite. A validation edit should run the unit and HTTP contract tests, and answer in seconds. A migration edit must run clean-install, upgrade, and repository tests. A shared UI change should run the critical browser tests.

The full suite still runs before release. But a developer fixing normalizeBook() shouldn’t wait for Playwright to learn that a pure function is wrong.

Decide the flaky-test policy now

Decide before the first flake. Once flakiness is normal, nobody decides anything.

Mine: on the first failure, keep the trace and environment details, assign an owner, and fix the cause. It’s always shared state, timing, or infrastructure. If you have to quarantine a test, keep it visible, with a reason and an expiry date. Retries collect evidence. A test that eventually went green still failed.

Review by unique value

Every few months, ask of each test: what does this prove that nothing else proves? If a stronger test covers the same risk and this one doesn’t fail faster or point closer to the cause, delete it.

Write deliberate gaps down. Testing every validation rule through the browser is too expensive when unit and HTTP tests already protect them. Saying so in the matrix stops the next person from adding those tests back.

Try this: write the Books app matrix from pure validation through API, database, browser, and the k6 smoke test. Mark one intentional gap, one duplicated test to remove, and the failure artifact each layer keeps. Give every flaky or quarantined test an owner and a review date.