Use tests as feedback

When an agent writes code, it runs is not enough. A test is an executable claim about behavior. It should stay true after the next change, and the one after that.

Tests are also the best feedback an agent can get. An error message is much more useful to it than my vague that's not quite right.

Say display names must be 1 to 50 characters. Useful cases:

• 1 character works
• 50 characters works
• an empty name fails
• 51 characters fails
• an existing valid profile update still works

These five lines define the boundary better than validate the display name ever could.

Run the new test before the fix

For a regression, the new test should fail before you change the implementation. And it should fail for the reason you expect. That proves the test reaches the missing behavior.

If it passes right away, stop and look. Maybe the behavior already works. Maybe the reproduction is wrong. Maybe the test isn’t exercising the path you think it is.

After the fix, run the focused test again. Then the nearby tests that could catch a regression.

Test what callers can see

A weak test mocks the validation helper and asserts it was called. The implementation can call the helper, ignore the result, and return the wrong HTTP status. Test passes, bug ships.

A stronger test sends a 51-character name to the actual endpoint and checks the response and the stored data. It describes what users and callers observe. The internal helper can change tomorrow and the test still means something.

Use unit tests for isolated logic and broader tests for important integrations. Pick the level that matches the claim.

Tests can be wrong too

A passing test is evidence only if the assertion expresses the real requirement. An agent can write a test that repeats the same wrong assumption as its code. Then everything is green and nothing is right.

Read the input, the assertion, and the failure output. Ask: would this test fail if the bug were still there? Include negative and boundary cases, not just the happy path.

Generated tests are great as drafts. Models are fast at proposing cases you’d forget. You still decide which behavior is correct.

Find one test in your project that asserts an internal function was called. Rewrite the requirement in terms of observable input and output. Then decide whether the test should move up a level. Often it should.