Review generated code for security

Generated code can work perfectly for the intended user and still be unsafe. Security review asks a different question from does it work?. It asks what happens when the input, the identity, or the order of events is not what we expected?

Here’s a route an agent might generate:

app.get('/api/invoices/:id', requireLogin, async (req, res) => {
  const invoice = await db.invoice.findById(req.params.id)
  res.json(invoice)
})

It checks authentication: the caller is signed in. It doesn’t check authorization: whether this caller may see this invoice.

Any logged-in customer can change the ID in the URL and read someone else’s invoice. The feature works. The trust boundary is broken. I’ve seen agents produce exactly this, more than once.

A small threat model

Ask five questions:

• What input can an attacker control?
• What data or action does this code protect?
• Which identity performs the operation?
• Where must the server enforce the boundary?
• What happens with missing, oversized, malformed, or cross-account input?

For the invoice route: scope the query by both the invoice ID and the current account. Then write a test that signs in as one customer and proves another customer’s invoice is not returned.

Watch for convenience

Generated code often removes friction by weakening a boundary. Look for:

• permissive CORS or wildcard permissions
• disabled certificate or signature checks
• SQL or shell commands built by string concatenation
• secrets in source or browser code
• verbose errors that leak internal data
• destructive operations without confirmation
• a new dependency added for a tiny task

Each of these is the kind of thing that makes a demo work faster. That’s exactly why a model reaches for them.

Also treat content the agent reads as untrusted. A document or web page can contain prompt injection aimed at your connected tools. Normal authorization and validation must still apply after the model picks an action.

Exercise the abuse case

A security suggestion is not a fix until you run the dangerous path. Add the cross-account test, the malformed-input test, or the injection test, and confirm it fails safely.

A second reviewer or a fresh model session can catch things you missed. Neither replaces the test or the trust-boundary analysis.

Go back to the invoice route above. Describe one abuse request, the server-side boundary that should stop it, and the negative test that proves the fix. If you can do that for this route, you can do it for the next one an agent hands you.