What would make an overnight agent useful?

Before leaving an agent running overnight, I would write down what I want to decide in the morning.

If I can’t name that decision, I probably haven’t found a useful assignment yet.

An agent can remain busy for hours. It can investigate things I hadn’t considered and produce a lot of code. None of that tells me whether its work will help with the project I’m trying to finish.

I already wrote about which tasks I let agents do unattended. The question here comes earlier. What would make the work worth starting?

Start with something you’re putting off

Imagine a search feature that feels slow. You’re considering replacing its database query, but you haven’t measured where the time goes.

An overnight investigation could compare the current query against a proposed index using generated data. The result would help you decide whether the index is worth adding.

That is a better assignment than “improve search.” It has an answer that can change tomorrow’s work.

The agent might discover that the index barely helps. That would still be useful. You can stop pursuing that idea and investigate another part of the request.

I would ask it to preserve the commands and the generated dataset settings. Otherwise I would have a conclusion I couldn’t reproduce.

The example in the drawing ends at a decision. The measurement is hypothetical; the useful result is enough evidence to accept or reject one proposed change.

Some long jobs need no agent

Once you know exactly which checks to run, a script or CI job may already do the work.

Running the same test suite every night doesn’t need a model to rediscover its command. A scheduler can start it, save the output, and notify you when it fails.

An agent becomes more useful when the next step depends on what it finds. It can inspect a failure, trace the affected code, and prepare a possible explanation with evidence. I would still keep that investigation separate from permission to deploy a fix.

This also changes how I think about a swarm of overnight agents. Several workers make sense when there are genuinely separate questions to investigate. Starting more workers because the dashboard supports them gives me more output to read without necessarily resolving more uncertainty.

Design tomorrow’s reading

Suppose you wake up to six large changes.

Each one needs review, and they might depend on one another. Your first task becomes reconstructing the night’s work before you can decide whether to use any of it.

I would prefer one short report about the search question. It should show the comparison, explain what wasn’t checked, and point to the files needed to repeat it. A proposed patch can wait beside that evidence.

There also needs to be a limit. An inconclusive investigation should stop and say what blocked it. Spending more time on the same missing credential or unavailable dependency won’t make the morning easier.

Leaving a question unanswered is allowed

Some assignments turn out to need a product decision.

Perhaps the search is slow because it includes archived records. Excluding them would speed things up, but users might need those results. The agent should surface that choice and stop there.

I would give the investigation access to a disposable local dataset and keep its permissions small. Even a useful question doesn’t justify handing over production access by default.

For a first attempt, choose a question you’ve postponed because gathering the evidence takes time. Decide how you will check the answer before starting. If the report saves you that investigation tomorrow, you have a reason to run it again.

Want me to talk about your product? You can sponsor this site.