What would make an overnight agent useful?

Before leaving an agent running overnight, I would write down what I want to decide in the morning.

If I can’t name that decision, I probably haven’t found a useful assignment yet.

An agent can remain busy for hours. It can investigate things I hadn’t considered and produce a lot of code. None of that tells me whether its work will help with the project I’m trying to finish.

I already wrote about which tasks I let agents do unattended. The question here comes earlier. What would make the work worth starting?

Start with something you’re putting off

Imagine a search feature that feels slow. You’re considering replacing its database query, but you haven’t measured where the time goes.

An overnight investigation could compare the current query against a proposed index using generated data. The result would help you decide whether the index is worth adding.

That is a better assignment than “improve search.” It has an answer that can change tomorrow’s work.

The agent might discover that the index barely helps. That would still be useful. You can stop pursuing that idea and investigate another part of the request.

I would ask it to preserve the commands and the generated dataset settings. Otherwise I would have a conclusion I couldn’t reproduce.

The example in the drawing ends at a decision. The measurement is hypothetical; the useful result is enough evidence to accept or reject one proposed change.

Some long jobs need no agent

Once you know exactly which checks to run, a script or CI job may already do the work.

Running the same test suite every night doesn’t need a model to rediscover its command. A scheduler can start it, save the output, and notify you when it fails.

An agent becomes more useful when the next step depends on what it finds. It can inspect a failure, trace the affected code, and prepare a possible explanation with evidence. I would still keep that investigation separate from permission to deploy a fix.

This also changes how I think about a swarm of overnight agents. Several workers make sense when there are genuinely separate questions to investigate. Starting more workers because the dashboard supports them gives me more output to read without necessarily resolving more uncertainty. The same applies to the chats those workers leave behind. I wrote about that in Too many conversations with coding agents.

Design tomorrow’s reading

Suppose you wake up to six large changes.

Each one needs review, and they might depend on one another. Your first task becomes reconstructing the night’s work before you can decide whether to use any of it.

I would prefer one short report about the search question. It should show the comparison, explain what wasn’t checked, and point to the files needed to repeat it. A proposed patch can wait beside that evidence.

There also needs to be a limit. An inconclusive investigation should stop and say what blocked it. Spending more time on the same missing credential or unavailable dependency won’t make the morning easier.

Write the assignment before starting

Keep the question in a file so the morning review has something to compare against. For the search investigation, I would use this brief:

Investigate whether an index could reduce the search query's latency.
Use only the disposable local test database and generated records.
Do not change the application, connect to production, or deploy anything.

Record the database version, schema, record count and generation seed.
Run the original query and the indexed version against the same data.
Keep the query text, query plans and timings for each run.
Explain which results would change the decision and which remain uncertain.

Finish with a report containing:
- the decision these measurements support
- commands and results needed to reproduce the comparison
- what you could not check
- the next action for me, including any unanswered product question

If a required credential, dependency or decision is missing, report that
blocker and stop. Do not replace a measurement with an invented result.

The application files stay unchanged in this first exercise. Temporary database objects belong in the disposable database, and the report goes to a separate output folder you choose before starting. Give the agent access only to that test environment.

A sentence in the brief cannot enforce a time or spending limit. Configure those in the runner or service that starts the task. Test the stop behavior with a short disposable run while you’re present before leaving it overnight. Check that stopping the agent also stops any test processes it launched.

In the morning, compare the report with the original brief. Re-run one measurement before accepting the conclusion. Missing raw timings, a different dataset, or an unexplained configuration change means the comparison still needs work.

Leaving a question unanswered is allowed

Some assignments turn out to need a product decision.

Perhaps the search is slow because it includes archived records. Excluding them would speed things up, but users might need those results. The agent should surface that choice and stop there.

I would give the investigation access to a disposable local dataset and keep its permissions small. Even a useful question doesn’t justify handing over production access by default.

For a first attempt, choose a question you’ve postponed because gathering the evidence takes time. Decide how you will check the answer before starting. If the report saves you that investigation tomorrow, you have a reason to run it again.

Want me to talk about your product? You can sponsor this site.