# Give your agent setup a week of ordinary work

> Try a stable agent setup on ordinary work before changing it again. Record finished tasks, review effort, and repeated failures to see what needs improving.

Author: [Flavio Copes](https://flaviocopes.com/about/) | Published: 2026-11-03 | Topics: [AI](https://flaviocopes.com/tags/ai/) | Canonical: https://flaviocopes.com/stop-tuning-agent-setup/

An agent setup needs enough ordinary use for you to notice what it helps you finish.

If you replace it after every demonstration you see, you spend a lot of time evaluating the first few minutes. The awkward parts of a working week may never enter the comparison.

I would keep one setup stable for a short trial and record what happens. A week can be a convenient starting point, provided it contains the kind of work you actually need to do.

## Start with the work waiting for you

Suppose you're maintaining a small membership application.

There is a broken receipt link to investigate. A settings page needs a clearer error state. You also need to understand an older import function before changing it.

Those tasks give the setup something useful to do. They exercise investigation, implementation, and explanation without requiring you to invent a benchmark project.

For each one, write down what a finished result looks like. The receipt link needs to open the correct receipt for the correct account. For the settings page, check that a failed save produces a visible error. An explanation of the import needs references you can inspect in the code.

Now you can distinguish finishing a task from producing an answer about it.

The drawing follows a repeated receipt-link failure into a small correction to the setup. The check stays with the project so you can see whether the next change preserves it.

## Change the setup for an observed problem

Imagine the agent keeps running the wrong test command.

That gives you a specific thing to fix. Put the correct command in the project's development instructions, check that the agent discovers it, and use the next relevant task to see whether the confusion returns.

Installing another orchestration layer would make the experiment larger without answering that question.

Other failures deserve different responses. If the environment cannot reach the local database, fix that connection or choose an environment that can. If the agent consistently misunderstands which behavior must remain, inspect the brief and the project record.

Changing models could be reasonable too. I would want to preserve the task and starting files so the alternative has a comparable question to answer.

The aim is to connect each setup change to a problem you can recognize afterward.

## Keep the repair as a fixture

For the receipt link, save a small case that exposes the original failure.

It could be a test account with a known receipt identifier in a disposable environment. The check should establish which account can open that receipt and what another account receives.

Use fake records and avoid production billing actions. The setup trial needs an observable result, not a reason to give every experimental tool broad access.

After changing an instruction or integration, run the case again. If the problem returns, you have evidence that something regressed.

A collection of these checks is useful because it reflects the work your setup has already failed. Start with the ordinary commands your project can run.

## Record your part of the work

At the end of the trial, inspect the finished tasks.

Which changes survived review? Where did you have to take over? Did you spend time fixing the environment before the agent could begin? Were there tasks you abandoned and restarted elsewhere?

Keep those failed attempts in the notes. The final successful run doesn't include the cost of the earlier ones unless you record it.

You don't need a complicated score. A short entry describing the job, the outcome, and the intervention can reveal a repeated problem.

If you spent most of the week tuning the setup, the result is still informative. You haven't yet learned how it behaves during a normal week of product work. Decide whether another setup change is necessary or whether it is time to use what you have.

## Experimenting can be the thing you enjoy

There is nothing wrong with treating agent tools as a hobby.

You can enjoy testing plugins, comparing interfaces, or building a new harness. The experiment itself can be worth your time.

I would just give that activity its own place in the week. If the goal is to finish the membership application's fixes, look at those fixes when deciding whether the setup improved.

Choose one change to the setup that addresses a recorded failure. Keep a way to restore the previous configuration. Then use it long enough to discover whether ordinary work becomes easier to finish.