Self-improving agents: what refinement actually means

A self-improving agent is one that changes its own inputs — its instructions, its skills, its memory of a repository — based on what previous runs taught it. It is worth being precise about the verb: today's self-improvement edits the context around a model, not the model. Nothing about your weights changes, and any product implying otherwise is describing a different, far more expensive thing.

What are the three kinds of self-improvement?

They are frequently conflated, and they differ by orders of magnitude in cost, risk, and how much of it actually exists in shipping products.

Most of what is marketed as a self-improving agent is the first row. That is not a criticism — the first row is genuinely useful, and it is the only one you can have while still using your own provider's model. It is a criticism of the word "learning" when it is doing work the mechanism cannot support.

What does refinement look like in practice?

An agent finishes a task and records something durable about the repository: this project's tests need a running database, this module's owner rejects changes that touch the public API, this build step is slow and should be run last. The next run starts with those facts already present instead of re-discovering them from scratch.

In Relay that state lives on your machine and is editable by you — it is a file, not a black box. The value is real and unglamorous: the second run on a repository is better than the first because it does not repeat the first run's dead ends.

Is refinement the same as search?

No, and the difference is worth stating plainly because it is where the honest limit sits. Search means generating multiple candidates, scoring them against a measurable objective, and keeping what wins. Refinement as it ships today — in Relay and, as far as we can tell, in everything comparable — writes, updates, and deletes state. There is no candidate generation, no fitness function, and no accept-or-reject step.

Refinement remembers. Search chooses. Today's self-improving agents do the first and call it the second.

The prerequisite for the second is a way to tell whether a change helped, which means a benchmark you trust more than the thing being benchmarked. That is the order we are building in: measure whether refinement helps at all, then decide whether search on top of it is worth having.

What are the risks of a self-improving agent?

Two, both structural. The first is that a system optimising itself will optimise whatever you measure, including the measurement — which is why the thing being improved must never be able to weaken the thing that verifies it. The second is that stored state is a memory of a moment: a note that was true in March quietly becomes a wrong instruction in September, and nobody reviews it, because reviewing your agent's private notes is not on anyone's calendar.

Keeping that state local, in a file you can read and delete, is not a small thing. It is the difference between a wrong belief you can find and one you can only observe the effects of.

Does a self-improving agent need my code on someone's server?

No. Relay's refinement state, sessions, and memory all sit on your machine, and the model call goes to your own provider with your own key. Nothing about improving over time requires uploading a repository — what a signed receipt proves covers what does leave when you choose to share evidence, and the pricing page covers what a team pays for, which is governance rather than access.