Changelog
What changed, and what did not.
Every release says what it fixed and what it deliberately left undone. The second list is the one worth reading.
All notable changes to this project.
0.6.70 — 2026-09-01
Two places where the interface said one thing and the system meant another.
A card headed optional sat two lines above the words "required by this
repository's policy". relayevals verdict appends that phrase to a next
action when coverage_enforcement is block — and under that setting those
surfaces are precisely why the verdict is BLOCK. The drive's Recommendation
card hardcoded optional in its header regardless. So the card naming the
thing that stopped your build called it optional.
On a product whose whole proposition is that PASS, BLOCK, required and
advisory each mean exactly one thing, an interface caught contradicting
itself once leaves a reader no way to know which half to believe — or reason
to believe the next claim either. The distinction already existed; it was
prose at the end of a sentence. It is now a field, set from the same
condition that writes the sentence, and the header derives from it. Absent
reads as advisory, because an advisory mislabelled required is annoying and a
requirement mislabelled advisory is the bug.
The installer can no longer call a shadowed install a success. It
verified itself by absolute path, which proves the bytes on disk are the
release and says nothing about what happens when you type relayevals. Those
differ whenever something else with that name is earlier on your PATH, and a
/usr/local install from 0.4.2 or earlier is exactly that. The old handling
was a note at the top of the run, hundreds of lines before a success banner
and a "Next: relayevals try".
Measured on the machine this was written on: command -v relayevals under
login bash resolved to /usr/local/bin and answered 0.4.2 — sixty-five
releases behind — while the same machine's zsh answered correctly. It made
this project's own 0.6.69 release verification report a good build as broken.
The installer now finishes by asking your question with your PATH:
Installed: 0.6.70
Invoked: 0.6.70
Path: ~/.local/bin/relayevals
Relay is ready.
When those disagree it prints no "Next" at all — it names the binary that
wins, its version, the exact command to remove it, and exits non-zero. A
pipeline asking the exit status of curl … | bash is asking "can I use this
now", and the answer is no. Not yet on PATH is reported as its own state
rather than a failure: that is the ordinary first install, and the fix is one
export line.
NextActiongains an optionalrequiredfield. Consumers of
--format json see it on actions the policy made mandatory; its absence
means the artifact never claimed the action was required.
0.6.69 — 2026-09-01
**The workflow we hand paying teams did not gate and did not report. The audit
log stops asking to be believed.**
The team console handed a customer a .github/workflows/relay.yml that ended
after setup: no verdict, so nothing gated, and no ci report, so no run
could ever reach the console that job exists to fill. Paste it exactly and you
get a green build that proves nothing and an empty dashboard, with no error
anywhere to connect the two. /docs/ci's hand-written job had the other half
of the same hole — it gates correctly and never speaks to the server, so a
team on GitLab or a self-hosted runner loses runs, catches, catch rate and
per-repo health, again silently. Production agreed: six checks, all on one day
in August, none since.
The published action was fine throughout and is now what both surfaces show.
Both render one string from apps/web/lib/ci-workflow.js, substituted into
the markdown at read time rather than pasted, so they cannot drift again. Two
guards in the hand-written job are load-bearing and both look like noise:
if: always() on the report step, because verdict exits 1 on BLOCK and a
report step without it is skipped on exactly the runs worth recording; and
continue-on-error on the test step, because receipt run propagates the
child's exit code and a red suite is a verdict, not a crash.
- The audit log is hash-chained. Every entry carries the hash of the entry
before it in its team, written by a BEFORE INSERT trigger — there are nine
INSERT sites across seven files, and a chain maintained by callers would be
a promise repeated nine times. The console recomputes the whole chain on
every read, in the application, because reading a column the database filled
in answers nothing. It says tamper-evident, not tamper-proof: anyone who
can write to the table can rebuild the chain, and the page says so in the
same breath as the result.
- A check now records the policy it ran under.
ci reportforwards
policy.id from the verdict artifact — sha256 over the canonical policy, so
it cannot carry a path. It is the same construction the server already uses
for a published team policy, so a repository running exactly its
organization's baseline reports a digest the console recognises and the run
page names the version. Omitted rather than nulled when absent: the
signature covers the canonical body, so old and new clients both verify.
- Runs. Every check the gate ever reported, filterable, keyset-paginated,
CSV and JSON. The run page refuses to show a receipt and explains why — we
never received one — then shows the entire stored row and the
relayevals receipt replay command that re-verifies the real artifact where
it lives.
- Trust replaces "active" on the members page, from live certificates.
Per-repo health replaces a pair of GitHub ids with what the gate actually
did. The overview leads with catches rather than checks.
- Fixed: every pending invitation read "expires 0m from now", because the cell
used a helper that clamps at zero for past events. Fixed: the CI snippet and
/install pinned 0.6.23 — forty-five releases stale, on the copy-paste
path — now one RELAY_VERSION with a test that fails when the release
bump leaves it behind. Fixed: ` **code** ` rendered its own backticks
on every long-form page.
Invariant 7 holds throughout: the new column is a digest, the audit chain
hashes only fields this table already stored, and listChecks returns rows
whose only free-text field is validated against /^[A-Z][A-Z0-9_]{2,47}$/ at
the door.
0.6.68 — 2026-08-31
The CLI says which plan you are on, and how to fix it when it is wrong.
A Team-plan sign-in printed byte-identical output to a free one. whoami
held the whole teams array and dropped it in text mode, printing raw
feature flags — ci, pr_gate, org_policy, … — under a label reading
Plan:. The only string in the product containing the words "Team plan" was
a 403 failure message. A paying member could work for weeks with no evidence
from the terminal that the plan existed.
Worse, entitlements refresh only on login, by design, with a seven-day cache.
So when an owner bought a plan, every teammate's machine went on behaving as
free until they happened to sign in again — and nothing told them, because
whoami reported Plan: free without naming a remedy.
loginnow names your teams, your role and your plan, and prints the
console URL. The claim was already fetched and written seconds earlier;
this reads it rather than adding a network call.
whoamiderivesPlan:fromteam.plan— a field parsed since the claim
shipped and never once consumed — and lists every team.
- `relayevals login --refresh` re-reads the claim with the stored token:
no device flow, no browser. This is the remedy for a stale free claim after
someone buys, and it is deliberately explicit rather than automatic — a
claim that refreshed itself would be a licence check on a schedule, and no
verdict may ever depend on one.
/team(alias/account) answers the same question inside a session.logoutnow says what it is about to cost: a machine on a team plan loses
teammate-receipt verification, which otherwise resurfaces later as an
unexplained PRODUCER_KEY_UNRESOLVED.
The fence holds: refresh-entitlement-cache.ts reaches the network and is
unreachable from all eight verdict entry points, proved by the import-graph
walk in tests/drive-fence.test.ts.
Alongside this, the web console — which was fully built and almost perfectly
orphaned — became reachable: the site links to it, the billing links carry
the team id they always required, the minute after paying no longer says
"Free plan", and the analytics the pricing page sells now has a page.
0.6.67 — 2026-08-30
A drive measures what the agent wrote, and what that work reaches.
On a tree with your own uncommitted work in it, a drive used to hand the task
every changed line in the repository. A Stripe task counted twenty of the
founder's own files as evidence for itself. This release scopes drive to the
task — and verdict and try deliberately keep measuring the whole tree,
because there the uncommitted change in front of you IS the work.
Scope follows authorship, not the clock
The obvious rule — "what changed since the task was bound" — is unsafe, and 51
assertions across nine suites said so. It stops verifying work you did BEFORE
typing the task: write half a feature, ask an agent to finish it, and only the
agent's half is measured while yours ships unverified. A PASS that covers less
than the reader thinks is the failure this product exists to prevent.
So a pre-existing changed file belongs to the task when the agent's own work
reaches it. The agent's new test imports the module you were editing; that
is a real edge, found with the same basename search the repo map already uses,
so it needs no index and no graph. A dirty file nothing the agent wrote refers
to stays out, which is the whole point.
The reference test is deliberately generous — a near-miss keeps a file in
scope. Every ambiguity resolves towards measuring more, because dropping a file
is the only direction that can turn a BLOCK into a PASS.
Which command is asking decides it
drive already told the loop it was driving; nothing consulted that fact.
taskScoped now travels with the call rather than living in an environment
variable, and next hands the same decision to the verdict it invokes — a
next that measured the task while its own verdict measured the tree would
have the loop's two answers disagree about what was done.
verdict, try and a bare next measure the whole tree, exactly as before.
So does a drive with no baseline recorded, a task that is not the bound one, or
any turn before an agent has authored anything.
Two corrections found by tests, not by reasoning
Filtering the tree dropped work the agent committed mid-run — clean in
status, identical to HEAD, invisible to both probes. A drive that commits its
own work would have been measured against nothing. Surfaces and changed lines
are now a union of both diffs: a line is demanded if either found it.
And narrowing before any authorship exists reported `No code changes to verify
yet` over a worktree full of changes, which looped every failover scenario. A
crashed agent authors nothing. Scope narrows only once there is authorship to
narrow to.
Two holes an independent audit found before this shipped
Both were BLOCK turning into PASS — the drive demanding less than
verdict would on the same tree — and both were reproduced before being
believed.
git diff never reports an untracked file, so the HEAD-side pass contributes
nothing for one and an all-lines fallback is what covers it. The baseline union
broke that by supplying a partial entry — the agent's lines alone — which then
won over the fallback. **A 300-line untracked file the agent appended five
lines to had 5 lines demanded by drive and 305 by verdict.** The union no
longer outranks the fallback for a file git could not diff.
And the authorship gate armed on the raw name-only diff, taken *before*
.relayignore, lockfiles, bytecode and Relay's own metadata are excluded. A
turn whose only change was package-lock.json — or whose only change was a
deletion — armed the filter with something that can never be measured and
dropped the entire dirty tree, reporting No code changes to verify yet over
it. Narrowing now requires authorship that could be a surface and has content
to reach with.
Known, and not fixed here
The reference edge is textual. A module reached only through a barrel file, a
dynamic import, or a name shorter than three characters is kept in scope rather
than resolved precisely — the generous direction, but not a real dependency
graph.
0.6.66 — 2026-08-30
Test infrastructure only. Nothing in the CLI changed.
No file under src/ differs from 0.6.65 — the second release running. If you
are on 0.6.65 there is no reason to take this one.
The suite stopped filling the disk
A run failed with ENOSPC: no space left on device, raised from inside an
unrelated assertion in a receipts test. The disk was at 99% and 9,132 relay-*
scratch directories had accumulated. That is the worst way for this to fail:
the symptom names the wrong culprit, so it reads as a flaky test rather than a
full disk.
| | |
|---|---|
| leaked per full run | 158 directories |
| average size | 278 KB |
| cost per run | ~43 MB |
| accumulated before the sweep | 9,132 dirs · 2.6 GB · about 58 runs |
The per-run cost was never the danger. Nothing ever removed them was.
It is systematic rather than crash residue: of 152 test files that call
mkdtempSync, 40 register no cleanup hook at all, and the four largest
offenders have none between them.
What does not work, measured before relying on it: wrapping mkdtempSync
in a setup file and deleting what it hands out. The tests use
import { mkdtempSync } from "node:fs", and a named import binds the original
function — patching fs.mkdtempSync afterwards never reaches them. It would
have shipped as a fix that changed nothing while the suite stayed green.
So the sweep is age-based and runs before the suite, not after it. Deleting
"everything this run created" at teardown would delete a concurrent run's live
directories; an age threshold cannot, because anything a live suite is using
was created minutes ago rather than hours.
It bounds accumulation to roughly one run's worth rather than preventing the
leak, and says so in its own header. No per-file fix covers the other half
anyway: a test that times out never reaches its own afterEach.
This ships inside the tarball, so the end-to-end sandbox gate — which runs the
whole suite from that tarball — stops accumulating too.
0.6.65 — 2026-08-30
Release tooling only. Nothing in the CLI changed.
No file under src/ differs from 0.6.64. If you are running 0.6.64, this
version does exactly what yours already does, and there is no reason to
upgrade for it. It is cut so the fix below is in a tagged release rather than
loose on main.
The publish token was never stale — the publisher was holding an old copy
Two releases shipped their web surface 21 objects short, every one of them a
bracket-key route (blog/[slug], docs/states/[code], the team pages) coming
back HTTP 401. It read as an expired credential, and it was not.
Measured before changing anything:
| | |
|---|---|
| the on-disk token, against the raw API | 200 |
| token lifetime (expiration_time − the config's own mtime) | exactly 1 hour |
| a full publish, 191 objects | 10–20 minutes |
publish() read the token once and reused it for the whole run. The
wrangler CLI calls interleaved through the same loop kept renewing the token on
disk as they went — so the file stayed healthy while the script's in-memory copy
rotted. Any run that began with a token older than about forty minutes watched
it expire mid-flight.
That accounts for every symptom: only bracket keys fail, because they are the
only ones using the captured token; wrangler's own uploads never fail, because
wrangler refreshes; and it struck two consecutive releases, because both were
long runs.
The token is read fresh per call now, and a 401 — the one status worth retrying
here — renews it and retries once. Re-running the publish that had just failed
took it from 21 object(s) did not land · exit 1 to 0 failures · exit 0.
The end-to-end sandbox gate has room for a slow day
verify0622_full died once on context deadline exceeded. The binding limit
was the sandbox's own lifetime, not the step's, so raising only the step would
have changed nothing. The first explanation — that the suite had outgrown the
budget — was wrong: a clean re-run finished the whole gate in 1137s, inside
both original budgets. The headroom stays as slow-day resilience, and the
comment in the gate now says that instead of the guess.
Known, and not fixed here
A full test run still leaks about 158 temporary directories and a gigabyte of
disk. It surfaces as ENOSPC inside an unrelated assertion, which reads as a
flaky test rather than a full disk.
0.6.64 — 2026-08-30
**A PASS Relay could not aim is not a PASS — and the ending tells you what
happened.**
The verdict says what it understood
A drive reported PASS — 730 / 730 changed line(s) executed and `done 1 of 1
planned step(s)` on a task it had already said it could not place:
```
INTENT build · TARGET no target located · low confidence
plan ▸ nothing located from your words
plan ▸ no plan came back — using Relay's own
```
The completion check was circular. fallbackMission gives its single subtask
the loop's own coverage measurement as its done-check, so "every planned step
passed" restated the verdict instead of testing it. With nothing located
either, no part of the run bore on the goal at all.
A run that located nothing and used its own fallback plan now ends
NEEDS CLARIFICATION, with the reason and a way forward. The verdict
artifact is untouched — the code evidence really is PASS and is still printed.
Only the claim that this finished the task is withdrawn. Both halves must be
absent before it fires, so a repository with no planner, or a plan drafted from
the repository's shape, is unaffected.
The ending is a story
Five stages, in the order they happened:
```
this run
1 UNDERSTAND no target located · low confidence
nothing in the repository matched "stripe", "gatway"
2 PLAN Relay's own fallback — not drawn from your words
3 BUILD claude · 1 authorship turn(s)
your tree already carried 3 changed file(s) when bound
4 VERIFY relay · 730 of 730 changed line(s) executed
5 GOAL 1 of 1 planned step(s) proven
```
BUILD names the agent and VERIFY names Relay, from the actor already recorded
on every transcript row. That the agent does not grade its own work is visible
in who appears where.
Four more things the screen was getting wrong
- A task-shaped PASS printed before the agent ran. The guard that catches
it was computed a hundred lines below the two lines that announced it.
- Your files, rendered as the agent's. A Stripe task drew
.gitignoreand
store.mjs; the correctly attributed change set was computed thirty lines
earlier and used for nothing.
- A resize left copies of the composer behind. Four resizes, five
composers marching down the screen. The dock cleared the rows it was about
to paint and never the rows it used to own.
- Four identical `cd` cards. Collapsed to one line with a count.
A receipt that can mean "this task"
captureTaskBaseline records the worktree as a git tree when a task is bound,
so git diff <baseline> <current> reports the agent's five lines in a file you
had also touched, not your three hundred. It writes through a redirected index:
your index, worktree and HEAD are untouched.
Opt-in, behind `RELAY_TASK_SCOPE=1`, and the reason is the finding. On by
default it broke 51 tests across nine suites, all of them right. "I wrote this,
verify it" is how Relay is most often used, and scoping the surface to what
changed after binding makes that change invisible — Relay then measures the
agent's test file alone and reports success. Narrowing a surface can only turn
BLOCK into PASS. Off is over-demanding, which is the safe direction.
Known, and not fixed here
A full test run leaks about 158 temporary directories and a gigabyte of disk.
It surfaces as ENOSPC inside an unrelated assertion, which reads as a flaky
test rather than a full disk.
0.6.63 — 2026-08-29
**The ending says what was done — and the caret, the transcript and the rail
all follow the column.**
The column, everywhere
Centring in 0.6.62 moved the content and left three things behind, each
visible on a wide terminal:
- The composer drew its box at the centred gutter and parked the caret at
a fixed margin, so it sat alone against the left edge, a hundred columns
from the box it belongs in.
- A drive runs as a child with piped output, so it could not see the terminal
at all: it rendered at 80 columns and never centred, while the composer —
drawn by the session, which can see it — sat centred. Two layouts on one
screen, from one missing variable.
- The rail was a prefix at column 0. That was right while the column
hugged the left edge and wrong the instant it centred: a lone vertical line
fifty columns from the content it borders. It sits four columns left of the
content now, which is what the design's border does.
Measured at 200 columns: rail at 46, transcript and composer boxes both at 50.
The ending says what was done
A drive on a tree that already carried the work correctly takes one agent turn
and passes — and looked exactly like a drive that did nothing. Every ending
now carries the plan's own state:
```
done 3 of 3 planned step(s)
☒ st-1 Implement and export applyDiscount(lines, percent)…
☒ st-2 Add node:test cases for applyDiscount: 0% equals…
☒ st-3 Run the whole suite and the check.mjs verifier
note your tree already carried 9 changed file(s) when this task was bound
— some of this may predate the run
```
`/follow` queues a recommendation
The verdict artifact's next action was printed and then left to be retyped by
hand. /follow makes it the next task, pre-filled and editable in the
composer, opening at the head of the line so it can be read before it runs.
Known, and not fixed here
On a tree with uncommitted work, the changed-surface set counts your files
alongside the agent's, so a PASS number spans both. The sentence stays true —
those lines are changed and are executed — but it is not scoped to the task.
Fixing it needs a bind-time baseline, which changes how coverage is computed;
it will get its own reproduction rather than a patch.
0.6.62 — 2026-08-29
One column, sized to your terminal — and the blocks the design actually draws.
It fits the window you have
Three different widths were hard-coded: the session at 74 columns, the drive
renderer at 72, and the composer at the full width of the window. On a
209-column terminal that drew a narrow strip of transcript beside a full-width
box, which is two layouts rather than one.
There is one column now. It follows the window and caps at 100 — the design
system's own session is 720px, about a hundred monospace columns, and prose
past that is harder to scan. The column is centred, as the design centres
it, so the transcript, the new-session screen and the composer share one axis
instead of hugging the left edge of a wide window.
Long lines no longer break the shape: a receipt command is 209 characters and
is clipped to the column. The exact command stays in the transcript.
Errors read like errors
A failing check used to dump node's module loader into the middle of a run —
a throw site, a message, and a dozen at … frames of internals. It collapses
to the message and a count:
```
Error [ERR_MODULE_NOT_FOUND]: Cannot find module '…/login.mjs'
11 stack frame(s) hidden
```
The repo map draws the surface it touched
It said INTENT test · TARGET no target located — and could never have said
more, because only a formatted string reached the renderer. Each target's
callers and tests have been measured since the map shipped; nothing
downstream could see them. Now:
```
cart.js (applyPercentageDiscount)
└─tested-by──▸ cart.test.js
```
Every edge is measured. A target with no callers and no tests shows neither
rather than an invented arrow.
The gate reads down its status column
Each check is one line — PASS eslint — 0 errors, 0 warnings — so the eye
runs down the statuses, which is what a fixed status column is for. A check
whose name and cause cannot both fit keeps them on two lines rather than
truncating both; neither half is ever half-said. Evidence gets three aligned
columns.
0.6.61 — 2026-08-29
The harness draws its whole sequence, and stops repeating itself.
A drive now renders contract → schedule → thinking → planning → agents →
reading → diff → bash → verification gate → evidence → repo map → approval →
recommendation, above the docked composer.
Three blocks that were always possible
- Schedule. Five real phases with human labels, and every outcome already
carried a measured duration. What cannot be drawn is the *forecast* a
mockup shows — minute budgets nobody measures. This shows the opposite
number: what each phase actually took. Phases the run has not reached stay
unpriced, because a drive cannot know how many more cycles a task needs.
- Diff.
observeTree()has measured the tree every turn for the lease
check and it never reached the screen. Per file, +n −m from git, clipped
to a hunk.
- Recommendation. Every verdict artifact carries
next_actions[]with a
real reason and usually a command. Nothing had read them since the
dashboard was removed.
And it stops repeating itself
Three blocks repainted on *events* rather than on *change*:
| | before | after |
|---|---|---|
| schedule | 11 draws in a 4-turn run | 2 — the shape, then the measurement |
| agent lanes | 15 draws in a 4-turn run | 0.69 per turn |
| recommendation | 3 cards, two already resolved | 1, beside the verdict |
Nine copies of a four-line card was over a tenth of a 311-line transcript.
The agent lanes were the worst: marked dirty once per turn and drawn once per
phase outcome, and an outcome fires for every phase.
Renderer tests asserted content — a card printed nine times passed all of
them. They now assert frequency too.
Not drawn, deliberately
A mockup's confidence 0.82 has no source: the artifact carries no score. A
bash block's stdout has none either — the mechanical port returns an exit
code and nothing else, so real output needs that port to capture it rather
than a renderer to invent it.
0.6.60 — 2026-08-29
A green verdict is not a finished goal, and a finished drive now exits.
The drive claimed goals it had not reached
Reproduced with a real agent and recorded in docs/REPRO-goal-vs-coverage.md:
a task naming five handlers got `PASS — 34 / 34 changed line(s) executed …
required checks passed`, exit 0, with the named file never opened and none of
the five handlers validating.
Coverage is computed over the diff, and work that was never done produces no
diff — so an agent satisfies the verdict by adding any well-tested new file,
and the code the task named stays invisible. Relay's own plan held five
subtasks with executable checks, one of them its own final proof, and four
were never run.
Before settling on green coverage, the drive now runs whatever of its own
plan is still unproven. Same task, same agent: 4 turns and 0 of 5 handlers
validating became 22 turns and 5 of 5, with every subtask gated.
INCOMPLETE is a new ending for a goal the drive could not reach. It never
exits 0, names what is outstanding, and gives the command to continue — with
the measured verdict printed beside it, because a verdict is not rewritten
because the goal was unfinished.
`relayevals drive` never exited
The dashboard's server.unref() covered the listening handle and not accepted
sockets, so once a browser attached, the drive's own event stream held the
process open. A run printed its verdict, its evidence and its summary, and
then hung forever. Live since the dashboard shipped, and in every 0.6.59
install.
The localhost dashboard is gone. It re-rendered what the terminal already
showed and charged every run for it — a browser launch, an HTTP server, a
repo-wide git ls-files, a shortstat per outcome and an artifact parse per
eval, none of which any verdict read. Measured: never exits in 90s became
exits in 63s, with no listening socket left behind.
The hosted dashboard on relayevals.com — team analytics, audit, CI rows — is
untouched.
The session is a screen
relayevals opens a new-session screen with the composer pinned to the bottom
of the window: wordmark, version, what the harness found on this machine, and
a box that stays put while the transcript scrolls above it. A scroll region
reserves the rows; readline could not do this, so the input is a raw-mode line
editor with word motion, history and bracketed paste.
A pipe, a dumb terminal or a window too short keeps the plain prompt — 0
escape bytes into a pipe, exit code unchanged.
The terminal draws more of the harness
Thinking, file reads and bash calls each become the block the design draws,
filled from real events. Three parts of the reference are deliberately absent
because nothing measures them: the schedule's minute budget, the
recommendation's confidence score, and a bash block's stdout — the vendor
stream emits a tool result only on error, so successful output never arrives.
A first cut of that block put Relay's own lifecycle line inside a frame
labelled as npm test output; tests now assert the agent's prose and Relay's
lines land outside it.
Onboarding is five steps, and sign-in is required
01 github · 02 detect · 03 agents · 04 models · 05 ready. "Stay local" is
gone: a failed sign-in offers Try again or Stop here, and nothing else.
Setup now needs the network and stops outside a git repository. Machines that
are already onboarded are unaffected — the verdict path reads no entitlements
and still works offline.
Language grammars ship at install
The seven tree-sitter grammars (5.2 MB) were downloaded the first time a drive
touched a .py, .go or .rs file — a network fetch in the middle of a run.
They are fetched during install now, best-effort: a failure can never fail the
install, names exactly which are missing, and RELAY_SKIP_GRAMMARS=1 opts
out. Relay needs no Python and no compiler; web-tree-sitter is pure wasm.
Fixed
- The session banner never closed. It drew a top rule, a bottom rule, a
left edge and no right edge — the first thing a customer saw was the one box
on screen that did not close, in 16-colour dim while every card below it was
a filled panel in the exact palette.
0.6.59 — 2026-08-29
`relayevals` is a session, and it looks like the harness it is.
The terminal is a session, not a command
Bare relayevals in a terminal opens a prompt that stays. Type the task;
Relay maps, plans, drives your agents, opens the live dashboard, proves the
result — then hands the prompt back for a follow-up. Slash commands
(/agents, /model, /connect, /dashboard, /quit) and bare
exit/quit. Ctrl-C stops the running task and returns to the prompt; at an
idle prompt it takes two to leave.
A task typed here spawns this same binary's own drive — one code path, no
fork of the loop, and a session crash cannot touch a running drive. Scripts
and CI are untouched: the session opens only when stdin and stdout are
terminals, so bare relayevals in a pipe still prints help with the exit
code it always had.
Typing during a run no longer echoes into the transcript or queues itself as
your next command: the session holds the terminal and forwards only Ctrl-C.
Pick the model when you pick the agent
connect asks per agent. Claude offers Fable 5 / Opus / Sonnet — exactly
the aliases its own --help names. codex and hermes have a measured
--model flag but a catalogue that only renders in a TUI, so they get
Default plus a typed id rather than an invented name that fails on the first
turn. /model changes it mid-session. Default stores nothing, so a chain
written before this release means what it always meant.
The onboarding finishes, in five visible steps
● 01 account ─ ● 02 detect ─ ○ 03 agents ─ ○ 04 repo ─ ○ 05 ready, with
FOUND / — detection showing every adapter and its version, the model
menu, the machine-key vouch, and a ready card. First run inside the session
runs it inline.
It draws the harness, in the harness's own palette
The terminal now renders the components the design system specifies —
contract card on a filled surface, agent lanes, gate rows with a fixed status
column, evidence with a bordered tag, letter-spaced eyebrows — in the spec's
exact hexes.
Earlier attempts reported that filled surfaces and those exact colours were
impossible in a terminal. They are not; that was 16-colour reasoning stated
as a property of the medium. Terminals speak 24-bit colour, and terminals
that cap at 256 get a computed nearest match rather than garbage.
What a terminal genuinely cannot copy is behaviour: the reference page
mutates its verification gate in place, and an append-only transcript has no
in-place. Findings stream as rows while they happen; the complete gate is
drawn once, at the verdict.
The dashboard speaks the same language
The drive dashboard was migrated in six audited phases: the spec palette
(the purple progress bar is gone), zero-radius hairline cards, dotted rail
tabs, gate rows, agent lanes, and an approval card. The pixel-art office
world stays — it is the one illustration inside a spec-compliant frame, and
scripts/dashboard-color-audit.py now allows exactly two vocabularies and
fails on anything else. Its first run caught three real violations.
/harness is a fourth screen: a byte-for-byte carbon copy of the reference
harness design, wired to the same live event stream. Panel and harness render
the same run in two registers.
Fixed
- The session's rail was silently killing the dashboard. Piping the
drive's output so it joins the rail made process.stdout.isTTY false, and
the dashboard was gated on exactly that — so from the moment the rail
shipped, no task typed into the session opened a dashboard at all. Both the
dashboard and the renderer now ride one explicit signal, and a test pins
the class rather than the instance.
- A READY finding rendered as `PASS`. The gate printed its colour lane's
word instead of the state's, so a run that had verified nothing displayed
PASS changed lines executed by signed tests. Colour is a category; the
word is the fact.
connectasked you to sign in with a key it had not created yet.- The trust prompt asked
[Y/n] [Y/n].
0.6.58 — 2026-08-28
Supersedes 0.6.57 within the hour.
0.6.57's acceptance gate ran the shipped tarball's own suite on a clean
Linux host and one test failed — ours, not the product's. The test glued
TMPDIR to a filename with no separator; macOS's TMPDIR ends with a
slash and Linux's is unset, so the path landed at the filesystem root.
Every runtime gate passed; the artifact's own suite did not, and an
artifact whose own suite is red does not stay the latest. Fixed with
mkdtempSync and cleanup. Nothing else changes.
0.6.57 — 2026-08-28
`relayevals` is a session now.
The terminal is a chat, like the ones your team already lives in
Bare relayevals in a terminal opens a session: a prompt that is always
there, the work flowing above it, a follow-up typed the moment the verdict
lands. Type the task; Relay maps, plans, drives your agents, opens the live
dashboard, and proves the result — then hands the prompt back.
```
● RELAY · your-repo
claude (fable)
claude (fable) · your-repo · / commands
──────────────────────────────────────────────
→ add a percentage discount to the cart, with tests
```
Slash commands: /agents, /model, /connect, /dashboard, /quit —
plus bare exit and quit, because people type them at every REPL ever
made. Ctrl-C stops the running task and returns to the prompt; at an idle
prompt one Ctrl-C warns and a second quits, so a finger trained by
interrupting tasks cannot end the session by habit.
A task typed here spawns this same binary's own drive with the terminal
attached. One code path — the renderer, the dashboard, the receipts, every
fence — zero forks of the loop, and a session crash can never touch a
running drive.
Scripts and CI are untouched: the session opens only when both stdin and
stdout are terminals. Bare relayevals in a pipe still prints help with the
same exit code it always had.
Pick the model when you pick the agent
connect now asks, per agent with a measured flag. Claude offers
Fable 5 / Opus / Sonnet — exactly the aliases its own --help names,
probed against 2.1.233. codex and hermes offer Default plus a typed id,
because their catalogues only render in a TUI and a menu that invents an id
is a broken first drive. /model changes it mid-session.
Default stores nothing, so every chain written before this release means
exactly what it meant. The flag is spliced into the vendor argv after the
subcommand — never appended after the prompt, because whether trailing flags
parse differs by vendor.
Measured end to end: /model → arrows → enter → a real drive whose Claude
init event reports claude-fable-5 — menu to model, every link measured.
0.6.56 — 2026-08-28
Relay said DONE about work it never did. That is fixed.
A PASS the drive did not earn
Type a task into drive while your tree has uncommitted work in it, and Relay
would map the repository, draft a plan, resolve, and print
```
PASS — 90 / 90 changed line(s) executed for "add team members add so they
can focus on the goal together"
```
in under a second, with turns 0 and raw reads 0. No agent was ever
dispatched. There was no team-member code anywhere in the repository. The 275
changed lines it measured were an earlier, unrelated piece of work that
happened to be sitting in the tree.
The verdict was true about the lines it measured and false about the task it
named, which is the one failure this product cannot have.
PASS is computed over changed lines against HEAD, so any uncommitted work
satisfies it, for any task string. The guard that catches "this task has
not been started" tested for a clean tree, and a clean tree is exactly the
case where the bug cannot happen. Every gate in the release ceremony drove a
clean repository, which is why this shipped in v0.6.51 and survived four
releases.
The drive now refuses to end on a PASS with no authorship behind it when the
tree already carried changes at bind time, and says whose they are:
```
build ▸ your tree already carried 11 changed file(s) when this task was
bound. They are not evidence for it — sending claude to author it.
```
Measured on the repository where it was found: the same command went from
turns 0 in 0.6s to 5 turns with 2 authoring turns in 149s, and the feature
it had claimed was finally written.
This is the same rule the loop already applied to failures. Failures present
before any agent ran are not the agent's doing; neither are changes.
What it does not do yet: with pre-existing work in your tree, a PASS still
measures your lines and the agent's together. Separating them means diffing
against a bind-time baseline rather than HEAD. Until then, the honest advice
is unchanged — commit or stash before you drive.
The gate that should have caught it
scripts/e2b/verify0628.py now drives a second task on the tree the first one
left dirty AND covered. It was verified in both directions on a clean Linux
host: red against the unfixed build, green against the fixed one. Its old
fixture left one UNTESTED change, so resolve answered UNRESOLVED and the loop
ran — a gate that could not fail on this.
0.6.55 — 2026-08-27
**The drive you can read afterwards, and the team tier that changes a
developer's day.**
The drive is one continuous transcript
The live panel repainted a full-screen frame on every event: one 56-second
run captured 18,100 frames of the same box. It answered "what is happening
now" and destroyed everything else — no scrollback, nothing to copy, no way
to see two turns at once. A transcript you can scroll IS the record of what
happened to your repository.
The drive now flows downward, append-only, with the room to be read: a blank
line before every section, the plan reprinted when a subtask changes state,
the agent's thinking kept and dimmed rather than dropped, findings and the
verdict given space instead of a border. The panel is still there behind
RELAY_DRIVE_PANEL=1.
CI output is unchanged. A non-TTY stderr still gets the plain renderer, so
nothing about a pipeline's log moves.
Five defects were found by running it against a live agent rather than by
reading it, because each is a property of the event stream and not of the
renderer: the map arrived before the goal and opened every run on a detail;
the planner's JSON mission document landed as one 2,000-character line; tools
were announced twice, so every file read printed as two; a subtask carrying
its whole done_check ran 411 characters over six lines; and a turn was
keyed by number alone, so everything the agent did during turn 1 printed
under a header reading "Turn 1 · Asking Relay what is next".
connect asked you to sign in with a key it had not made yet
Step 1 offered Relay sign-in. Step 4 created the repository key that sign-in
needs. Run connect outside a project and it asked for something that could
not exist yet, with nothing on screen saying so. The repository is now
prepared before step 1, and when there is none the step says why instead of
failing at it.
The trust prompt asked "[Y/n] [Y/n]"
confirm() appends the suffix and three callers brought their own. One of
them is the machine-key approval the entire trust model rests on. Nothing
looked wrong in review because each half was correct alone; it is only
visible by running the command and reading the screen.
Team tier: the org's standard steers every drive
A team policy can now state how the authoring loop must RUN — required agent
order, a max_turns ceiling, mandatory --red-first, --mutation-gate —
delivered inside the entitlement claim that was already signed and cached.
drive reads it while authoring and says which settings the org imposed.
It is never consulted on the verdict path, and the fences prove it: a verdict
that read a licence would be a verdict somebody could buy. max_turns is a
ceiling, so a smaller request is left alone.
Team tier: a lesson learned on one machine becomes the team's
relayevals memory promote selects lessons the local ledger has earned —
ones with a finding code, seen more than once, short enough to be read — and
writes them into RELAY.md as an ordinary working-tree change. You commit
it; teammates get it by git pull; their next drive reads it.
Nothing crosses our servers. Memory records carry file paths, and Invariant 7
says we never see them. The transport is your own git, and a human reviews
what the team's agents will believe.
Also
- The ledger now counts recurrence. A lesson arriving that it already holds
is the repository teaching the same thing twice, which is what separates a
lucky one-off from a property of the codebase.
RELAY.mdcomment blocks are skipped whole. The old check dropped only the
opening line, so every line inside a multi-line comment became a pinned
memory record in every teammate's prompt.
0.6.54 — 2026-08-26
Python works properly, and SWE-bench is reachable.
Relay was failing pytest users for obeying Relay
Point Relay at a pytest repository and it tells you to run
pytest --cov --cov-report=lcov:.... coverage.py writes its .coverage data
file into the repository root while that command runs, the snapshot sees the
tree change mid-command, and the verdict is BLOCK / RECEIPT_STALE. The
user is blamed for following the instruction Relay just gave them — on their
first coverage run, every time.
The default .relayignore had coverage/ but not .coverage. It now carries
.coverage, .coverage.* (parallel mode), __pycache__/ and *.py[cod].
Existing repositories can add those four lines; new ones get them.
A number that read as its own opposite
SURFACE_UNTESTED printed (1/9 lines) — uncovered over changed — directly
after the words "the coverage report shows their changed lines never ran". A
surface with eight of nine changed lines executed rendered as `(1/9
lines), which every reader takes for "1 of 9 executed". It now reads 1 of 9
changed lines never ran`. A verdict whose headline number can be read
backwards is worse than one that prints no number at all.
SWE-bench Lite, end to end, in a sandbox
scripts/e2b/swebench-lite.py runs a real SWE-bench Lite instance on a clean
Linux host. Measured on pallets__flask-4045, 42 seconds:
```
env ok installs at base_commit
red 2 failed FAIL_TO_PASS genuinely assert-fails before the fix
green 2 passed and passes after it
noreg 50 passed no PASS_TO_PASS regression
relay PASS 10 / 10 changed line(s) executed; signed evidence valid
```
It is not a leaderboard submission and says so in its own output: the
official evaluation builds a Docker image per instance, and this pins
dependencies by hand. The agent slot holds the instance's own gold patch, so
it measures harness correctness — the ceiling — never model performance.
The harness caught itself being vacuous first: an early run scored a red on an
ImportError, where the tests never ran and the bug was never reached. A red
that is only a collection error proves nothing, which is the exact shape Relay
exists to catch. It now separates collection-error from failed from passed.
Telemetry the modern flow actually sends
connect, start and drive recorded nothing. "installs 30d" counted only
the deprecated bare setup, and the funnel's verdict steps described the
manual loop nobody uses. All three now record, the consent notice still
strictly precedes any first transmission, and /admin explains that zeros
before this release are an instrumentation boundary rather than failed users.
0.6.53 — 2026-08-24
Any stack can reach a verdict now, and the field widens to nine agents.
The four-stack wall falls
A Ruby repository with one added function answered NO_TEST_RUNNER on the
morning of this release — a full stop with no road forward, and the same wall
for every Java, PHP, C++ and shell shop. Relay auto-detects four stacks and
that has not changed; what changed is that every other stack has a **stated
road**, committed and reviewed like every other rule:
```json
"test_command": ["bundle", "exec", "rspec"],
"coverage_evidence": "declared"
```
The loop gates on the stated command in any language. The receipt proves the
suite ran, and the PASS names what it did not measure, in the verdict line
itself: *"PASS with 1 criterion not enforced by your policy
(coverage_evidence:declared)"*. Under "measured" a stated command is
deliberately refused with the exact half to change — a receipt without a
coverage report would demand evidence no test could ever produce, the
unclearable-BLOCK shape measured on a config file the same morning.
Three things the build itself surfaced, kept because they are the point:
- The policy_id landmine. The id is sha256 over the assembled policy, so a
field that were always present would have re-identified every policy already
committed — POLICY_CHANGED in repositories nobody touched. test_command
is optional and enters the identity only when present on disk. Pinned by a
test.
- The live probe caught two defects before any user did. resolve-next has
two prescription sites and the first cut patched one, so agents saw a
literal <your test command> placeholder; and the stated receipt omitted
--surface flags, so in declared mode a green suite counted for nothing.
Both fixed, both lessons in comments.
- Relay's own invariants pushed back mid-build. Editing the policy blocked
at POLICY_CHANGED until confirmed, and policy accept refused the
non-interactive attempt — a rule change is a human's to approve, and the
ceremony had to say yes like anyone else.
The drive's shell allowlist threads the stated command too: an agent driving a
stack detection cannot name is allowed to run the suite the gate will judge it
by. Omitting that would have recreated the 0.6.45 defect — write a test,
forbidden to run it — for every non-detected stack.
Measured end to end on the stack that was refused that morning: a real drive
on a pure POSIX-sh repository — build-from-clean fired, the agent authored in
an isolated worktree, Relay ran the stated command itself and signed it.
PASS in 2 turns, 29.2s, 0 handoffs, and the suite the agent wrote passes
standalone.
Nine adapters
pi — measured against v0.84.3 installed from npm: --print, -p confirmed
against the real binary. prime-agent — a fork of pi (its packages are
@earendil-works/pi-*), binary name from its own installer; `npm install -g
prime-agent` returned 404 on release day, so it lights up only where their
installer has provisioned the binary. Both ship verified: false until a
driven turn is measured end to end, exactly like the registry's other
unproven entries.
Quickstart
/docs/start is four commands now — install, relayevals login,
relayevals connect, relayevals drive "<task>" — replacing the manual
try/setup/next walk that stopped being the short way two products ago.
0.6.52 — 2026-08-24
Relay decides preferences. It does not hand them to you.
A founder ran a six-subtask mission that stalled on st-4, whose done-check was
a test file encoding "premium minimal" as concrete design tokens nobody had
ever stated. The agent answered with prose for three turns and edited nothing.
The rethink then diagnosed all of it correctly — its own note reads *"the loop
is not failing on style decisions — it is failing before any edit lands"* —
and escalated with:
> Human decision needed on: the exact design tokens (scale ratio, spacing
> unit, single accent color/token name).
Nobody installs an orchestrator to be asked what spacing unit to use. In the
founder's words: *"Relay is not using any kind of brain to remove the friction.
It is adding the friction."* That was the correct diagnosis, and this release
is the fix.
The line this draws
A preference has no wrong answer, only an unmade one — design values,
naming, thresholds, which of two files is authoritative. Those are the
harness's to make. Ground truth is what a repository genuinely cannot
answer — missing credentials, an external system's state, contradictory human
instructions. Only ground truth may interrupt a person.
Four changes
- A new `decide` correction. The rethink can answer with concrete values
instead of a question. The decision is recorded to the lifecycle — a
reviewer can see what was chosen and overrule it — and leads the next
attempt exactly as a human's answer would. It feeds prompts only; the drive
fence still proves no verdict path can read it.
- An escalation must survive a challenge.
escalateis no longer obeyed
on sight. One more scoped call asks: is this ground truth, or a preference
you should decide? Only a second, confirmed escalation reaches a human.
The guarantee that a confirmed escalation buys no authorship attempt is
unchanged.
- The planner may not ship adjectives. If a task carries a subjective
quality (premium, minimal, clean, fast), the plan states what it means in
the subtask title — "type scale 1.25, spacing unit 8px, one accent". An
agent must never guess what an adjective means, and a done_check must never
test values no subtask states. That is the defect that produced st-4.
- A no-op turn is named, not tolerated. A turn that changed no files opens
its next prompt with it: prose is not work, nothing you say is read by
anyone, only the tree is measured — choose the unspecified value yourself
and record it in a comment, never stop to ask. Three silent turns previously
looked to the similarity ladder like "same approach" and rode toward the
ten-attempt ceiling.
Measured, in the founder's exact scenario
A fresh repository and `drive "a premium, minimal to-do CLI for founders who
need extreme focus: add, list, done"`, driving a real agent.
| | before | after |
|---|---|---|
| ending | NEEDS_HUMAN / FIX_LOOP asking for design tokens | PASS |
| turns | rode toward the ceiling | 4 (1.3× of the minimal 3) |
| lines executed | — | 255 / 255 |
| human interruptions | 1, and it was homework | 0 |
And the application it produced runs: add, list and done all work, the
status line reads 1 open · 1 done · 2 free, and its own suite reports 100%
coverage on store.js and tasks.js.
What this does not change. Ground truth still stops and asks — inventing a
missing credential would be worse than friction. The mutation gate is still
opt-in, the memory ablation is still a recorded null result, and 14 of the 22
dashboard liveness items remain unbuilt.
0.6.51 — 2026-08-24
Two founder recordings, two different reasons a drive produced nothing.
0.6.50 fixed a dashboard that could not paint. These are the two that made the
drive itself do no work — and neither had anything to do with fonts.
The git root was not the project — a blank dashboard with a new cause
The founder unzipped a project into ~/Movies/test-projects/founder-todo and
drove from it. That folder had no .git, so the root walk climbed until it
found one: an accidental, commitless repository at ~/Movies. The
changed-surface scan then ran git status --porcelain --untracked-files=all
across the whole media library — iMovie bundles, video, node_modules —
synchronously, with no timeout, sha256-hashing every untracked file it found.
All of it on the one thread the dashboard server lives on. The page could not
be answered for the length of the crawl: blank tab, spinner running, terminal
silent after dashboard ▸, indefinitely.
The terminal had said so, quietly: fatal: ambiguous argument 'HEAD' leaked
raw above the repository line during connect, naming the media folder. A commitless repo, and
the wrong one.
Three fixes:
- A root preflight (
src/infrastructure/git/drive-preflight.ts). Two
conditions knowable in milliseconds, checked before any heavy work.
NO_COMMITS: the repository has no commits, so every measurement would fail
identically and no agent turn can change that. UNBORN_BRANCH: the branch you
are on has no commits though the repository has history elsewhere — a
different sentence, because telling that person their repository is empty is
simply false. Each names the command that fixes it. Same design as the
pre-existing-red guard: zero agent turns, the verdict does not move, what
moves is who is asked and when. Measured at 0 seconds in the founder's
exact shape.
A third condition, UNTRACKED_PROJECT, only *speaks*: nothing in the driven
folder is tracked by the repository the walk found. The first draft made it
a stop, and the release audit proved that wrong — the signal is identical
for the founder's unzipped project and for packages/new-feature/ on the
day you create it, and the remedy it printed (git init) would have created
a nested repository inside the user's own. A guard that STOPS must be
certain. It says the thing and drives anyway.
- A ceiling on the scan. Every git call in
collect-changed-surfacesnow
carries timeout: 60_000, so no git subprocess can pin the event loop the
dashboard serves from. **The sha256 pass over untracked files is still
unbounded** — a pathological tree could stall there, and that is named here
rather than implied away.
The ceiling introduced a worse bug than it fixed, and the release audit
caught it before publication: execFileSync throws on timeout, the throw
was swallowed, and the scan returned an empty surface list — byte-for-byte
what a genuinely clean tree returns. Measured on one repository with one
receipt, changing only the git binary: real git said BLOCK, timed-out
git said PASS. The scan now reports incomplete as a fact rather than a
note (the verdict engine is a pure function of requirements and evidence and
never reads notes), and both consumers fail closed: relayevals next
answers UNRESOLVED, and the verdict refuses to be computed at all. Pinned by
tests/scan-fails-closed.test.ts, which reproduces the real asymmetry — the
scan has a ceiling and the git snapshot does not, so on a huge tree the
snapshot succeeds while the scan times out.
- The leaked `fatal:`. connect's branch probe ran
rev-parsewith
inherited stderr, so a handled condition printed as a crash in the middle of
the numbered steps. Piped now, like every other probe.
READY meant "not started", and the drive read it as "done"
A fresh repository — one commit, a readme — and
drive "Design a premium, minimal to-do app…". Four subtasks planned. The
drive ended at turn 0:
VERDICT ● READY — No code changes to verify yet.
turns 0 · relay mechanical 0
No agent ever received a subtask. READY is resolve's honest answer to a clean
tree, but with a task in hand and no authorship spent it means the opposite of
done: the person named work that does not exist yet, and making it exist is
the entire point of drive.
When the loop now resolves READY with no authorship spent, the chain alive and
budget left, the document is rewritten into a build order before dispatch —
state UNRESOLVED (nothing measured, because nothing exists), action
AUTHOR_CHANGE, and a BUILD-FROM-CLEAN prompt leading with the current
subtask. That rewrite matters because READY's own action reads *"Tell your
human… Then stop."* — right for relayevals next standing alone, and exactly
the wrong sentence to hand an authoring agent.
Bounded at two clean-tree turns: a no-op agent must not ping-pong
READY → author → READY forever, and the READY ending after two silent turns
shows both turns in the summary rather than hiding them. PASS is untouched —
it means measured evidence over real changes, which a turn-0 clean tree can
never be.
Measured in the founder's exact shape with a real agent: the build ▸ note
fired on turn 1, the agent authored an application from the empty tree
(src/, test/, package.json), and the rest of the loop then did its job
on the agent's broken tests — three identical attempts, root-cause demand,
NEEDS_HUMAN / FIX_LOOP at turn 8. That ending is the product working: the
work exists, its tests fail, and Relay says so instead of calling it done.
Before this release the same command produced nothing at all.
Three things the release audit caught in this work before it shipped. The
build-from-clean rewrite initially kept state: READY, so the terminal branch
two lines below re-caught it and nothing changed — zero agent turns, the exact
failure it was written to fix; the new tests caught that one. The audit found
two more. A build-from-clean turn where the agent wrote nothing had its
subtask marked DONE, because the loop gate counts READY as a pass — Relay
would have reported a to-do app built over an empty directory, and written it
to the durable mission ledger; the gate no longer runs on such a turn. And the
!authored guard the first commit message described was not in the code,
so a clean tree seen *after* real authorship would have handed the agent
"NOTHING EXISTS YET" at the customer's expense. Both are fixed and pinned.
Separately: an ending that needs a person no longer exits 0. A failed
build-from-clean turn left a clean tree, the re-resolve answered READY with
exit 0, and needs-human reached CI as success. needs-human,
chain-exhausted and parked now floor at 40.
0.6.50 — 2026-08-24
The dashboard was opening. It was waiting on Google Fonts.
The founder recorded a drive: dashboard ▸ http://127.0.0.1:53624 printed,
Brave opened it, and for the whole 45-second recording the tab showed a blank
white page with the stop button still lit — against a localhost server that
answers in one millisecond.
The page could not paint
Every screen opened its stylesheet with render-blocking @import rules —
JetBrains Mono from fonts.googleapis.com, Switzer from api.fontshare.com.
A CSS @import blocks *first paint* until it resolves. Brave's shields refuse
those hosts, so it never resolved and the browser painted nothing, forever.
The same failure hits uBlock, a pi-hole, a corporate proxy, an offline laptop
and a plane. Five imports across three pages, every one on the critical path
of a page whose entire premise is that it is local.
All five are gone. System fonts only; the dashboard now makes **no network
requests at all**. tests/dashboard-offline.test.ts fails the build on any
absolute URL in any dashboard page, requires a generic fallback on every
font-family, and parses every inline script — a stray backtick in a comment
terminated the page's template literal during this work, and only tsc caught
it.
Then: what the drive knows, that the page never showed
A five-agent audit of every event the drive emits against what the three
screens render found 73 gaps. The eight that break the experience are fixed.
- The drive ended and the page did not. The ending was inferred by
regexing English out of the outcome headline, so exactly three endings were
recognised: PASS, a hard BLOCK, an interrupt. READY, the turn ceiling, a
failed criterion, a refused mutation gate, chain-exhausted, an environment
stop and a park never set final — the lamp pulsed LIVE, the sprites kept
bobbing and the clock kept counting over a run that had finished ten minutes
earlier. The drive now broadcasts one end event carrying outcome, state,
code, reason, turns, elapsed and next commands; the page maps it in a table
and renders a HOW IT ENDED card. It is sticky, so a browser opened after the
run still gets it.
- The live feed kept seven lines, in a box that could not scroll. One
npm test streams every runner line through as activity, so a single verify
turn erased every tool call the agent had made. Now 500 lines, scrollable,
newest at the bottom, repeats collapsed to × N, rows updated in place
rather than the list rebuilt — which was re-firing the row animation on
every row and strobing the panel unreadable.
- The green lamp meant nothing.
onerrorrecoloured the text and left the
dot green and pulsing. The stream now sends an unref'd comment-frame
heartbeat every 10s; the lamp reports LIVE, DISCONNECTED or the final
verdict; the clock stops when the drive does; body.stalled freezes all
animation. Nothing moves unless something is happening.
- The mid-turn blackout. A tool call was announced only after its
arguments finished streaming, so a large Write or a Bash running the suite
meant tens of seconds of total silence — indistinguishable from a hang. The
call is now announced the instant it opens and filled in by vendor call id,
so one call is one row.
- Extended thinking was parsed by nothing. A model that thinks for a
minute before acting produced a minute of silence. It streams a line at a
time as ◌ … — the agent's own words, never summarised.
- The station was guessed from the text.
▸ Write src/bash-utils.ts
matched the bash regex and sent the agent to the terminal. tool and
target now travel as fields on the activity event and the page switches on
the tool name exactly, keeping the regex only for vendors that emit plain
lines.
- Paths were unreadable.
▸ Read …t2gmzqld30s21_ng_f51tqv80000gn/T/relay-wt-turn-2-6Qat15/tree/src/m.js
now reads ▸ Read src/m.js. A path too long to show keeps its tail —
cutting from the front leaves the reader with directories they already know
and no filename. A command keeps its head, because its head is the verb.
- `--verbose` blanked the dashboard. It printed raw lines and returned
before the parser ran, so the flag that shows you more showed you less and
the office sat idle through a working turn.
Also: FILES TOUCHED was scraped from every activity line including Relay's own
mechanical output, so it filled with package.json, coverage/lcov.info and
the receipt Relay had just written. Only a write counts as a touch now.
Not fixed, and named rather than buried: 14 of the audit's 22 ranked items
remain — a per-turn clock, the top bar's four wrong counters, live /map
updates, per-turn token cost, and the mission's gates on screen.
Documentation
docs/CONNECT-AND-DRIVE.md is new: what connect sets up, what
drive "<task>" does, what you hold at the end, and what it is worth. Every
claim was checked against the source by an adversarial pass — 939 claims, 849
confirmed, 72 overstated, 18 wrong — and the draft itself was attacked twice
more before it shipped.
0.6.49 — 2026-08-23
**Everything in this release came from one founder report: a one-line task, a
pnpm monorepo, 54 minutes, turn 4 of 40, 0%, and a popup that would not close.**
- The drive stops charging you for a repository that was already broken.
TESTS_FAILED is decided from the test command's exit code, so a monorepo
carrying 138 failing assertions that had nothing to do with the task charged
every one of them to the drive. The agent was told to "reproduce, fix, then
re-run" work it had not done and could not undo, and **the loop had no way to
end**. Relay's first receipt runs before any agent authors anything, so it
had always measured which failures pre-existed — and always thrown it away.
Now it says so and stops, in zero agent turns:
Every one of the 138 failing assertion(s) was ALREADY failing before this
drive started. Nothing this task did broke them, and no agent turn can
clear them.
The verdict does not move: a red suite is still not a PASS. What moves is who
gets asked. When only some failures are new, your agent is told which ones
are its own and told to leave the rest alone.
- The task map aims at the right file. "add the team invite feature where
customers can invie the team and work simeltaniesly" resolved to
packages/brain-ast/src/worker-pool.ts. Three causes: "can" was not a
stopword and matched the identifier part can for full marks; the ranking
SUMMED per file, so eight declarations brushing a common word beat two exact
matches; and the symbol matcher's scores were discarded for every symbol
git grep had already found — which was most of them, leaving the whole
layer dead. It now picks invites.ts (TeamInvite, createInvite) and shows
its reasoning: matched by: exact:team, exact:invite, prefix:invie→invite.
- You can close the popup. It appeared when Relay needed you and was hidden
by exactly one thing — a PASS that never comes on a run that stopped to ask.
Now: ✕, Escape, and it collapses to a chip that reopens it. The BLOCK stamp
underneath it is dismissable too.
- HISTORY shows your past runs. It never could: the event was built and
then never sent, because of an error that the surrounding code swallowed as
"no ledger". Past runs were also titled with the agent chain instead of what
they were for.
- MAP stops saying things that are not true. Your whole workspace was one
box reading packages/ 322; it now shows the modules you actually have.
"17 relationships" was the number of lines the page had drawn, not imports —
it says "links drawn". "8 modules" was a cap printed as a total; it says
"8 of 23". Cross-package imports appear. And MAP or HISTORY could render
empty through no fault of their own, because the events they need were the
first ones sent and were evicted from the replay buffer before you opened the
browser.
- "Bash failed" told you nothing. A refused command and a failed one
printed the same four words, and the reason was parsed and dropped. In a
monorepo the agent was being refused pnpm --filter, -w, -r and
pnpm exec — exactly the commands for narrowing a big failing suite. Those
are permitted now, and a refusal says REFUSED, with the reason.
Both invariants unchanged: no model output is ever an input to a verdict, and
Relay never says "safe to merge".
0.6.48 — 2026-08-23
Relay can finally tell whether a test of NEW code checks anything.
Last of the five cuts in plan8.
- The addition blind spot is closed. Red-first asks whether your tests
would fail without your change, and it asked that by putting the old source
back. For a change that MODIFIES behaviour that is the right question. For a
change that ADDS behaviour it is the wrong one: take the addition away and
the symbol goes with it, so any test that so much as imports it fails —
before a single assertion runs. Relay reported "your tests depend on this"
about tests that checked nothing, and has printed a caveat saying so on
every verdict line for four releases.
The counterfactual for an addition is not deletion, it is breakage. Your new
function keeps its name and signature and stops doing its work. A test that
asserts anything about it fails; a test that merely calls it does not, and
is correctly called out.
**Measured: 10 of 10 vacuous tests of added behaviour caught, 0 of 10
genuine tests wrongly accused, where the old check caught 0 of 10.** The
40-case corpus for modifications is unchanged at 20/20 with 0 false
accusations.
- The verdict line says which question was asked — `red-first genuine
(behaviour emptied) or (source reverted)` — instead of the caveat.
- A brand-new test file is now actually run. Red-first's counterfactual
did not carry untracked files, so a new test file was not there, the
command failed because the file did not exist, and Relay read that as "your
tests depend on the change". It was reporting a pass about a test it had
never run.
Said plainly: the mutation gate is still opt-in (--mutation-gate). Turning it
on by default was tried in this release and backed out — it stopped twelve
scenarios reaching a verdict and one ordinary drive ran past ten minutes
without finishing. It remains the check that catches a test which executes
your code without checking it, and it remains something you ask for.
Both invariants unchanged: no model output is ever an input to a verdict, and
Relay never says "safe to merge".
0.6.47 — 2026-08-23
**Relay remembers this repository — and the plan stops telling agents the
finding is someone else's job.**
Fourth of the five cuts in plan8.
- Memory that is worth reading.
relayevals driveused to carry twelve
lines of grep over past run outcomes: "a past run passed in 3 turns" is
true and tells an agent nothing it can act on. Every line now says who
wrote it — *your note* (you typed it, or committed a RELAY.md), *measured*
(detection: how tests run here, where they live), *learned here* (read off
Relay's own attempt ledger: which shape of attempt moved a finding in this
repository), or *distilled* (a model's summary — and every line of one must
cite a path, code or agent the run actually contains, or it is dropped
before it is written).
- `relayevals remember "…"` pins something you know, and it leads every
agent prompt from there on. A committed RELAY.md does the same, and
travels with the repository so your team reviews it like code.
relayevals remember --list shows what Relay holds.
- The finding governs. Given a plan that split a task into a source
subtask and a test subtask, an agent wrote the source and stopped — in its
own words, "I did not touch test/math.test.js — Relay listed the test case
as a separate subtask". It had satisfied its subtask and left the finding
standing, costing a full turn. The prompt was giving it two authorities and
no precedence. Relay only ever measures the finding, so the finding is what
"done" means, and the prompt now says so. **Measured: a planned drive of
that shape went from two agent turns to one.**
- Agents are no longer pointed at Relay's own output. The task map was
matching coverage/lcov.info as a place your task lives.
Said plainly, because it is in the record either way: memory is NOT proven to
save turns. The ablation that would have proved it returned a null — both
arms cleared the task in one turn, because the three releases before this one
had already removed the costs memory would have paid for. The harness ships
with the release so the question can be re-asked on a repository with real
headroom.
Both invariants unchanged: no model output is ever an input to a verdict, and
Relay never says "safe to merge". The drive fence now covers the memory
ledger, so "a model's summary of a past run can never reach a verdict" is a
structural proof rather than a promise.
0.6.46 — 2026-08-23
A turn that does not land leaves your working tree exactly as it was.
Third of the five cuts in plan8. Until now your agent edited your repository
directly, so a crash left its half-written files behind — for the next turn to
measure, and for you to find in your editor. Work nobody finished and Relay
never measured.
- Every attempt runs in its own git worktree, seeded from your current
state: HEAD plus everything uncommitted, tracked and untracked, so the agent
sees exactly what it would have seen working in place. A turn that finishes
cleanly is adopted. A turn that crashes, times out or is interrupted is
discarded, and your tree is byte-for-byte what it was.
--no-worktree restores the old behaviour if you want it. Adoption is
fenced: your own diff is captured before it and re-applied if applying
fails, so the worst case is the tree you had a moment ago.
- A file the work CREATED now lands.
git diff HEADreports tracked
changes only, so adopting by patch alone silently dropped every new file —
and "write the test in a new file" is the most ordinary shape of work there
is. The same hole had been under racing since the day it shipped.
- Proved on a clean Linux host, not just claimed. A new acceptance suite
runs the whole thing in a sandbox: a crashed agent's junk never reaches the
tree, your own uncommitted work survives it, a clean turn lands including
the file it created, no worktree is left behind, and opting out measurably
costs you all of it.
What this does NOT do, said plainly: Relay's own measurement still runs in
your working tree, not a container. Your agent is isolated; the receipt run is
not. That is also why a receipt can still say "mutated during run" — Relay's
coverage report lands in the tree while the command runs.
Both invariants unchanged: no model output is ever an input to a verdict, and
Relay never says "safe to merge".
0.6.45 — 2026-08-22
Your agent stops coding blind.
Second of the five cuts in plan8. Until now a driven agent could edit files
and run nothing. Measured on claude 2.1.233, in a throwaway repo, asked to
run node --test and report what happened:
--permission-mode acceptEdits alone → "RESULT: not-permitted"
+ --allowedTools "Bash(node --test:*)" → "RESULT: pass"
+ --disallowedTools "Bash(relayevals:*)" → "RESULT: not-permitted"
The first line was the defect: your agent wrote a test, could not execute it,
guessed, and ended its turn — and Relay had to tell it, a turn later, whether
the guess held.
- It can run your test command now. The drive derives an allowlist from
what detection already measured — your repository's own test command first
(npm test, pytest, go test, cargo test), then the stack's runners,
read-only git, and reading files — and renders it into the agent's own
permission system. Planning and diagnosis turns get nothing at all: they
are told not to edit, and they run no commands.
- `relayevals` is refused, and so are installs and the network. Not asked
for politely in a prompt — refused by the vendor's permission system.
- And refusal is not the only line of defence. Relay hashes everything it
owns — receipts, verdicts, the attempt ledger, relay.policy.json and its
baseline — before each agent turn and again after it. A turn that wrote to
one is voided: the files are restored, the attempt is not counted, you are
told, and the agent's next prompt opens with why. An agent cannot produce
evidence, because producing evidence unwrites the turn.
The distinction the whole change rests on: an agent running your tests to
SEE is not an agent producing EVIDENCE. Relay re-runs the measurement in
its own sandbox and signs that. Nothing the agent prints reaches a verdict.
- A failing test now says where and why. The prompt carries
file:line
and the assertion message, not only the test's name.
Both invariants unchanged: no model output is ever an input to a verdict, and
Relay never says "safe to merge".
0.6.44 — 2026-08-22
The prompt stops contradicting itself, and the map starts finding things.
First of the five cuts in plan8, which sets a measured gate on every axis of
the harness. Two gates in this one, both measured on the published binary.
- The contradiction is gone. A driven prompt said "you may not be
permitted to run relayevals" two screens above "Produce evidence:
relayevals receipt run … Continue the loop: relayevals next". An agent
reading that either burned a turn getting denied or stopped to ask its
human for permission. Inside a drive Relay runs both itself, so the
finding it hands an agent no longer tells the agent to: the relayevals
steps are dropped and the brief reads "Relay produces it after each of
your turns". The standalone prompt you can copy and paste is unchanged —
a person running it still needs the loop re-entry.
- The task map finds what you meant. "add a subtract function" used to
locate nothing in a file whose export is sub. Every declaration in the
repository is now scored against your words by named rules — exact, stem
(sorting/sort), part (verifyPaymentAuth → verify, payment, auth), learned
(the repository's own text pairs the short and long form), prefix (sub →
subtract) — and the artifact prints which rule fired: `matched by:
prefix:sub→subtract`. Deterministic, no model, and you can check the
reading. **Measured: 40 of 40 labeled tasks put the true file in the top
three (gate: 90%).**
- The prompt carries the code, not the file name. The numbered lines
around the matched symbol, and the head of the test file that shows how
this repository writes tests. The context budget goes from 14,000 to
40,000 characters, and when it is exceeded whole low-priority sections
are dropped rather than the text being cut mid-sentence.
- A task that maps to nothing is still planned. The planner now gets the
repository's measured shape — modules by file count, the test files that
exist, the detected test command — and its draft is validated exactly as
any other. Only an empty repository skips planning.
- The verdict names every check it ran, and every one it did not.
`checked coverage 1/1 · red-first genuine (caveat: pure addition — the
counterfactual removes the symbol, so this proves the test imports it, not
what it asserts) · mutation skipped (off — --mutation-gate)`. A PASS never
reads as more than it measured.
- The attempts ledger names your agent's files. It listed everything the
working tree held, including files Relay itself wrote — coverage output,
.relay/, the policy. It now attributes by difference across the agent's
own turn.
Both invariants unchanged: no model output is ever an input to a verdict, and
Relay never says "safe to merge".
0.6.43 — 2026-08-22
The loop remembers.
The founder's words: "it doesn't have the memory of project and context of
the goal — how does it know what to do? it retrying some shit and asks user
for everything." He was right. Every authorship turn is a fresh agent
process; turn one got the context artifact and every later turn got a
two-line header plus the same coverage complaint. Nothing told the agent
what it had already tried or why the measurement did not move, so it tried
the same idea again — and the rethink that should have fired was wired to
the wrong counter.
- Every turn gets the same prompt, in this order: MEMORY (what Relay has
recorded about this repository — who resolved or stalled here, the last
similar runs and how they ended, the plans they used; read from Relay's
ledgers, never written by a model) → ATTEMPTS SO FAR (who, which files,
+n −m, and the finding afterwards — with "the measurement did not move"
said in those words when it did not) → the task and subtask → the context
rebuilt for that turn at full budget, so the diff it carries is the
record of what was tried, with how this repository writes its tests → the
finding.
- A stall is rethought before it is handed to you. When the same finding
survives repeated attempts, a corrective rethink runs first — on plain
drives too — and its answer buys one more attempt with the new approach
leading. Then the consultant, then the next agent, and only then a person,
with the ledger in the ask. Spent once per finding; no limit was raised.
What still comes to you at once, correctly: trust, policy and platform — a
machine cannot vouch for itself.
Both invariants unchanged: no model output is ever an input to a verdict, and
Relay never says "safe to merge".
0.6.42 — 2026-08-22
The dashboard is the Control Panel · Live Office World — all three screens.
0.6.41 opened a live page with relayevals drive. It was a panel of cards in
the design's colours. The founder's instruction was "it should be exactly
like this", with the design attached. This release is that design.
- PANEL — the pixel office, ported verbatim: every station, the sprite
maps, the palette, the keyframes, the swarm, the inspector, the feed and
SYSTEM panel. Each sprite is an actor the drive actually has — RELAY the
boss, each connected agent under its real name, BYTE the verifier, ORBIT the
planner only when a mission was planned — and every move, bubble, badge,
check and feed line derives from an event the drive emitted. The design's
scripted simulation does not ship; a test asserts it.
- MAP — the Codebase screen, bound to the drive's own task map: the files
the task's words resolve to, the tests that reference them, the callers the
graph knows, the changed surfaces, and per-file verified / partial /
unverified read from the verdict artifact. The footer counts are measured,
not the design's 1,842.
- HISTORY — the Results screen, whose goal folders are the repository's
real runs from .relay/lifecycle.jsonl, with the current drive first and
filling in live. WHY is answered the way Relay answers it; where the ledger
has nothing the page says so.
- `message` and `interrupt` are real. They POST to
/intervene, which
enters the drive through the same queue as the i and s keys — one
queue, two doors. A note leads the next agent turn, attributed to you; a
stop ends the drive cleanly. Bounded to 4KB and two shapes; never touches
evidence.
- The three-part ask — what happened, what you need to do, the command —
appears as a paper note over the office when Relay needs you, while RELAY
walks to the HUMAN desk.
Still local only (127.0.0.1), still terminal-first (no TTY, no dashboard),
still inert (closing the page changes nothing, the server never holds a
finished drive open). Off with --no-dashboard or RELAY_NO_DASHBOARD=1.
Both invariants unchanged: no model output is ever an input to a verdict, and
Relay never says "safe to merge".
0.6.41 — 2026-08-22
Relay now speaks to you, and you can watch it work.
Both changes answer one complaint, in the founder's words: "when you have
human needed, tell them exactly what they need to provide — even I cannot
understand." He was right. relayevals next is the agent interface, and the
drive was printing the agent's instructions straight at the person — "Stop.
Tell your human to run…" — a message addressed to someone else, about you.
- Escalations are written for the person reading them. Every state that
needs a human now renders in three parts, printed up front: what happened,
in plain words; what you need to do, in second person; and the exact
command on its own line, copyable. An audit found 7 of 11 human-owned
states told you what to do but not how — all 13 now have a full ask, and
the test suite tests the words: nothing may say "tell your human", and
every command must be bare enough to paste.
- `relayevals drive` opens a live dashboard. A localhost page that
live-syncs the run: goal, pipeline, per-actor activity, evidence with root
causes — and when Relay needs you, the same three-part ask takes over the
top of the page. Local by construction (127.0.0.1 only; the stream carries
your file names), terminal-first (a drive in CI or a pipe behaves exactly
as before), and inert (closing the page changes nothing; the server can
never hold a finished drive open or touch a verdict). Off with
--no-dashboard or RELAY_NO_DASHBOARD=1.
There is still no dashboard to log into — no account, no cloud. This is a
window onto a run that is already yours, on your machine.
Both invariants unchanged: no model output is ever an input to a verdict, and
Relay never says "safe to merge".
0.6.40 — 2026-08-22
A gate that asks whether your tests would notice the change being wrong.
Relay's PASS has always meant your changed lines executed under signed evidence.
Measuring what that stopping condition does to an agent produced an
uncomfortable result: across 72 live runs, a drive that stopped at coverage
verified less than the same agent given the same context and left alone —
64% against 86% on an independent mutation score, at three times the wall clock.
The cause was not subtle. 23 of 24 runs ended in PASS rather than exhaustion, at
a median of 7 turns out of 16. Coverage is satisfiable before the work is
verified, so the agent stopped there. A gate weaker than the goal made the work
weaker than it would have been unsupervised.
- `relayevals drive --mutation-gate`. After the loop reaches PASS, Relay
breaks your changed behaviour in small, specific ways and re-runs your tests.
A suite that stays green did not check that behaviour. When something goes
unnoticed the gate does not merely refuse — it hands the agent the surviving
behaviours and sends the loop back to work, bounded by
--mutation-attempts (default 3).
Measured on the same corpus: 87% against 86% for the unconstrained agent.
The penalty is gone. It is a tie, not a win.
- It names behaviours, not lines. "An inclusive bound became exclusive and
every test still passed" is a work order. "Line 42 is uncovered" is not — and
that difference is the whole finding.
- A threshold, not perfection. Semantic equivalence is undecidable, so some
mutants cannot be killed by any test and a gate demanding all of them cannot
be passed. The bar is 80%, and it is a stated choice rather than a discovery.
OPT-IN, deliberately. It costs one full test run per mutant — 215s median
against 61s ungated — and nothing about your verdict changes. Relay still
reports exactly what it measured; only the drive's claim of doneness is
withheld.
Also: relay-evals/verify-action@v1 is published, so the CI gate is one
line. Red-first's limit is now documented where the rule lives: it discriminates
on changes that MODIFY behaviour, and is inert on ones that ADD it, because
reverting an addition removes the symbol and any test that imports it fails.
That is what the mutation gate is for.
Both invariants unchanged: no model output is ever an input to a verdict, and
Relay never says "safe to merge".
0.6.39 — 2026-08-21
The vacuous-test check now runs where merge decisions are made.
0.6.38 shipped red-first as a drive gate and nothing else. If you review
changes on a pull request — which is where most people meet Relay — you got a
PASS with no vacuous-test check at all. The strongest thing Relay measures was
missing from the surface you actually look at.
- `relayevals red-first`. Rebuilds your repository as it was before the
branch, keeps your new tests, and runs them. Tests that genuinely cover a
change *fail* there. Tests that pass there proved nothing.
relayevals red-first --base origin/main --format json
Exits 0 for genuine, 21 for vacuous, 0 for inconclusive. Pass the
report to ci report --red-first-file <path> and a VACUOUS_TESTS finding
is attached to the check.
- It needs history. "Before the branch" is the merge base with your target
branch, so fetch-depth: 0 on actions/checkout is required. Without it
the check reports inconclusive and tells you why. It does not fall back to
HEAD — on a branch whose change is already committed, HEAD *contains*
the change, so that counterfactual would be identical to the change and
every honest test would be called vacuous. A check that guesses confidently
is worse than one that declines.
- Advisory, deliberately. Red-first runs a fresh counterfactual, and an
execution is not a signed receipt — so it never moves a verdict. It reports;
your workflow decides. In the action, enforcement is off by default
(fail-on-vacuous-tests: false) so you can watch it before it can block
anything.
- Go tests are targeted properly. The previous command was
go test ./... — the whole module, so any unrelated failure cleared a test
red-first never actually interrogated. It now runs only the packages your
changed tests live in.
Both invariants unchanged: no model output is ever an input to a verdict, and
Relay never says "safe to merge".
0.6.38 — 2026-08-21
A PASS now means something checked what those lines did.
Relay has always proved that your changed lines executed under signed
evidence. It never proved those tests *asserted* anything — a test that runs
a line without checking its result satisfied the gate completely:
test("classify", () => { classify(9); }); // covered. proves nothing.
That was the largest hole under the central claim, and the one an agent
optimising for a green gate finds first. This release closes it.
- Red-first. After the loop reaches PASS, Relay puts your changed tests
against the OLD source in an isolated worktree. If they still pass, they
never depended on the change — so the coverage they produced was execution
without verification, and the drive declines to call the task done. Your
verdict is unaffected and is reported exactly as measured; this decides
only whether the DRIVE claims success. Turn it off with
--no-red-first.
Measured against a 40-case corpus of fix-shaped work — ten ordinary
defects, each with two genuine test shapes and two vacuous ones:
20/20 vacuous flagged, 0/20 honest tests falsely accused.
It stays silent rather than guessing. A test runner it cannot target
precisely, a run killed by a signal, a change with no test or no source —
each reports inconclusive. Telling you a test proves nothing, on the
strength of a run that did not happen, would be worse than saying nothing.
What it still does not prove. That your assertions are *right*. A test
asserting the wrong expected value still fails without the change, and is
correctly scored genuine — red-first measures whether a test depends on your
change, not whether it describes the change correctly. Mutation testing is
the check for that, and it is not shipped.
Both invariants unchanged: no model output is ever an input to a verdict,
and Relay never says "safe to merge".
0.6.37 — 2026-08-21
So the tarball passes its own suite on a machine that is not ours.
No product behaviour changes. 0.6.36's binary was correct; its bundled test
suite was not, and Relay's pitch is that you can check it yourself.
- A test in the shipped tarball failed on a clean machine. It asserted
that the contradiction rule stays quiet when someone genuinely fixes code —
which it does, proven on Linux — but it also assumed a machine whose signing
key was already approved, and a coverage instrument that only exists in a
store built up over time. On a fresh box it hit TRUST_SETUP_REQUIRED
instead. The product was right in both cases: an agent must not approve its
own machine, and a suite that never executes your code cannot earn a PASS.
The test now checks the thing it is named for and brings its own approved
key.
- The release checklist names the variable its acceptance script needs.
verify0622_full.py uses RELAY_VERSION to find the install and download
the tarball, not merely to assert a version — so run without it, the script
reports the RELEASE as broken when the stale thing is the expectation.
Both invariants unchanged: no model output is ever an input to a verdict,
and Relay never says "safe to merge".
0.6.36 — 2026-08-21
The refusal reaches the check that gates the merge.
- The contradiction rule now guards the signed artifact and `ci report`.
0.6.35 shipped it into relayevals next alone, so the agent loop refused a
laundered receipt while the artifact a required CI check consumes still came
back green — the loop protected the agent and left the merge gate open. The
rule is now applied where the verdict's other pre-decision facts are
assembled, as an UNRESOLVED requirement, so the decision function itself
stays pure: an unresolved requirement can only ever prevent a PASS, never
manufacture one. Measured across all three surfaces, 60 trials: 0/20
laundering attempts reached PASS and 20/20 were caught on next, on the
verdict artifact, and in the ci report payload; 0/40 false positives on
every surface.
- A failing check now says why.
ci reportused to send policy-drift
codes or nothing at all, so an UNRESOLVED check reached a team with no
reason attached. It now forwards the artifact's own codes. Path-keyed
findings are filtered out deliberately — a repository's file names are not
something to hand a server, and the API's schema would reject them anyway.
- The lease is enforced against a test that can fail. Each subtask may
only change the files its plan named. That claim had no end-to-end test:
deleting the refusal broke nothing in the suite. It now has one, verified
by deleting the refusal and watching it fail.
- A subtask can no longer escape its lease by CREATING a file. Scope was
measured with git diff HEAD, which reports nothing for a file git is not
tracking — so a new out-of-scope file passed the check on an empty result.
Creating files is precisely how an agent drifts. Scope now also compares
the set of files present, captured when the subtask starts, so your own
untracked work is never counted against the agent.
- The ladder's two numbers are named separately. A root cause is demanded
at the second identical failure; the run escalates at the third. Both
always shipped and every user-facing document already said so; the internal
shorthand named one event while measuring the other. No behaviour changed.
Both invariants unchanged: no model output is ever an input to a verdict,
and Relay never says "safe to merge".
0.6.35 — 2026-08-20
A suite that answers differently for the same code has not answered.
- Evidence contradiction. Receipt discovery keeps the newest receipt per
kind, so an agent that re-ran a flaky suite until it passed handed Relay a
green receipt and Relay never learned a red one existed for the same code.
That is the inverse of the failure people expect: the worry is a stale red
sticking, and the real hole was a red being laundered into a PASS by
rerunning. Two receipts covering the same code — same commit, same surface
hashes — that disagree are now UNRESOLVED · EVIDENCE_CONTRADICTION, never
a softer path to PASS. Measured over 60 trials against the built CLI:
0/20 laundering attempts reached PASS, 20/20 were caught, and 0/40
deterministic runs were false-flagged — including 20 legitimate
red-then-fix-then-green runs, which are not contradictions because a real
fix changes the surface hashes.
Scope, stated plainly: this guards relayevals next. The signed
verdict artifact and relayevals ci report do not consult it yet, so a
laundered receipt can still produce a green required check. Routing it
there is the next release's work.
- `drive --race <n>`. Race n attempts at one subtask, each in its own git
worktree, and keep one. Measurement decides: a branch whose check fails is
out, the smallest change that passed the SAME check wins, and the
repository's ordinary gate still re-runs before the subtask advances.
Racing buys attempts, never authority. Applies only to subtasks with their
own executable check — a subtask Relay measures with the loop is never
raced, because only the repository can answer that.
- `--resume` resumes its mission. Every resumed drive had been running
with an empty plan: no subtask focus, no gate, no lease, and no mention of
it. The mission ledger was written for exactly this and had no reader.
- Corrected documentation. Five docs promised mutation testing "in
0.5.0", thirty releases past; docs/AGENT_GUIDE.md told agents it would
catch them. A deterrent that does not exist is worse than none — a
version-checking adversary defeats it in one command and an honest reader
over-trusts the gate. All corrected to NOT SHIPPED, with a test that
distinguishes a promise from a stated limitation.
- Nothing ships that nothing reaches. A new fence fails the build on any
module no entry point can reach. It removed a superseded run panel and
parked two policy-drift modules with the reason written down: nothing
writes .relay/org-policy.json, so ci report --drift asks the operator
to name drifted fields by hand. A green ci report says nothing about
policy drift today.
Both invariants unchanged: no model output is ever an input to a verdict,
and Relay never says "safe to merge".
0.6.34 — 2026-08-20
The loop that circles gets caught, and a plan's scope is kept.
- Same-approach detection. Relay's stall counter only ever fired on an
identical finding, so an agent repeating one idea while the evidence
drifted could spend ten attempts. Relay now also measures what an attempt
DID — the change it made, the error it got, the files it touched — and
says so in one line with all three numbers. Two repeats demand a
different approach; three end that agent's run and hand the work on.
- Execution leases. Each subtask may only change the files its plan
named. Relay checks before running the subtask's check and refuses on
anything outside, naming the files — so a planned drive cannot quietly
drift back into "the agent did something and we measured it". Your own
uncommitted work is never counted against the agent.
- A rethink instead of a retry. When a subtask stalls, one supervised
turn must answer with a concretely different approach, a SPLIT into
smaller subtasks, or an honest escalation — validated the same way a plan
is, and bounded to two corrections per mission so a drive converges.
- Branch-execution machinery (not yet driven): transactional git
worktrees and a selection rule that keeps measurement in charge —
branches that fail their check are out, the smallest passing change wins,
and an adversarial reviewer may demote a passing branch but never promote
a failing one.
Both invariants unchanged: no model output is ever an input to a verdict,
and Relay never says "safe to merge".
0.6.33 — 2026-08-20
Relay reads your task, finds where it lives, and drives it as a plan.
The first cut of the mission harness (plan3), plus the panel fixes the
founder's own failing run exposed.
- Your words are mapped to your codebase. "add the test case for the
payment auth" is no longer an opaque string: Relay classifies the intent
(a TEST task, not a build task), locates the symbols and files it names —
through the code graph when installed, git grep when not, with the
provenance shown either way — finds the tests that already touch them and
the repository's own exemplars, and states its gaps instead of inventing a
target. The context artifact now OPENS with that map.
- Missions and gated subtasks. A mapped task becomes a plan: 2–7
subtasks, each with the files it may touch and the check that proves it.
The plan is validated mechanically — paths must exist, every check must
compile the same way --criterion does, no cycles — with one redraft and
a deterministic fallback, so planning can never make a drive worse. Each
subtask advances only when Relay EXECUTES its check. A failed gate keeps
the work on that one subtask instead of re-attempting the whole change.
- Every turn knows the task. Before this, turn 1 got the context and
every later turn got a bare coverage complaint. Now each authorship turn
carries the bound task, the located targets, and the one subtask it is
for — with what it is explicitly NOT for.
- The panel tells the truth. Fixed: the panel resurrecting under an
escalation menu (two stacked panels), the selector painting on the wrong
stream and never erasing itself, intervention keys going dead after the
first escalation, and PIPELINE showing five empty boxes at escalation. A
PLAN region shows the mission; --no-plan restores the previous behaviour
exactly.
Both invariants unchanged: nothing in the map, mission, plan, or panel can
reach a verdict — the fence names every new module — and Relay never says
"safe to merge".
0.6.32 — 2026-08-19
Errors defined, questions durable, the panel becomes an instrument. All
three plan2 workstreams in one cut (the slicing planned three; main carries
them together, and the tarball is the tag).
- The error registry. Every condition Relay can emit — the loop's codes
plus CHAIN_EXHAUSTED, AGENT_UNAVAILABLE, VENDOR_LIMIT, INTERRUPTED,
TREE_MOVED — is one table: category, severity, owner, the one recovery
action, inbox behavior, ladder data. Unregistered codes do not compile;
every code has a docs page; reasons are summarised at word boundaries and
never truncate mid-word. Vendor usage limits are detected from the
vendor's own words, reset time included.
- The inbox. Escalations write durable items BEFORE any menu; unanswered
ones PARK the drive (state + working-tree fingerprint preserved).
relayevals inbox lists (exit 40 when open — a CI gate), `inbox answer
<id> --note "…" records your guidance, drive --resume <id|last>`
continues the SAME lifecycle run with your answer leading the next prompt
and one granted attempt at the judged finding. A tree that moved while
parked refuses cleanly (TREE_MOVED). Ctrl-C leaves a resume card.
- The control panel. Fixed bordered regions own the screen — VERDICT
first (status dot, code, intact summary, eval stroke bar), PIPELINE
(stages, chain position, attempt pressure, armed delegation), ACTIVITY
(the only scroller), NEEDS YOU, boxed keys. e evidence, t timeline,
ending cards. Golden-frame tests pin exact output at three widths.
Both invariants unchanged: nothing in the registry, inbox, parked state, or
panel can reach a verdict — the fence names every new module — and Relay
never says "safe to merge".
0.6.31 — 2026-08-19
**The five questions at a glance, evals that compile, skills that compound —
and the numbers, head to head.** Four pieces of the master plan (plan-internal
Phases 2–5), every gate measured before it shipped.
- The drive panel answers the five questions. GOAL · TODO · FOCUS ·
EVAL · NEEDS YOU — fixed regions, always painted, repainted in place with
scrollback preserved. Single keys act while it runs: s stops cleanly at
the turn boundary (measured under 1s to acknowledge), d shows the diff
so far, i hands the NEXT agent turn a note attributed to you, ? help.
A drive without a keyboard behaves identically.
- Eval Harness 2.0.
--criterioncompiles every stated success
criterion to an executable check (cmd: / script: / test:) or the
drive refuses to start with the exact reformulation to make — there is no
state where a criterion is accepted and never verified. Compiled checks
run after the loop's PASS; a failure means the drive declines to call the
task done (exit 20) while the measurement stands untouched. Every failure
is classified from the loop's own vocabulary — never by matching runner
text — and every drive exports its path as OTel-shaped spans
(.relay/trajectory.jsonl) with a deterministic path score in the summary.
- Skills that compound, per repository. Every finished drive appends a
measured row — who resolved, who stalled, categories, elapsed — to
.relay/skills.jsonl. The next drive in that repo reorders its failover
chain by those counts and tells the agent what worked here before. Rows
are measurements with a run id attached, never prose an agent wrote about
itself. --no-skills opts out.
- A second opinion at the stall. At the second identical attempt, the
next agent in the chain runs as a supervised consultant — diagnosis only,
no edits — and its answer rides into the working agent's prompt,
attributed. Two agents visibly at work on one finding.
- Drives that outlive terminals.
drive --detachsurvives the terminal
closing (own process group, log + pid under .relay/), heartbeats every
60s, and drive --attach replays the log through the finished verdict.
- `relayevals graph install`. The verified code-graph binary, one
command: sha256 pinned per platform, verified BEFORE extraction, Sigstore
attestation recorded, provenance written beside the binary. Without it,
drives keep the brief+diff tier — smaller, never an error.
- The golden gate grew. 50 hermetic drive scenarios across 5 repository
archetypes now run in every suite; all 50 end in a clean PASS or a precise
NEEDS_HUMAN, asserted mechanically.
- Head to head, measured and published. On 5 identical tasks against the
same model driven solo: relay resolved 10/10 to solo's 5/10; solo's five
terse-phrasing failures were all "declared done, tested nothing"; on the
runs both resolved, relay used 2.08× fewer total tokens at half the wall
clock. Judged independently — never by any harness's self-report — and
published with raw rows in the repo (docs/bench/).
Both invariants unchanged: no model output is ever an input to a verdict —
the fence now covers the skills ledger and the graph downloader too — and
Relay never says "safe to merge".
0.6.30 — 2026-08-19
The lifecycle becomes data, and the drive starts knowing the repo. Two
pieces of the master plan (plan-internal Phases 0 and 1), both measured
before they shipped.
- The seven-state lifecycle. Every drive now emits its own discipline —
INIT → CONTEXT → GOAL → PLAN → (EXECUTE ⇄ EVALUATE) → CLOSE — into an
append-only log (.relay/lifecycle.jsonl), and a validator judges it at
the end of the run. A skipped state is a reported bug; a NEEDS_HUMAN
ending without the exact question and concrete options FAILS the log. An
interrupted drive closes with the command that resumes it, and the resumed
run validates alongside it in the same file.
- Deep context, fed forward. The drive builds a budgeted Repo
Understanding artifact — the brief, the diff itself, which tests reference
the changed files, and (where the verified code-graph binary is installed)
who calls the changed symbols — and hands it to the agent's first turn.
Measured across 12 live drives on three benchmark repos: ~2× fewer total
tokens, reads at the floor, every run PASS. --no-context preserves the
bare baseline.
- Metrics, extracted never estimated. Tokens per agent turn come from
the agent's own usage report; raw content reads are counted in the
stream. Both land per-turn in the transcript and as counts in the drive
summary, alongside the context tier that ran.
- The second identical attempt owes a root cause. Before a third try at
an unmoved finding, the prompt demands a stated reason the last change
failed — the three-strike handoff is unchanged.
The code graph (codebase-memory-mcp, MIT) is NOT bundled: where present it
was verified by checksum and Sigstore attestation before first run, and
customer installs simply run the brief+diff tier. Both invariants unchanged:
no model output is ever an input to a verdict — the fence now covers four
more modules — and Relay never says "safe to merge".
0.6.29 — 2026-08-19
The driver stops lying to the agent it drives. Found by the 0.6.28 live
smoke, against real signed-in agents — the thing a sandbox cannot test.
The first-turn prompt told the agent to run relayevals next --format json in
a loop. Under drive that was false twice over: the driver already runs next
before and after every turn, and a headless `claude -p --permission-mode
acceptEdits` may not run Bash at all. So the agent spent an entire turn
discovering it could not run the command Relay had just told it to run, and
signed off asking a human for an approval no headless session can give. The
drive still reached PASS — the driver measures either way — but a turn and its
tokens went to a misunderstanding Relay had created.
The prompts now describe the arrangement the agent is actually in: Relay has
already run next, Relay runs the measurement the moment the turn ends, and
the agent's job is authorship. Being unable to run a command is named as
expected rather than as something to ask permission for. **No permission was
loosened** — the fix is honesty, not access.
Measured on the same scenario, same repo shape, same agent:
| | 0.6.28 | 0.6.29 |
|---|---|---|
| claude turns | 2 — one spent on the misunderstanding | 1 |
| driver turns | 5 | 3 |
| times the agent asked for approval | several, including its closing message | 0 |
| result | PASS in 88.7s | PASS in 84.7s |
Both invariants unchanged: no model output is ever an input to a verdict, and
Relay still never says "safe to merge".
0.6.28 — 2026-08-19
A drive you can watch, and stop. relayevals drive used to inherit the
terminal, and claude -p writes nothing until its turn ends — so a working
agent and a hung one looked identical, the vendor's own warnings landed raw in
Relay's output position, and Ctrl-C killed the parent before it could say what
had stopped. Relay now reads the agent instead of handing it the screen.
- Streamed activity. claude runs with `--output-format stream-json
--include-partial-messages`, and its tool calls appear as they happen —
▸ Read src/api.ts, ▸ Edit src/api.test.ts. codex and hermes pass through
line by line. Every event shape was measured against the real CLI and the
capture is committed, so a vendor format change fails Relay's tests rather
than a customer's drive.
- A checklist derived from the loop. Finished turns collapse to one line
each and stay in your scrollback; below the active turn are only the phases
that must still happen. No model is asked what the steps are, and Relay does
not guess how many rounds a task needs.
- Every line is attributed. Vendor noise appears once, as
⚠ claude noise: …, and can never read as something Relay said.
- Ctrl-C is an ending, not an accident. The drive stops where it stands,
prints the state you stopped in and the command to resume, exits 130, records
the turn as interrupted — and does not advance the failover chain. An
interrupt is not a crash, and the next agent is not spent on cancelled work.
- `--verbose` shows every raw line, attributed, with no panel. In CI or a
pipe the panel becomes append-only lines: a drive is never silent.
- A drive summary: turns, turns per agent, handoffs, interrupts, elapsed —
every field a count or an elapsed time read from the transcript.
- No path ends in a bare stack. An unexpected error now prints what
happened, says it is a Relay bug, and exits 70.
Both invariants unchanged: no model output is ever an input to a verdict, and
Relay still never says "safe to merge". The drive fence was strengthened, not
relaxed — all four new modules are proven unreachable from every verdict entry
point, and a drive transcript on disk still moves no decision.
0.6.27 — 2026-08-19
Enter always means "go". In the connect wizard's agent picker, pressing
enter with nothing toggled used to do nothing — no feedback, which reads as a
hang, and stalled a real onboarding at the first screen. Enter on an empty
selection now picks the row under the cursor and confirms; space-toggling for
multi-agent chains is unchanged.
Both invariants unchanged: no model output is ever an input to a verdict, and
Relay never says "safe to merge."
0.6.26 — 2026-08-19
Onboarding looks like onboarding. relayevals connect now renders the
staged screen: numbered steps with ✓/○ progress, a detection board that shows
every known agent (hits and misses), vendor sign-ins performed inline with
their result lines, the repository section (path · git · branch · policy ·
vouch), and a final status board that ends with "Ready." and the one command
left to type.
The duplicating menu is fixed. In a real terminal, long lines wrap — one
logical line becomes two physical rows, the repaint arithmetic under-counts,
and the menu re-drew downward stacking copies of itself. Every rendered line
is now clipped to the terminal width, counting visible characters (color
codes are zero-width), so the menu redraws in place.
`relay` now works. The installer links relay alongside relayevals —
it is what people actually type. Guarded: a different relay already on your
PATH is never shadowed.
Both invariants unchanged: no model output is ever an input to a verdict, and
Relay never says "safe to merge."
0.6.25 — 2026-08-19
`relayevals connect` finishes everything. The first real-terminal run of
0.6.24's wizard left the connecting to the user — agents showed "NOT logged
in" and stayed that way. Now one command completes the whole onboarding, in
order: the Relay account choice first (local remains free, no account
required), pick your agents (the pick order is the failover chain), and where
an agent is signed out, **Relay launches that vendor's own sign-in right
there**, waits, re-checks, and reports signed in ✓ — the screen that appears
is the vendor's, and Relay never sees the credential. Then modes, and — when
run inside a repository — the strict policy is written and the machine-key
vouch happens on the spot. After connect, drive "<task>" needs nothing.
Two detection fixes, both measured. The claude CLI's auth store is
separate from the Claude Code app's — "NOT logged in" was true, so the hint
now ends "Relay will sign you in" instead of reading like an error. And
hermes' auth state is honestly unknowable (hermes auth status requires a
provider argument), so Relay no longer claims to know it — no hint rather
than a guess dressed as detection.
Both invariants unchanged: no model output is ever an input to a verdict, and
Relay never says "safe to merge."
0.6.24 — 2026-08-19
The orchestrator release: you choose, Relay drives.
Until now relayevals start ended with "hand this to your agent" — and a
human ferried the handoff block and every fix prompt by hand. 0.6.24 removes
the ferrying. Connect your agents once, type the task — the only thing you
ever type — and Relay calls the connected agents and manages them to a
verdict. Both invariants hold, untouched and newly proven: no model output is
ever an input to a verdict (a 48-assertion import-graph fence makes the
orchestrator unreachable from the verdict path), and Relay still never says
"safe to merge."
`relayevals connect` — onboarding as choices. Relay detects the agent
CLIs on your machine (claude, codex, hermes verified; cursor-agent, aider,
opencode, gemini recognized), and everything is an arrow-key choice: which
agents, each one's mode, the failover order (the order you pick them), and an
optional team account. If an agent isn't signed in, Relay launches the
VENDOR's own login right there — it never sees, stores, or transmits a
credential. The choices land in ~/.relay/agents.json; a test asserts nothing
credential-shaped can be in it.
`relayevals drive "<task>"` — the loop, driven. Mechanical actions
(setup, receipts, grammar installs) run through Relay itself — the 0.6.22
blind-agent finding made structural: only authorship needs a model. Agents
are invoked solely to write what's missing, fed the same fix prompt the loop
already produces, and after every authorship turn Relay runs the verification
half itself, so evidence always moves. When an agent stalls — three identical
attempts, by the ledger's own arithmetic — the next agent takes over with the
task and the attempt history. Nothing cleared, nothing forged: the ledger's
rules are the failover policy. Every turn is attributed in
.relay/drive-transcript.jsonl, and the final transcript shows exactly who
cleared what.
Escalations are choices too. Chain exhausted, toolchain installs (npm i
-D c8 is proposed, never auto-run), trust setup — all render as options.
"What would you like to do?" appears nowhere. The one decision that stays
human-and-terminal-only is approving this machine's signing key, exactly as
before.
How we know. A hermetic sandbox run drives a deliberately stalling fake
claude into an honest FIX_LOOP, hands the chain to a fake codex with history,
reaches a measured PASS with zero human input after the task — and proves the
verdict byte-identical with and without the drive transcript on disk. Two
real bugs were found by exactly this test and fixed before release: an
authorship action misclassified as mechanical (the run still passed via
failover — only the attributed transcript exposed the skipped agent), and the
missing verification half that let a stalling agent spin 38 turns without
ever escalating.
0.6.23 — 2026-08-16
**A maintenance release, cut because testing 0.6.22 end to end found a bug in
0.6.22.** No feature changes. Both invariants untouched: no model output is ever
an input to a verdict, and Relay still never says "safe to merge".
Detection no longer reads the machine it happens to be running on. 0.6.22
added Relay's own vendored coverage tool under ~/.relay/vendor, and detection
looked there by default — so the moment you installed Relay, every "this
repository has no coverage tool" case silently became a vendored case. On a
developer's machine that turned four passing tests red for no reason, and CI
would have gone red for any contributor who had installed the product they were
contributing to. RELAY_VENDOR_ROOT now relocates the toolbelt, an explicit
option still outranks it, and the test suite pins it at nothing — beside the
line that already keeps the suite off the real OS keychain, which exists for
exactly the same reason. A default that reads the invoking user's home is the
kind of hidden input this codebase refuses everywhere else.
The test suite passes from an unpacked source tarball on Linux. Nine tests
failed there in 0.6.22 — not because anything was broken, but because they asked
questions only a git checkout can answer, and one modelled a macOS tool that a
bare Linux box does not ship. They now decline, out loud, with a reason:
- the vendored-toolbelt drift checks need
release/, which is deliberately not
shipped, so they skip rather than error — and they *declare* the skip instead
of returning quietly, because a test that reports "ok" while checking nothing
is exactly the hollow green this product exists to refuse;
- two tests ask
git archive HEADand two read an internal release record that
is deliberately not shipped — all four are checkout-only questions;
- the keychain stub hex-encoded with
xxd, which ships with macOS but not with
a bare Linux image, so exactly one test failed and read like a product bug. It
uses POSIX od now, verified byte-identical.
A full acceptance suite you can run yourself. scripts/e2b/verify0622_full.py
walks every acceptance item on one clean Linux box installed the way a stranger
installs it, and takes RELAY_VERSION so a release is never verified against
the previous release's artifacts.
0.6.22 — 2026-08-16
The agent asks the binary. The binary tells the truth.
When a customer hands a task to an agent, that agent has to be able to diagnose
itself by talking to Relay — every command, every state, every exit code, every
finding and its fix — without ever having seen a website. In 0.6.21 its
knowledge was whatever the handoff block and --help prose happened to say. Now
the binary is the documentation. Both invariants are untouched: no model output
is ever an input to a verdict, and Relay still never says "safe to merge".
**relayevals agent --format json — the whole contract, generated, not
written.** 61 commands, 14 states, 43 codes, each with what clears it, plus the
commands that apply to THIS repository: the task bound here, the test command,
and the exact evidence command with its flags already resolved. It is walked out
of the commander program and the real exit-code and remediation tables, so it
cannot drift from the binary the way a hand-written inventory would. Advisory
and fenced: a test proves the verdict path cannot import it.
`relayevals help` works. It used to exit 64 — the self-service front door
returned an error. help, help <command>, and unknown-command suggestions
("did you mean receipt?") all work now, and help exits 0.
Every code Relay emits can be explained. --explain covered 28 of 43 codes;
the 15 loop codes had no records at all, so an agent that met one had nowhere to
look. A fence now fails the build if any emitted code lacks a remediation, which
is mutation-tested by adding a fake code and watching it go red.
Actions are runnable, not prose. action.description was written for a
human to read; a driver that executed it got Run:: command not found. Actions
now carry action.command — the command alone, nothing around it.
Relay brings its own measuring instrument (rung 3), and discloses it. A
repository with no coverage tool of its own is no longer told to go shopping:
Relay measures with the c8 it installs for itself under ~/.relay/vendor,
without writing anything into your repository and without ever shadowing a tool
you already have. Because those numbers came from equipment you did not choose,
the signed receipt names it — package, version, and the sha256 of the exact file
that ran, measured from what executed rather than declared. Delete the toolbelt
and repositories degrade to a runnable install command, never a dead end.
The maintenance contract is written down and enforces itself. Vendoring
third-party tools is a promise to keep them current. vendor-manifest.json is
the single source of truth; a test fails the build when the installer's pin
drifts from it, when the manifest claims to ship a tool no installer stages, and
when the monthly review goes 30 days overdue — with the fix in the failure
message. Its first real review immediately held c8 at 10.1.3 (12.x drops the
Node 18 users rung 3 exists for) and demoted an unshipped coverage.py from a
claim to a plan.
Also fixed: try no longer adds --coverage to a script that already has
it, so a passing suite is never reported as your tests failing; rotate accepts
a producer name; the verdict path now sees an unaccepted policy edit the same
way check does; the run supervisor falls back when a socket path is
unsupported; a revoked producer says so in the receipt's reasoning.
How we know. A deliberately dumb driver — allowed only to read the contract,
call next, run the command it is given, and branch on the exit code — drives a
clean Linux box from BLOCK to a measured PASS in three turns, on two
repositories: one measured by Node's built-in coverage, one by the vendored c8.
It has never seen documentation and never guesses a flag. When it stalled, the
product got fixed, not the driver: that run is what found the unrunnable actions
and a missing coverage directory.
0.6.21 — 2026-08-15
The 2/10 release: install, `start`, paste — nothing to wire, nothing to learn.
0.6.20 measured 6.5/10 for a new human, and two walls were most of it: a repo
with no coverage tool hit NO_COVERAGE_TOOL, and the loop took four setup
commands. Both are gone. Not one check weakened — every unit of complexity moved
off the human onto Relay or the deterministic loop. Strict-by-default
(measured + block) stays; the two invariants — no model output is ever an input
to a verdict, and Relay never says "safe to merge" — hold through all of it.
`node --test` is now measured natively (the coverage wall, gone). A repo
whose only test command is node --test — the modern zero-dependency way to
write tests — used to be told to install a coverage tool. Node 22+ emits lcov
itself; Relay now uses it. Nothing to install, no flag to learn, and the word
"lcov" never reaches the human.
`relayevals start "<task>"` — one command for the whole ceremony. It sets up
the repo (strict policy), resolves coverage silently, binds the task, asks for
the one signature that vouches for this machine's signing key, and prints the
paste-ready block for your agent. The four-step setup → trust approve → task →
hand-composed prompt is now one command that ends in a paste. Scripted or
non-TTY, it still leaves the key unapproved and the loop still stops at
NEEDS_HUMAN — the security model does not move.
`relayevals task --brief` — advisory planning evidence, provably fenced. A
map for the agent before it edits: the real changed surfaces, how Relay will
measure them, and what PASS will require. It can never feed a verdict, and that
is enforced structurally — the verdict path cannot import it, no verdict module
reads its artifact, and a verdict is byte-identical whether or not a brief exists
on disk.
The human view is three things. next shows state, task, and one action;
the counts and the attempt ledger move behind --verbose (and `explain
--verbose). The machine contract (--format json`) is untouched — every field,
every state, every code still ships to an orchestrator.
Continuity to the merge gate. The bound task now rides the fix prompt and the
signed receipt (task_hash), so CI can name what a change was for without
reading the gitignored .relay/task.json. task_hash is inside the signature —
tampering with it fails verification — and additive, so older receipts still
verify. And the local loop never nudges toward login: signup lives where trust
crosses machines, pinned by a guardrail test.
Verified: 2459 CLI tests across 149 files (36 new), 314 web. Acceptance
proven end to end on a clean Linux box with ZERO coverage tooling — install →
start → paste → BLOCK → PASS, measured via node --test, task named through to
the signed receipt — in 19 seconds (scripts/e2b/verify0621.py).
Still on the roadmap: the vendored c8/coverage.py toolbelt (for old-Node and
exotic stacks) and the 60-second sandbox tour. Neither weakens the gate; both
are separately-verified installer/loop work.
0.6.20 — 2026-08-14
State your task once; Relay carries it to PASS.
The loop had no memory of what you were building. relayevals next verified
against a placeholder — "current changes" — and to name the actual work you had
to re-type it into verdict on every call. So the promise "give Relay the task
and it handles the rest until merge" was missing its first word: the task.
One command binds it. relayevals task "add rate limiting to /api/upload"
writes the intent, verbatim, to .relay/task.json (per-repo, per-machine,
gitignored — the current work in this checkout, not a committed team rule).
From then on every relayevals next is judged against that exact sentence,
names it at each verdict, and holds the change to measured evidence until PASS.
Done, or switching subjects? relayevals task --clear.
The task text moves no verdict on its own — the gate is still coverage and
signed evidence, and the invariant holds: **no model output is ever an input to
a verdict.** What the bound task changes is that the loop, and the human reading
it, always know what is being judged. It is the continuity spine the ladder
(task contracts, the ten rungs) will hang from.
Verified: the continuity scenarios replay a task riding a BLOCK→PASS loop
and being named in the reason; the task lifecycle (bind → read → clear) is
confirmed on the built binary; full CLI and web suites green.
0.6.19 — 2026-08-14
A PASS can no longer contradict its own coverage count.
A cofounder ran Relay against a live landing-page redesign and it printed:
```
PASS — 0 / 1703 changed executable line(s) executed. The task can end.
```
A verdict and its own refutation in one sentence. It happened because Relay
decided "is the changed code tested" in two places that could disagree: the
gate accepted a surface a receipt merely *declared*, while the counter refused
to credit a declared line — and nothing tied a PASS to verified == total.
Strict by default, everywhere. A new repository, and the policy Relay
invents when a repository has none, now both require *measured* coverage that
executes every changed line; a gap BLOCKs. A repo with no committed policy can
no longer gate on nothing. Existing policies are never rewritten — the new
relayevals policy strengthen upgrades a legacy weak one in one confirmed,
TTY-only step.
Relay no longer hands the agent the rope. relayevals next used to
compose receipt run --surface <every changed file> regardless of policy, and
an agent obeying it manufactured the exact declared receipt that faked the
pass. It now adds --surface only under measured coverage, matching the guard
guard already had — so following Relay's own instruction can never fabricate
its own evidence.
Loud when weaker. A team may still choose a lenient policy, but a PASS then
names every unenforced criterion, carries policy_gaps in the JSON so an
orchestrator can refuse it, and points at policy strengthen. And "the task
can end." is retired: completion is the human's word, never Relay's.
Linked git worktrees work. relayevals threw "unable to resolve a Git
root" in any git worktree add checkout, because there .git is a file
pointer, not a directory. It resolves them now, still rejecting the
core.worktree escape. This is why the cofounder had to copy changes into the
main checkout by hand.
Also: coverage_enforcement: "off" no longer suppresses a *missing* coverage
report (only per-line gaps), and changed lines are called "changed", not
"executable", because on a markup change most are not.
This is the first of a larger effort. A task-contract loop and a ten-rung
evidence ladder are planned as separate phases. Two invariants hold through all
of it: no model output is ever an input to a verdict, and Relay never says
"safe to merge."
Verified: 2416 CLI tests across 142 files (four new scenarios replay the
incident and its accomplice), 318 web tests across 23. Confirmed on the built
binary: an uncovered change BLOCKs, a covered one PASSes cleanly, and a repo
with no policy no longer fake-passes.
0.6.18 — 2026-08-14
**A fresh macOS signing key could not sign anything, and had not been able to
since 0.6.4.**
On macOS a new producer key defaults to the OS keychain. Reading it back uses
security find-generic-password -w, which is not a transparent pipe: it returns
a value verbatim only when every byte is printable ASCII, and returns it
hex-encoded — silently, with no flag — the moment the value contains a
newline. A PEM is multi-line by definition. So the key was stored, read back as
hex, and handed to a PEM parser that threw a raw OpenSSL
error:1E08010C:DECODER routines::unsupported. Every fresh macOS install could
set up a repository and then never produce a single signed receipt — which
means never a verdict, because a verdict needs a receipt.
This is very likely a large part of why the usage dashboard shows machines
setting up and never reaching a verdict: most of them are macOS.
Why it shipped green for fourteen releases. The test that proves a keychain
key signs used a stub that round-tripped bytes verbatim instead of modelling
security's hex behaviour. The suite could not see its own headline feature
failing. The stub now hex-encodes exactly as the real tool does, and with that
one change, reverting the fix fails two tests. The fix was additionally proven
end to end through the real /usr/bin/security in a throwaway keychain.
The fix. Values are stored base64-encoded behind a marker, so they are
always single-line printable ASCII and the hex path is structurally
unreachable. Keys written by 0.6.4 through 0.6.17 are recovered on read and
migrate to the new form the next time they are written — no re-setup, no lost
keys.
Two more bugs on the same path, both fixed:
- The opaque OpenSSL string is now a named error naming the fix, instead of a
decoder failure three layers from its cause.
relayevals producer protectleft a broken encrypted key on disk when its
own round-trip check failed *by throwing* (the cleanup only ran on a returned
mismatch). Because an encrypted key outranks a keychain key, that file then
hid a still-recoverable one. Any round-trip failure now cleans up.
Also fixed, from the same external report:
- The GitHub-Actions CI snippet on the docs site and in the manual passed a
producer key without the authority public key that --profile ci requires,
so the documented command failed outright. It is the same omission the team
console had, fixed there in 0.6.15 and missed here.
docs/CI.mdanddocs/MANUAL.mdpiped a pull-request title straight into
a shell run: step — a title like "; curl … | sh # would have executed in
the runner. The title now travels through an environment variable.
Found by an external tester who exercised the whole product against 0.6.16 and
wrote it up precisely. Every claimed defect was verified independently before
being acted on; several were confirmed, and several were refuted.
Known and not fixed here: relayevals does not auto-detect Node's built-in
--experimental-test-coverage reporter (pass --coverage lcov:<path>
explicitly), and producer rotate / producer revoke do not accept
--format. Both have working workarounds.
Verified: 2406 CLI tests across 141 files, 318 web tests across 23 against a
live database.
0.6.17 — 2026-08-14
**We could not tell whether anyone was reaching a verdict, and read that as
nobody reaching one.**
The dashboard said 169 machines and zero PASS/BLOCK. That was taken as a fact
about users. It was a fact about our own source code:
```
== events by name ==
name | rows | producers
'install' | 194 | 169
```
install was the only event name in the database. Not the most common — the
only one. The CLI had exactly one place that recorded anything, and its name
was a hardcoded literal. Six more names were declared on both sides of the
wire, accepted by the ingest endpoint, given columns in the schema, and used to
build the funnel screen. None had ever been emitted.
So "not a single verdict" was unfalsifiable. It might have been true. There was
no instrument that could say either way, and the absence of evidence got read
as evidence of absence — which is the exact mistake this product exists to
prevent, made by the product's own instrumentation.
Every name in the vocabulary now has a caller: setup, all three verdict
outcomes from both verdict and guard, fix.requested, and
loop.escalated.
A local verdict still never touches the network. That invariant is
load-bearing and it did not move. try, guard, next, verdict, report,
fix and explain only *record* — one line to your own disk, no socket.
Sending happens from check and receipt run, rate-limited to once every six
hours or a full batch, so an agent loop never waits on a network call.
Where people get stuck is now recordable. Knowing the verdict mix does not
answer the question. "Ninety machines ended UNRESOLVED" is not actionable;
"ninety hit TRUST_SETUP_REQUIRED and were never seen past it" names a missing
line of documentation. Events can now carry a state code — Relay's own enum
names, constrained to ^[A-Z_]{1,40}$, a shape no file path, repository name or
task string can take. The consent notice names the new field, because a notice
that under-describes is not consent.
Agent runs were invisible, and are not any more. --format json returned
early *above* the recording step, so a machine driven by an agent wrote nothing
to its own disk either — there was never anything for a later run to carry.
Every number we have describes people who ran Relay by hand. Recording is
unconditional now; only sending waits for the notice, and stdout is untouched
either way.
---
Three walls that would have made the new numbers depressing, removed.
`relayevals setup` contradicted itself eleven lines apart. It wrote
coverage_evidence: "measured" into your policy and then printed:
```
Next:
relayevals receipt run --name tests --kind test -- npm test
```
with no --coverage. That command cannot satisfy that policy. Anyone who
followed the CLI's own instruction landed on UNRESOLVED, every time — and
because coverage enforcement defaults to warn, they could not reach BLOCK
from a coverage gap either. The value moment the whole product is built around
was unreachable on the documented path. It now prints relayevals guard, which
detects your stack, substitutes the coverage-enabled command, and can actually
end in PASS.
`relayevals next` stopped every new user on its first call. It refused with
NEEDS_HUMAN · TRUST_SETUP_REQUIRED until a person vouched for the machine's
signing key — correctly, because an agent must not approve its own key. But the
command that cleared it, relayevals trust approve, appeared in none of
the six documents a new user reads. And the agent contract we ship said "repeat
until PASS, READY, or NEEDS_HUMAN", so a correctly-behaving agent stopped there
and said nothing. Silence and success looked identical.
setup now approves the machine when a person answers its prompt. An agent
still cannot, because an agent has no stdin — the separation that made the gate
worth having is intact; what changed is which question the human answers. Run
setup --yes, --non-interactive, or piped, and nobody answered, so the key
stays unapproved and trust approve is still required. The approval record
says which command produced it.
The agent contract now reads "repeat until PASS or READY", with an instruction
to surface NEEDS_HUMAN verbatim.
Not caught for a reason worth stating. The 31-scenario suite that proves
the loop always terminates pre-approves the machine in its harness — so it had
never once run from the state a real user is in.
---
Also in this release
- The ingest endpoint answers 500 on a failed write instead of swallowing it
into a 204. A broken pipeline and a healthy one looked identical from both
ends, and the CLI, told everything was fine, discarded what it was holding.
/admin/funnel: four steps with the drop between them, a table of which wall
machines hit and how many never got past it, daily activity, and cohort
retention. Every step separates machines that came through the installer from
source checkouts, because a blended total is not a user count.
- Fixed a crash on the admin team page: it read a variable ten lines before
declaring it, so the branch that exists to refuse gracefully threw instead.
Verified: 2404 CLI tests across 141 files, 311 web tests across 22 against a
live database. Both fixes exercised against the built artifact in throwaway
repositories — a real pseudo-terminal for the approval, and the queue inspected
after a real run.
0.6.16 — 2026-08-13
A lockfile is not a surface, and Relay's own output is not one either.
Both of these stop an ordinary pull request from ever going green, and both
were found on a real one against released 0.6.15.
Lockfiles are exempt now. A change was made, a test was written that fully
covered it, and the verdict was still BLOCK — the source file no longer listed
at all, and package-lock.json the only thing standing between the author and
a green check.
This reverses a deliberate decision, and the old comment in the source deserved
an answer rather than deletion:
> NOT exempt, deliberately: lockfiles. A dependency version change DOES alter
> runtime behaviour, so package-lock.json / uv.lock / go.sum stay in the
> inventory. Exempting them would be convenience bought with a real hole.
Every clause of that is true. It is also a gate no ordinary pull request can
pass, and the hole is not closed by pretending a lockfile is source: a
dependency change that alters behaviour alters it *through code*, and that code
is already in the inventory. What this removes is a demand for evidence that
cannot exist — no test anyone can write executes a lockfile, so the demand
could only ever be ignored, and a gate that must be ignored teaches people to
ignore gates.
Thirteen names: package-lock.json, npm-shrinkwrap.json, yarn.lock,
pnpm-lock.yaml, bun.lockb, Cargo.lock, go.sum, poetry.lock,
uv.lock, Pipfile.lock, Gemfile.lock, composer.lock,
packages.lock.json.
It is in code, not in the default .relayignore, for the reason that
file's own header records: ensureRelayignore() never updates a .relayignore
that already exists, so a new default would reach new repositories only — and
every repository already using Relay is exactly the one stuck behind this. No
reinstall, no file to edit, no migration.
Manifests are still counted, on purpose. package.json and
pyproject.toml are bytes a human chose to write; that is a different question
from a generated lockfile, and it is not the one being answered here.
`ci report` no longer blocks on its own paperwork. --verdict-file was a
required option, which taught everyone to write:
```bash
relayevals verdict "…" --format json > verdict.json
```
into their own worktree — and Relay then counted verdict.json as a changed
surface no test executes, so the *next* verdict blocked on Relay's own output.
On the same pull request, after the source was fully covered, the two surfaces
left were a lockfile and verdict.json.
verdict has always written its artifact to .relay/verdicts/, which is
already exempt. The flag is optional now and defaults to the newest artifact
there, so the documented flow never puts a file where it can block anything.
With nothing to report it says so — *"no verdict to report. Run `relayevals
verdict "<the task>"` first"* — and exits 64, rather than complaining about a
missing option.
Verified: 2378 CLI tests across 139 files, 292 web tests against a live
database. Four mutations caught: counting lockfiles again, dropping one name
from the list, removing the default lookup, and the manifest guard. Both fixes
re-checked against the built artifact in a throwaway git repository, not only
in unit tests.
0.6.15 — 2026-08-13
The paid plan's onboarding did not work. It does now.
relayevals ci auth publishes four values as GitHub Actions step outputs, and
the workflow the team console tells you to paste uses them:
```yaml
- id: auth
run: relayevals ci auth
- run: |
relayevals setup --profile ci --non-interactive \
--producer-key @${{ steps.auth.outputs.producer-key }} \
--authority-public-key @${{ steps.auth.outputs.authority-key }}
```
It never wrote a single one of them. The function that appends to
$GITHUB_OUTPUT reached for require() inside a module that ships as ESM,
where require does not exist; it threw on every run, and its own error
handler swallowed the throw and reported failure as an ordinary false. So
every ${{ steps.auth.outputs.… }} resolved to an empty string and the next
step stopped with:
```
--profile ci requires --authority-public-key <value|@file>.
##[error]Process completed with exit code 65
```
Anyone who bought the Team plan and followed the setup exactly would have hit
this. The console's YAML was also missing --authority-public-key itself,
which --profile ci requires — so the same paragraph was wrong twice.
How it was found, and why nothing caught it. By creating a real GitHub
repository, pasting the documented workflow, and pushing. The outputs are only
written when GITHUB_OUTPUT is set, which no unit test does, and the failure
was silent by construction — the code reported "I did not write them" and
nothing was listening. A first attempt at a regression test passed with the
bug restored, because the test runner supplies a require shim the real CLI
does not have; the test now runs the built artifact through a real Node ESM
process.
Also confirmed in the same run, against production, for the first time: a
repository that has never run the gate registers itself from its own CI run —
the numeric owner and repository ids come from a GitHub-signed token rather
than from anyone typing — and a pull-request run carries its PR number while a
push to main carries none.
0.6.14 — 2026-08-13
You can watch it work now.
try and guard run your test suite and could take twenty seconds with
nothing on screen. Dead output feels like a dead product — and worse, a
verdict that appears out of nowhere is an assertion. You have no reason to
believe it.
So the run shows its work. Only the phase in progress shows detail; a finished
phase collapses to one line, which is what keeps the screen the same height on
a four-file repository and a four-thousand-file one:
```
✓ Repository read · node · npm test · c8 0.4s
✓ 12 files · 138 executable lines changed 0.6s
⠹ Running your tests 12.4s
$ npm test
use --stream to see full test output
□ Measuring what ran
□ Deciding
```
Every line is a fact that was actually established. There is no percentage and
no ETA for someone else's test suite — Relay cannot know how long yours takes,
and a bar that guesses is the same category of lie the product exists to
catch. Under 400 ms nothing animates at all, because a spinner that flashes
once is worse than none.
guard says the thing a skeptical engineer is actually weighing:
```
✓ bound to 4 changed files at their current contents
✓ Receipt signed · local-default 0.1s
```
That is the 0.6.0 advisory fix as an observable fact rather than a changelog
entry — the run stating that its receipt is pinned to the worktree that was
tested. The panel this replaces could not say it: it was driven from outside
the function and only ever knew "run suite".
Every run now ends with what it did.
```
relay 0.6.14 · measured · nothing written · 19.4s
```
The coverage mode is there permanently, so a repository still in declared
mode sees that word on every single run and not only on the run where it
changes the answer.
Agents and pipes see none of it. --format json is byte-identical to
before, with an empty stderr and zero escape bytes, and that is structural
rather than a promise: the JSON path subscribes to no events, so there is no
code path from a progress event to stdout. A pipe gets the final frame only. A
CI log gets append-only lines with durations, because carriage returns render
badly there. TERM=dumb and NO_COLOR get the same lines in ASCII —
previously they got silence, which honoured the rule by being quieter than it
asked for.
Under the hood. try ran your suite with a synchronous spawn, which
blocks the event loop for the whole run — so no spinner could turn during the
one phase that needed it. It runs asynchronously now, and everything the
synchronous call gave for free is reconstructed, because each piece
distinguishes one real failure from another: a runner that is not installed
never ran, a suite that outran the output budget may have been passing, and a
timeout is us stopping it rather than it stopping itself.
`--stream` passes your test command's output through verbatim. The live
view stands down while it does, because interleaving unbounded output with a
region redrawn by counting lines is how a terminal gets corrupted. Nothing is
lost either way: the full output is hashed into the receipt regardless.
NOT in this release: receipt run does not publish phases yet; only try
and guard do.
0.6.13 — 2026-08-13
Two ways `try` could tell you untested code was fine.
Both were found by installing the previous release from the public internet
onto a clean machine and running ordinary things on it. Both are the product's
own thesis turned against it.
A function nothing called counted as covered. On the single most common
shape in modern JavaScript:
```js
export const surge = (n) => n * 3;
```
the assignment runs the moment the module is imported, so the coverage report
says DA:1,1 — line executed. The body never runs, and the same report says
so: FN:1,surge with FNDA:0,surge. c8 prints it as % Funcs 0. Relay read
the line record and skipped the function record, so try answered "Every
changed line is executed by a test." and exited 0 on a function no test ever
called — and the signed path agreed, guard returning PASS.
The evidence was inside the file being parsed. The parser skipped FN/FNDA
deliberately, for a reason that holds for branch records and not for these:
branch coverage is inconsistent across runners, so a gate built on it behaves
differently depending on who produced the report. Function records do not have
that problem — a report without them behaves exactly as before. Reading them
is additive; skipping them was a false clean bill.
A coverage report from an earlier run counted as evidence for this one. A
package.json whose test script delegates to another npm script drops the
extra flags, so the detected command runs the suite *without* coverage and
writes nothing. try read whatever was already on disk and never asked
whether the run it had just performed produced it.
Measured on a repository whose current suite touches nothing in the changed
file — vitest's own output reading calc.js | 0% | 0% | 0% | 0% | 1-2:
```
src/calc.js 1 changed 1 executed
Every changed line is executed by a test. exit 0
```
The report was real. It was written before the test was rewritten, and it
proved nothing about the state being judged. try now compares the report's
identity before and after the run and refuses one its own run did not write,
naming the delegating test script as the likely cause.
And a correction. 0.6.12's notes said commands outside a Git worktree
"now answer NOT_A_REPOSITORY with exit 64". That was true for verdict,
check and report and not for setup, which kept exit 70, or guard,
which emitted no JSON envelope at all on its failure path. The rewrite matched
one syntactic form of the exit assignment and missed the conditional form, and
the test covered three commands rather than every command that can fail
outside a repository. Both are fixed, and guard now answers in the format it
was asked for.
0.6.12 — 2026-08-13
The machine-facing contract now says what is true.
Two promises AGENT_PROTOCOL.md makes without qualification were false in
every release so far, and neither failed anywhere.
Error codes. §4 says relay:error:v1 "carries exactly one of these" and
lists thirteen. Nine others were reaching agents — EVALUATION_FAILED,
RECEIPT_VERIFY_FAILED, PRODUCER_INIT_FAILED and six more. An agent
switching on the documented enum fell through to its default branch on
ordinary failures like a bad receipt path.
The type system was already correct: errorEnvelope(code: ErrorCode, …)
cannot accept an undeclared code. Sixteen call sites built the envelope by
hand and walked past it. All sixteen go through the helper now, and a test
fails on any hand-written one — a type only protects the call sites that use
it.
Eight of the nine were redundant and collapsed onto the documented code that
already meant the same thing; a receipt that could not be inspected and one
that could not be verified are both RECEIPT_INVALID, and the difference
belongs in message. One was not redundant and is now declared and
documented: `HUMAN_APPROVAL_REQUIRED` is the only error whose remedy is
"stop and fetch a human", and an agent that cannot tell it from
PRODUCER_FAILED retries forever against a prompt no loop can satisfy.
`NOT_A_REPOSITORY` was documented and unreachable. Run any command outside
a Git worktree and it reported whatever its own catch block happened to name,
with exit 70 — which §2 defines as "internal error; treat as a bug". So an
agent whose human was in the wrong directory was told Relay was broken, and
would file that rather than say "cd into your project". It now answers
NOT_A_REPOSITORY with exit 64.
Schema discriminators. §5 says every JSON output carries one. try,
report and whoami carried none. eval and policy capture were worse:
their identifiers had been declared and named in the §5 table since 0.3.0 and
never once emitted, so a consumer who wrote that branch from the documentation
wrote dead code. All five emit now.
report --format json also ignored the format flag on its failure path,
answering a request for JSON with an English sentence on stderr. It answers in
the format it was asked for, whichever way it goes.
Two things deliberately unchanged. next keeps relay.next/1, which does
not match the relay:<noun>:v<n> shape the others use. It is the most-read
machine surface in the product and renaming it would break every agent driving
the loop in exchange for tidiness; §5 documents it as the exception it is.
try and report keep camelCase keys — §5 read verbatim states no casing
rule, and converting would break consumers to satisfy a convention the
document never made. §5 now says casing is not part of the contract.
0.6.11 — 2026-08-13
`relayevals try` blocked people for writing a test.
Add a function, add a test that covers it — the thing the product spends every
screen asking for — and try said this:
```
src/pay.js 1 changed 1 executed
test/t.mjs 2 changed 0 executed
Your suite passed. These lines never ran.
```
The source file was measured correctly. The test file's zeros were never a
measurement at all: coverage tools leave test files out of their own reports,
so there was no entry to read, and "no entry" was being rendered as "nothing
executed".
That is the one thing try is not allowed to do. Its own header says so —
never guess a number, because reporting "0 lines executed" for any reason other
than measurement manufactures exactly the kind of unfounded claim this product
exists to catch. It was doing it to the people who had done everything right.
The question was incoherent besides. try asks whether a test executes your
changed lines. Asking whether a test executes your test has one embarrassing
answer and no useful one.
Test files are excluded now, by the default globs rather than by policy —
try is ephemeral and has no policy to read, and the verdict path already does
the same thing through test_globs. Seven naming conventions are covered, not
one: test/, tests/, spec/, __tests__/, *.test.*, *.spec.*,
*_test.go, *_test.py, test_*.py.
A change to test files alone now says so. It used to report "nothing
changed that Relay can measure", which is both wrong — something did change —
and discouraging at the exact moment someone did the right thing. It now names
what happened and what would make it measurable.
This was found by installing 0.6.10 from the public internet onto a clean
machine and running the ordinary happy path on it. It is not a 0.6.10
regression: under the file-based rule that release replaced, the test file
already had zero executed lines, so the outcome was the same. It had been there
the whole time, on the first command every new user runs.
0.6.10 — 2026-08-13
`relayevals try` told people their untested code was fine.
It derived ok from the count of files where *nothing* ran. A file with one
covered line and five uncovered ones therefore counted as covered, so ok was
true, exit was 0, and the screen printed "Every changed line is executed by a
test." — while its own JSON, in the same response, reported `changedLines: 6,
executedLines: 1`.
try is what the installer prints as the next step, what QUICKSTART opens
with, and the only command available before setup. It is the product's first
word to every customer, and on the exact case Relay exists to catch it said the
opposite of the truth. It counts lines now: in the flag, in the block, in the
bar, and in the sentence. When some changed lines ran and some did not, it says
so with the number.
Rust could not reach PASS at all.
cargo-llvm-cov does not create the parent directory of its own
--output-path, so it exits 1 — and a non-zero exit from the coverage command
is read as "your tests failed". Every Rust repository was told to fix tests
that had all passed. One of four supported languages, broken from the first
run. The directory is now created for the one tool that will not make it, and
for no other stack.
A Python repository with no `.gitignore` could not pass either.
pytest rewrites src/__pycache__/*.pyc on every run, and each one counted as a
changed surface that no test executed — so a fully covered change came back
UNRESOLVED. Compiled bytecode is now excluded in code rather than in the
default .relayignore, because that file is never updated for repositories
that already have one. Lockfiles are still counted, deliberately: a dependency
version change really does alter behaviour.
An agent following the protocol could be driven into a dead end.
With a test runner but no coverage tool, next returned a receipt run
command carrying no --coverage. An agent doing exactly what it was told got a
valid signed receipt and exit 0, verified stayed at 0, the identical action
came back, and the third repeat reached NEEDS_HUMAN — where explain then
suggested the test did not reach the changed code. Nothing was measuring it.
The remedy was in the document the whole time, appended to reason, which the
protocol tells agents never to branch on. It is now a code: NO_COVERAGE_TOOL,
with SETUP_COVERAGE_TOOL as the action and a page of its own.
`relayevals check` could not see a broken policy.
On a repository whose relay.policy.json does not parse it reported
ready: true with nothing to do and exit 0 — while every verdict there
exited 78 and next exited 31. check is the signal the agent guide says to
gate on, so an agent doing as it was told walked into a wall it had just been
assured was not there. It reads the policy now.
The first-run notice named three of the five things it sends.
It omitted the event name and the per-repository hash. That hash is salted with
a random value that never leaves the machine and is meaningless anywhere else —
but it is transmitted, and a reader told only "a hashed machine id, the version
and the platform" would conclude otherwise. A notice that under-describes is
not consent.
Documentation, checked against a clean machine rather than memory.
The installer-verification procedure pointed at 0.6.7's manifest while 0.6.9
was served, so following it produced a mismatch on a perfectly good download —
and the same page said a mismatch means truncation. A mirror was advertised
that had never been published, sending precisely the reader who distrusts one
origin to a 404. The first code block did not work when pasted, because PATH is
set in a shell profile the running shell has already read past. The page said a
fresh install runs coverage_evidence: "declared" and that turning the real
gate on was a later step; setup has written "measured" since 0.5.0. It told
users with Node 20 to upgrade, when the installer brings its own Node 22.
TECHNICAL_ARCHITECTURE.md described telemetry as opt-in; it is opt-out.
MANUAL.md documented a producer trust <id> form the CLI rejects.
Three internal release-receipt documents were shipping a maintainer's absolute
home paths inside the public tarball — the same class of data .gitattributes
excludes .claude/ for, in as many words. They are excluded now, and a test
lists the archive and fails on any that survive.
Smaller, and each one real: --no-browser announced "Opening browser…" on
the headless machines the flag exists for. Every documented
/docs/states/<CODE> URL returned 404, because the route is keyed by the kebab
slug and the published table lists the enum; both forms work now. The site
served no X-Content-Type-Options, X-Frame-Options, Referrer-Policy or
Permissions-Policy, and token-bearing pages were cacheable and indexable.
Every upgrade left a full copy of the previous install behind — about 149 MB,
kept forever, and not named in the documented uninstall.
How these were found. An end-to-end verification of released 0.6.9 across
eight dimensions, each in a clean sandbox on a different machine, with every
reported failure re-run by an independent skeptic instructed to refute it.
Forty-eight findings were filed; eighteen were refuted and discarded, including
four that had been reported as critical. Thirty survived. This release fixes
twenty-eight of them.
Not in this release. relay:error:v1 still emits codes outside the
thirteen its documentation lists, and five commands still emit --format json
with no schema discriminator. Both are contract-wide changes to the
agent-facing surface and are being done as their own piece of work rather than
folded in here.
0.6.9 — 2026-08-12
A first run that could not show the notice was sending anyway.
setup prints a short notice the first time, before anything is transmitted,
and the function that does it says why in its own header: telling someone you
collected something is not consent; telling them before you do is the least
this can be.
Under --format json it skipped the print — correct, a human paragraph has no
place in a machine-readable stream — and then marked the notice shown and sent
regardless. --format json is what a coding agent runs. So the one caller who
cannot see a terminal was the one caller instrumented without being told, and
marking it shown made that permanent: every later run from a human terminal
found the flag already set and stayed silent.
It now returns instead. The cost is one deferred ping; the first run that can
actually show the notice shows it, and sending starts there.
`version:` in the GitHub Action pinned nothing.
It defaulted to 0.6.3 while 0.6.8 shipped, which is how this was noticed, but
the number was the smaller half. The step fetched /install.sh — whatever that
URL serves at the moment the job runs — and set RELAY_VERSION, which
install.sh has never read. Every run installed the current release regardless
of what the workflow asked for, and a pin written months earlier drifted
silently with each deploy.
Every release now publishes its own installer at
/releases/<version>/install.sh, and the action fetches that. A released
installer carries its version, tarball URL and tarball hash as literals, so
pinning it pins the whole chain. A version that never shipped answers 404
rather than quietly installing a different one. Backfilled for every prior
release that has an installer, so existing pins keep working.
Verified by pinning 0.6.7 on a clean machine and getting 0.6.7.
The privacy section was overstating what a team plan collects. It said
findings are recorded with their file and symbol names. They are not, and
cannot be: the endpoint validates every finding against a pattern with no
lowercase, no slashes and no dots, and the CLI drops the human-readable line
before sending precisely because it may quote a filename. A team plan records
what happened, never where.
0.6.8 — 2026-08-12
`curl … | sh` now says what to do instead of dying on a shell error.
The documented command has always been | bash, and the installer has always
declared #!/bin/bash. But | sh is muscle memory, and on Debian and Ubuntu
/bin/sh is dash, which has no set -o pipefail — so the second line killed
the install with
sh: 21: set: Illegal option -o pipefail
A shell error, about our own script, before the reader had seen a single word
from Relay. Confirmed on Debian 12 in a clean sandbox.
The installer now checks for bash before anything else. Handed a file it
re-execs bash on it, which covers sh install.sh and anyone who downloaded,
verified the hash, and then ran it. Arriving on stdin there is no file to
re-exec, and it will not fetch itself a second time to pipe into bash — the
bytes executed would then be a download nobody checked, which is the property
the published hash exists to give. That case prints the command to use and
exits without touching anything.
The guard is POSIX, because it has to parse under the shell that is already
wrong.
Billing: three ways a real purchase broke that no test could see.
Every Polar webhook had been rejected — 106 of them — because the signing key
is the literal secret from the dashboard, prefix included, and this verified
against the base64-decoded bytes the spec describes. Read out of
@polar-sh/sdk rather than guessed. Beyond that: order.created stored a
NULL amount on every invoice, because it read order.amount and Polar sends
total_amount; a failure mid-handler lost the purchase permanently, because
the dedupe row committed before the work and the retry answered "duplicate";
and a failed renewal kept the plan while advancing the paid-through date,
which made the only check that ends access push its own deadline forward.
A lapsed team could also still mint CI credentials.
An admin surface at `/admin`, founder-gated by an env allowlist of GitHub
numeric ids. Read-only apart from four audited writes, single-subject lookups
only, and every read of a customer's record recorded before it happens.
Nothing in it can reach a repository name, a file path, or a verdict's
contents.
0.6.7 — 2026-08-11
`relayevals login` stops asking you to arbitrate between signing keys.
A machine with more than one key got this, and it was indefensible:
```
This machine has several signing keys and this directory
is not set up for any of them.
probe2 producer:e5bab9e028a96ca…
local-default producer:f04e3d1ca51d528…
Name the one you mean: relayevals login --producer <name>
```
Producer ids on the login path — in the release that claimed to remove them
— and a question about internal identities nobody creates deliberately.
Login signs in a MACHINE, so picking its primary key is the product's job:
local-default (the name setup gives every key it creates), otherwise the
newest active key. Deterministic, and reversible with logout.
```
Links this machine's install pings to your account. Nothing else is sent.
Opening browser…
https://relayevals.com/device
Enter code RLAY-7QK2-M4XB
Waiting for authentication…
✓ Signed in as @sathwik
✓ Producer claimed
✓ 142 previous receipts attributed (3 repositories)
```
`key unlock` and `trust approve` still ask, deliberately. Both are
human acts whose wrong guess sticks: a passphrase cached against an
identity nobody meant, or a key approved that nobody meant, is not undone
by running the command again. Login costs one reversible command; those
cost trust. Silence is not always kindness.
0.6.6 — 2026-08-11
A failing vitest suite now names the tests that failed.
0.6.5 promised that and did not deliver it for the most common JavaScript
runner. relayevals try on a failing vitest project said "Relay could not
read test names from this runner's output" — there was no vitest parser at
all, and the dispatch table assumed jest's covered it. It does not; the
formats differ.
```
◆ Evidence
✗ npm test -- --coverage --coverage.reporter=lcov exited 1
Failing (2):
· upload > rateLimit blocks over the window
· upload > parseUpload rejects empty name
```
The parser reads vitest's FAIL file > suite > name blocks, falls back to
the × summary when a reporter suppresses them, and strips ANSI first —
colour between FAIL and the path would otherwise silently defeat the
match, and silence is exactly how this shipped.
How it was missed, and what changed about finding it. Every parser test
in the repository fed synthetic strings straight to a parser, and the suite
never runs the shipped CLI against a real vitest project — so the gap was
invisible from inside. It surfaced on a clean Debian box, installing from
the published tarball and running a genuine failing suite. The fixture in
the new test is captured from a real run rather than written by hand.
0.6.5 — 2026-08-11
A trust-repair release. Three failures a real user hit on a real machine,
each on a path where a first impression is the only impression.
Fixed — `relayevals login` never asks the terminal for anything
The reported session ended with a passphrase prompt and then a sentence that
welded a network failure to a key failure. Login now prints the browser
link, the code on its own line, and — on success — one fact per line:
```
Opening browser…
https://relayevals.com/device
Enter code RLAY-7QK2-M4XB
Waiting for authentication…
✓ Signed in as @sathwik
✓ Producer claimed
✓ 142 previous receipts attributed (3 repositories)
```
Nothing is typed. No producer hash appears — the /device page names the
machine before you approve it, which is what the hash was standing in for.
The passphrase prompt was never really about a passphrase. Login scanned
the data home for the first encrypted key in sorted order, ignoring which
producer the repository is configured for. On a machine with several keys it
quarantined on one the repository does not use, and key unlock then asked
for that key's passphrase — so a correct passphrase was rejected by a message
that was technically true about the wrong key. Resolution is repository-first
now, and both messages name the key and carry --producer into the command.
A repository pointed at a revoked key says so instead of binding it.
Fixed — checkout was sending fields Polar does not have
"Start a team" returned billing_unavailable for every possible cause.
Verified against Polar's live API spec: checkout takes external_customer_id
and seats; Relay sent customer_external_id and quantity. Unknown fields
are dropped silently, so the request was malformed in the one way that
produces no useful error. Failures are now classified by who repairs them —
credentials, a field we send, Polar being busy, Polar being down — each with
its own message, and the full detail in the server log.
Fixed — a failing suite names the tests that failed
try ran the suite and discarded its output, so the failing test names were
gone before anything rendered. It now shows up to five of them, the exit
code, and always two commands. A suite that never ran — a package manager
that is not installed — is its own state and says so, instead of claiming
tests failed and looping.
0.6.4 — 2026-08-11
The trust model gets its ceremony, and secrets stop living in files.
This release adopts the v7 build specification as the plan of record and
closes its Gate 1–2 hygiene items: a human now approves each machine's
signing key before the loop opens, new keys live in the OS keychain rather
than a 0600 file, no golden path takes typed input, and eleven trust
failures that shared one nameless sentence became six states that each name
their own cause.
Added — the one-time trust ceremony
After setup, the first relayevals next on any machine answers:
```
RELAY · one-time trust setup
NEEDS_HUMAN · TRUST_SETUP_REQUIRED
Relay needs you to approve this machine's signing key. Your agent
cannot approve itself — that separation is what makes every later
verdict worth trusting.
◆ Your move
relayevals trust approve 30 seconds · once per machine
```
Exit 40, on the first call by construction: the check sits before any
verdict work, so a precondition can never be discovered four layers deep.
relayevals trust approve is TTY-only and writes nothing when it refuses —
an approval created on the refusal path would be the gate approving through
its own back door. The record is per machine and per key: a regenerated
key needs its own approval, because the human approved a key, not a
directory.
The gate guards the loop — next, try — not the raw measurement
commands. verdict, check and replay still measure. And the marker is
cooperation enforcement, not cryptography: an agent with filesystem access
could forge it exactly as it could forge the policy baseline, but a forged
approval is deliberate misbehaviour that leaves an artifact. The teeth
against forgery stay where they live — member certificates and CI
attestations, signed by keys no agent holds.
Changed — the OS keychain is where a new signing key lives
A new producer key goes into the macOS keychain; unlock is silent and
Relay's terminal never asks. On disk, a private.keychain marker says where
it went. Existing private.pem files keep working — the resolver orders
.enc > .keychain > .pem, so a passphrase envelope always outranks the
keychain and a plain file is the last resort. Three honest exits from the
default: RELAY_KEY_STORAGE=file, no keychain on the platform, or a
keychain that refuses — each falls back to the 0600 file rather than
silently writing the key somewhere else. Windows DPAPI and libsecret are
follow-ups, not silent fallbacks.
A data directory copied between machines is now a named state: the marker
travels, the key by design does not, and the message says so with the fix.
Hardening migrates rather than duplicates — key unlock on a keychain key
seals the PEM in the passphrase envelope, verifies the round-trip, then
deletes both the marker and the keychain entry.
Fixed — eleven trust failures stop sharing one sentence
*"The approved receipt producer is missing, malformed, or fails integrity
checks"* is the plan's own canonical counter-example to invariant 11, and it
was still shipped: one error class, eleven throw sites, each of which knew
exactly what was wrong and discarded it. Now six states — missing, insecure
storage, malformed record, integrity failure, wrong repository, and an
approval no human made — each one sentence ending in a runnable fix.
The split surfaced two defects beyond wording. The "missing" leg had **no
code behind it**: the likeliest real failure, a policy naming a producer
nobody approved here, surfaced as a raw ENOENT. And one condition held
four different failures, so "approved for another repository" and "tampered
after approval" gave the same answer despite having different fixes.
Added — the four earned moments
Four one-line login nudges, each tied to the moment signing in becomes
concretely useful — first BLOCK→PASS, the report from week two, a
teammate's receipt you cannot verify, and the CI setup step — and each gone
permanently after three ignores. An ignore only counts when a human could
have seen the line: never in JSON, never without a TTY, never in CI. Being
signed in retires all four.
Added — `/pricing`, and a button that goes somewhere
The §13 copy verbatim, with "Start a team" landing on a real /team/new:
name the team, then Polar's hosted checkout. A CTA with nowhere to go would
have broken the copy rules before it broke anything technical.
Added — a documentation page per state
Every relay.next/1 document deep-links to /docs/states/<code>. Those 23
pages now exist, generated from one array, with a parity test in each
direction: a code the CLI can emit with no page fails the build, and a page
with no code behind it fails too. The link used to be an anchor precisely so
it could never 404; the pages are only safe to ship with that guard.
Added — `relayevals key unlock`, and the passphrase quarantine
The one place in the product a passphrase may be typed. TTY-only; verifies
the passphrase actually decrypts the key BEFORE caching it in the macOS
keychain; refuses non-interactive sessions with the escalation. On platforms
without keychain support it says so plainly — --passphrase-stdin remains
the automation path. Windows and Linux keychains are follow-ups, not silent
fallbacks.
Changed — login never asks
relayevals login no longer prompts for a passphrase, ever. A protected key
is answered by the OS keychain silently, or login stops with:
```
This machine's key is passphrase-protected (hardened profile).
Unlock it once, then re-run login: relayevals key unlock
```
The inline prompt this replaces is how at-rest key protection leaked onto a
golden path in the first place.
Fixed — one cause, one line, one exit
A corrupt key or wrong passphrase used to surface as *"Could not reach the
server: Could not decrypt the producer key"* — two unrelated failures in one
sentence. Login errors are now four distinct states with distinct exits:
| What happened | Exit |
|---|---|
| Network unreachable — "Relay works offline — logging in is optional" | 35 |
| Code expired — "relayevals login prints a fresh one" | 36 |
| Key locked → the key unlock state | 40 |
| Key fails integrity — named, and never silently regenerated | 40 |
Added — two named regression scenarios
- 33 · FALSE_PASS_REGRESSION — a signed receipt with exit 0 and no
coverage mapping can never become PASS. Pinned at BOTH layers: the receipt
cannot even be minted (generation refuses when an exit-0 command wrote no
report), and disjoint coverage still blocks as unexercised surface.
- 34 · TESTS_FAILED evidence — the state names at least one failing test
and hands back a runnable re-entry. A dead-end box shipped once; the
harness now forbids the shape.
Added — two release gates in CI
- The artifact allowlist.
git archivemay produce exactly eighteen
named top-level entries — one assertion, toEqual, no pattern-matching. A
new internal file at the root now fails the suite instead of shipping,
which is how four releases carried the operational memory file without
anyone noticing. The named files from that incident are additionally
asserted absent at any depth, forever.
- Golden paths with stdin closed.
try,next,setupandloginare
spawned as the real built CLI with stdin closed; hanging on a read or
printing a prompt fails the build. The TTY-gated commands (trust approve,
policy accept, key unlock) are deliberately absent — refusing this
environment is their job.
Plan of record
docs/PLAN.md is now the v7 specification (final merge of v4–v6). It stays
out of the published tarball — the plan is the one document that never ships.
0.6.3 — 2026-08-10
Relay becomes something you can sign into. Until now every install was an
island: a verdict was trustworthy on the machine that produced it and nowhere
else. This release adds an account, a team, and a way for CI to verify a
teammate's receipt — without changing what a local verdict does.
The local verdict is untouched. It still runs offline, still opens no
socket, and still reads no entitlement. That is invariant 1, and
tests/telemetry-fence.test.ts walks the real import graph from every verdict
entry point to prove it rather than asserting it.
Added — sign in from the CLI
```
relayevals login
```
An RFC 8628 device flow: the CLI prints a short code, you approve it in the
browser at /device, and the token lands on this machine. --no-browser
prints the URL instead, for SSH and containers. The token survives a restart,
is stored 0600 in a 0700 directory, and is scoped by server origin so a staging
login cannot overwrite a production one.
relayevals logoutrevokes the token server-side, then removes it
locally. Revoke-then-delete, in that order: reversed, a failed network call
strands a live token that nobody can revoke, because the only copy of it was
on the machine that just deleted it. Logout still works offline — it says
plainly that the token is still live rather than reporting a revocation it
did not get.
- Logging in retroactively attributes the verdicts this machine already
produced. That is the reason to do it.
Added — teams, and a receipt you can verify
- Teams, invites and an audit log. Ownership lives in
teams.owner_user_id
— one row Postgres can lock — rather than a count of owner rows, which write
skews under concurrency.
- Member certificates. A signed statement that a producer key belongs to a
team, so CI can verify a teammate's receipt with no key exchange. The
binding is self-certifying: the producer id is the sha256 of the public key
in the certificate, so the only (id, key) pairs anyone can write are true
ones, and a planted bundle is harmless.
- Team policies, versioned append-only, with a strictness order that is
explicit about which pairs it cannot order.
Added — the merge gate
- `relayevals ci auth` exchanges a GitHub OIDC token for a short-lived
credential. No secrets in the repository, and a grant cannot follow a
rename: repositories are identified by numeric id, never by name.
- `relayevals/verify-action@v1` runs the gate as a workflow step.
- Policy drift between the repository and the team is reported and resolved
rather than silently preferred one way.
Added — telemetry, off with one variable
setup — and only setup — sends a hashed machine id, the version, and the
platform. Never your code, file names, repository names, or task text: there
is nowhere in the schema to put them. The notice prints before the first
transmission, and RELAY_TELEMETRY=0 leaves nothing on disk at all rather than
merely stopping the send.
These pings are anonymous until you run relayevals login, which links this
machine to your account. The notice says so.
Fixed
- `verdict.pass` was rejected by the telemetry endpoint, because a path
heuristic fired on the event *name* rather than a path. Fields whose schema
already constrains them are now exempt, pinned by a test that reads the
schema source.
- A certificate could never be renewed — the insert conflicted against the
one-live-cert index and did nothing, so every team's trust graph would have
died seven days after first login.
- The flaky suite was a compiler in the test budget, not a race: a module
cache rebuilding inside a per-test temporary directory, 91 MiB and 40 s
against a 60 s ceiling. Every failure was a timeout.
- The installer's
install-record.jsonclaimed a signed single-file binary was
"planned for" the version being installed — in every release since 0.3.0. It
now states plainly that one is not yet available.
Verification
| | macOS arm64 |
|---|---|
| CLI suite | 2216 passed · 0 failed · 26 skipped |
| apps/web suite, against a database | 124 passed · 0 failed |
| Built and tested from the extracted tarball | npm ci + tsc + suite |
| Tarball reproducible from the tag | byte-identical git archive |
Not done
- The web service needs server-side setup — database, OAuth app, billing
account, environment. The CLI works without any of it.
verify-action@v1is not published to the GitHub Marketplace.- Certificate revocation is not instant for a verifier that is offline; a
certificate stops being accepted when it expires.
0.6.2 — 2026-08-10
Relay hands the finding back, with the evidence behind it. On a BLOCK the
agent gets what only Relay has — the measurement that produced the verdict and
the exact condition that clears it. Everything else it already had.
Added — `relayevals fix`
```
RELAY · fix prompt (copy everything below to your agent)
────────────────────────────────────────────────────────
Relay blocked this change. Here is the evidence:
• src/pay.js · 3 of 4 changed line(s) never executed (lines 5-7)
• receipt "test" (npx c8 --reporter=lcov npm test) exited 0
and covered 4 line(s)
• policy requires measured coverage of all changed executable
lines (relay.policy.json, committed)
Your task:
1. Add or update the smallest test that actually executes the
changed code named above. Do not modify unrelated code.
2. Do not edit relay.policy.json. Do not write, edit, or
fabricate receipts or coverage reports. …
3. Produce evidence:
relayevals receipt run --name tests --kind test …
4. Continue the loop until the state is PASS:
relayevals next --format json
Resolved when: the changed lines above are executed by the
evidence and SURFACE_UNTESTED disappears.
────────────────────────────────────────────────────────
```
No LLM anywhere. Deterministic templating, one template per finding code,
zero inference cost and zero new ways to be wrong. The agent already has the
code and the conversation; paraphrasing them would add nothing.
- The same text rides on
relay.next/1as an optional `fix_prompt` field,
present on BLOCK and UNRESOLVED only — additive, so the schema stays /1
and existing consumers are untouched.
- Templates for
SURFACE_UNTESTED,TESTS_FAILED,EVIDENCE_MISSING,
RECEIPT_STALE, NO_TEST_RUNNER, POLICY_CHANGED, EVIDENCE_UNSIGNED.
Each states evidence, the task, and the resolution criteria.
- The three forbidden rules are spelled out as instructions rather than as
enum names, because a pasted prompt arrives in a session with no schema.
- Every prompt ends with the loop re-entry, so a fresh agent self-connects.
fixis read-only: asking what to do never advances a loop counter.- BLOCK/UNRESOLVED terminals gain
Or hand it back: relayevals fix;
NEEDS_HUMAN renders Your move with explain / fix / next.
Fixed — a wrapped command is not a command
The first cut word-wrapped instruction text, which broke the shell commands it
contained mid-flag — in a document whose entire purpose is to be copied. Both
relayevals … and git checkout -- relay.policy.json were affected; the
latter is the one instruction a policy-blocked user most needs to paste.
Commands now sit unwrapped on their own line, and a regression test asserts it.
Changed
AGENTS.mddescribedrelayevals eval, the alpha command, and a two-cycle
fix rule that the soft/hard counters replaced. It now documents the next
loop, fix, and the exit-code contract.
- The v3 master build plan lands as
docs/PLAN.md.
Verification
| | macOS arm64 | Linux x86_64 |
|---|---|---|
| Suite | 113 files · 2060 passed · 0 failed | 1839 passed · 0 failed |
| Agent loop, driven by returned actions | ✓ | SETUP_NEEDED → READY → UNRESOLVED(20) → BLOCK(10) → PASS(0) |
| fix at a BLOCK | ✓ | ✓ evidence + task + criteria, commands unwrapped, counters unmoved |
| Scenario 32 — the prompt's own instructions clear the finding | ✓ | ✓ PASS 4/4 |
The Linux suite reports 112 of 113 files: the same load-dependent vitest worker
exit characterised in 0.6.1 — not a specific file, absent on macOS, and a
478 MB box with a fork per file. No test failed in any run.
0.6.1 — 2026-08-09
One command for agents. relayevals next replaces the guess-which-command
problem with a state machine: it resolves the repository into exactly one state
and returns exactly one allowed action. Everything an agent needs is a five-line
loop that never mentions a second command.
```text
This repo uses Relay for merge verification.
Loop:
1. Run: relayevals next --format json
2. Do exactly the returned action. Nothing else.
3. Repeat until state is PASS, READY, or NEEDS_HUMAN.
Never edit relay.policy.json. Never report a task complete
unless Relay's state is PASS.
```
Added
- `relayevals next` — the agent interface. Thirteen states, each with one
action, a forbidden list, attempt counters, and a docs link. Exit codes are
a stable contract: 0 PASS/READY/SETUP_NEEDED, 10 BLOCK, 20 UNRESOLVED,
30–39 environment, 40 NEEDS_HUMAN.
- `relayevals explain` — plain-language recovery for humans, after
NEEDS_HUMAN. Read-only: asking what is happening never advances a counter.
- `relayevals policy accept` — re-baselines
relay.policy.json. Interactive
only; an automated session cannot confirm a policy change.
- Hash-pinned policy. Any drift from the accepted baseline blocks with
POLICY_CHANGED until a human confirms it. No weakness classifier: deciding
which edits are "weaker" needs a correct ordering over every field forever,
and one missed case is a bypass.
- Fix-loop escalation. Three identical attempts, or ten in total on one
finding, hand control back with FIX_LOOP. Nothing resets the ceiling except
the finding clearing.
- Session lockfile — concurrent sessions get
SESSION_LOCKEDinstead of
racing each other's evidence. A lock whose process is gone clears itself.
- `--producer` is now optional on `receipt run`, defaulting to the producer
in .relay/config.json.
Fixed — receipts went stale against their own side effects
subject.snapshot_id recorded the tree *before* the command ran while
surface_hashes described the tree *after*, so any command writing an
untracked file — every coverage run on a repository that has not ignored its
report directory — invalidated its own receipt. The id now binds post-run.
The mid-run race the pre-run id caught by accident is now caught deliberately:
RECEIPT_SURFACES_MUTATED fires when changed code is edited *while* evidence
runs, which is never sound, and is distinct from the repository merely being
written to, which is normal.
Fixed — an untracked file invalidated every receipt
snapshot_id is sha256(HEAD + status_hash), and git status reports
untracked paths regardless of .relayignore — Relay's filter, which git has
never heard of. Creating a scratch note staled every receipt in the repository
and blamed the user for editing code they had not touched. HEAD is now recorded
and compared on its own; the status half is not compared at all.
Fixed — a verdict could move while its inputs held still
discoverReceipts broke mtime ties on a content-addressed filename, so two
receipts of one kind written inside a millisecond could resolve to the older
one. Ordering now prefers the receipt's own signed finished_at.
Verification
The 31 adversarial scenarios of the onboarding plan run as tests: policy
tampering, forged receipts, deleted identities, loop thrashing, concurrent
sessions, and the human-recovery path.
| | macOS arm64 | Linux x86_64 |
|---|---|---|
| Suite, built from the release tarball | 112 files · 2028 passed · 0 failed | 111 of 112 files reported · 1823 passed · 0 failed |
| Agent loop, driven only by returned actions | ✓ | SETUP_NEEDED → READY → UNRESOLVED(20) → BLOCK(10) → PASS(0) |
| Adversarial paths | ✓ | ✓ — policy (both directions), TTY gate, forged evidence, FIX_LOOP, all five environment states |
On Linux the default reporter loses one vitest worker, so the file line reads
105 passed + 6 skipped of 112. No test failed in any run. It is not a
specific file crashing — a --reporter=json run on the same box reported all
112 — and not worker contention, since it survives `--no-file-parallelism
--maxWorkers=1`. It does not happen on macOS (112/112, 0 errors). The sandbox
has 478 MB of RAM and vitest forks per file, so this reads as resource
exhaustion rather than a defect. Recorded rather than dismissed; confirm on a
Linux runner with real memory before calling the suite fully green there.
0.4.5 — 2026-08-07
Two bugs found by testing on real Linux distributions, in an E2B sandbox
rather than on a developer laptop. src/ is unchanged.
Fixed — verification silently unavailable on minimal images
The installer hashed downloads with shasum, which is a Perl script. macOS
ships it and the Debian image happened to have it; Alpine and many slim images
do not, so the install died with:
```
main: line 157: shasum: command not found
```
That is the step that makes a piped installer trustworthy, and on those systems
it could not run at all. Hashing now tries sha256sum (coreutils), then
shasum, then openssl — and refuses to continue if none is present,
rather than proceeding unverified.
Fixed — musl systems now fail in two seconds, with the reason
On Alpine the installer downloaded 40 MB of Node and then died on cryptic
loader errors. Two distinct causes, both now handled:
- Official Node binaries are glibc-linked and cannot run on musl. The
installer now detects musl and uses the sha256-pinned unofficial musl build.
- But the build still cannot succeed: TypeScript 7 ships a native compiler
and publishes no musl variant (@typescript/typescript-* has none), so there
is nothing to substitute. musl is now detected up front and refused with an
explanation and exit 78, before anything is downloaded.
Verified
- Debian 12, x86_64, non-root, node v20 present — full install and every
feature: measured coverage BLOCK with exit 1, Tree-sitter grammar download,
semantic exemption, producer protect, locked-key exit 77, supervised
receipts, artifact approve/verify, and tamper rejection on all three checks.
- Alpine 3.21 (musl) — clean refusal with the real reason.
Ubuntu, Fedora and Arch take the same glibc code path as Debian and are not
separately verified.
0.4.4 — 2026-08-07
No password, no prerequisites but git. src/ is unchanged.
0.4.3 bootstrapped Node but still installed into /usr/local, so it needed
sudo — and the bootstrap ran *before* elevation and died:
```
mkdir: /usr/local/libexec: Permission denied
error: cannot create /usr/local/libexec/relay/node
```
sudo curl ... | bash does not help either: sudo applies to curl, not to the
bash reading its output.
Changed — installs into your home directory
Everything now lives under ~/.relay, with the command linked into
~/.local/bin. No sudo, no password, nothing written outside `$HOME`. An
install that needs administrator rights to try a tool is a tool most people do
not try.
~/.local/binis added to PATH in your shell rc when it is not already there.RELAY_HOMEoverrides the location.- A pre-existing
/usr/localinstall is detected and reported, because it would
otherwise shadow the new one on PATH.
Fixed
- Replacing an install no longer deletes the bootstrapped Node. The previous
step moved the whole root aside, including the runtime it had downloaded
seconds earlier, and npm vanished mid-install. Re-installs now reuse it
instead of re-downloading ~50 MB.
- `ensure_path` was defined but never called, so PATH was silently never
wired. Defining a helper is not the same as running it.
Verified end to end as an unprivileged user with no Node, no npm, no brew and
no sudo: bootstrap → verified download → npm ci → tsc → working
relayevals --version, then a re-install that reused the runtime.
0.4.3 — 2026-08-07
**Installs on a bare machine, and fixes the lockfile desync that was breaking
npm ci all along.** No behaviour changes; src/ is unchanged.
Fixed — the actual cause of the install failures
package-lock.json was out of sync in a way only some npm versions notice.
npm 11 prunes optional transitive dependencies it considers inapplicable to the
current platform; npm 10 — the version bundled with Node 22 LTS — demands them
and refuses:
```
npm error npm ci can only install packages when your package.json and
package-lock.json are in sync.
npm error Missing: @emnapi/core@1.11.1 from lock file
```
The lockfile is now regenerated with npm 10, making it a superset that
satisfies every supported npm. Verified passing under both npm 10.9.8 and
11.6.2. The same shape nearly broke 0.3.0 (@emnapi/wasi-threads), so a
regression guard now asserts every required dependency has a lockfile entry.
Added — the installer bootstraps its own Node
A fresh machine has no Node and often no Homebrew. The installer used to print
brew install git node and exit — an instruction that cannot be followed on a
machine without brew.
It now installs a private Node.js when none is usable:
- Official build from nodejs.org, sha256-pinned against the digests
published for that exact release. An installer that pipes an unverified
binary onto a machine would contradict the product.
- Installed under Relay'"'"'s own directory. Your system Node and PATH are
untouched.
- The wrapper pins the Node the install was built with, so
relayevalskeeps
working afterwards.
- A previously bootstrapped copy is reused rather than re-downloaded.
Verified end to end on a simulated bare machine: no node, no npm, no brew →
verified download → npm ci → tsc → working CLI.
git is still required, and cannot be bootstrapped away: Relay binds evidence
to git snapshots. On macOS the message now points at xcode-select --install
rather than a package manager that may not exist.
0.4.2 — 2026-08-07
Fixes an install that failed with a misleading error. No behaviour changes.
A customer's npm ci failed while standing in a correctly extracted tree,
reporting ENOENT ... open '/Users/<them>/package.json' — npm had resolved its
prefix from their home directory rather than the staging tree.
Fixed
- `npm ci` now targets the staging tree explicitly (
--prefix "$tmp"),
making the working directory irrelevant. cd "" succeeds silently in bash
and leaves you where you were, so any path by which the staging variable came
out empty ran npm against the user's own directory. Naming the target removes
the whole class of failure rather than one route into it.
- The installer asserts the extracted tree is present before invoking npm,
and prints the directory contents when it is not.
- npm failures now show the HEAD of npm's output, plus node/npm/platform.
0.4.1 showed the tail — but npm prints the reason first and then dumps a long
usage block, so it displayed the help text and discarded the cause. It
reported a failure while still hiding why.
Note on 0.4.1: the os/cpu removal in that release was a real fix — those
fields genuinely broke every Intel Mac and every Linux user — but it was not
the cause of the report that prompted it. That customer is on arm64. The
diagnosis was inferred from an error message the installer had suppressed,
which is the defect this release finishes fixing.
0.4.1 — 2026-08-07
Fixes an install that failed for most users. No new features.
Fixed
- `npm ci failed` on every machine that is not an Apple Silicon Mac.
package.json declared os: ["darwin"] and cpu: ["arm64"]. npm enforces
those on the ROOT package, so npm ci — which the installer runs — died with
EBADPLATFORM on every Intel Mac and every Linux box:
```
npm error code EBADPLATFORM
npm error notsup Unsupported platform for relayevals@0.4.0:
wanted {"os":"darwin","cpu":"arm64"} (current: {"os":"darwin","cpu":"x64"})
```
Those fields describe the SEA binary, built by prepack, which a source
install never runs. They were never a statement about the source package.
Removed.
- The installer hid the reason. It ran
npm ci --silentand reported only
error: npm ci failed., discarding the one line naming the cause and making
the break unreportable. npm's output is now captured and replayed on failure.
Both are covered by regression guards in tests/install-portability.test.ts.
0.4.0 — 2026-08-06
Evidence an agent cannot manufacture. Reframed around the fact that Relay's
user is an agent — simultaneously the subject of the evaluation, the producer of
the evidence, and the consumer of the verdict.
Nothing breaks on upgrade. Every new gate is opt-in and every new policy field
defaults to 0.3.x behaviour. See [MIGRATION_0_3_TO_0_4.md](docs/MIGRATION_0_3_TO_0_4.md).
Added — measured coverage (the headline)
receipt run --coverage <format>:<path>folds a real coverage report into
the receipt before signing. lcov (jest, vitest, c8, nyc, coverage.py,
cargo-llvm-cov, JaCoCo) and go-coverprofile.
coverage_evidence: "measured"requires the changed lines to have
executed. A --surface declaration the report contradicts now produces
SURFACE_UNTESTED naming the lines, instead of a PASS. declared remains
the default; 0.5.0 flips it.
- A report predating the command is recorded with
generated_after_command: false and treated as absent.
COVERAGE_MISSINGis UNRESOLVED, never BLOCK — the code may be fine and the
harness merely unwired.
Added — semantic surfaces
relayevals grammar install|list. Grammars are sha256- and byte-count-pinned
.wasm fetched on demand, not npm dependencies (~65 MB avoided), verified on
install and on every load.
- Comment, formatting, and import-reordering changes stop demanding evidence.
Anything unclassifiable still does.
SYMBOL_UNTESTEDnames the function and its lines rather than the file.CHANGE_TEST_ONLY(opt-in) flags a change touching tests and no source.SEMANTIC_UNAVAILABLEforrequire_semantic, as UNRESOLVED.
Added — the agent loop
- Every verdict carries
attempt, withresolved_since_previousand
introduced_since_previous.
ATTEMPT_THRASHINGwhen the same finding survivesthrash_threshold
consecutive attempts that each changed the code. Its advice is to escalate.
verdict --explain <code>returns a machine-readable remediation record for
every code, including blocks_the_change.
Added — trust hardening
relayevals supervise -- <command>runs the agent as a child and signs
receipts for commands Relay itself executes. receipt run routes through it
automatically.
producer protectencrypts the producer key at rest (scrypt + AES-256-GCM).
After it, receipt run alone cannot sign (exit 77). The passphrase is
accepted only on stdin — argv and environment are both readable by any
same-UID process.
require_supervisedpolicy.artifact approvesigns a verdict with the authority key;artifact verify
checks signature, decision hash, and **re-derives the verdict from the
artifact's own evidence** — offline, no key required.
- Receipts carry a
relay:origin:v1claim recording how they were produced and
the key protection that actually applied.
Added — documentation
- [AGENT_PROTOCOL.md](docs/AGENT_PROTOCOL.md) is the new canonical reference,
including a required "What Relay does not prove" section. A test enforces
that every code and schema appears in it.
Changed
decision_hashno longer coverscreated_at,attempt,signed, or
signature. Two identical decisions now share an identity, which is what
makes attempt chaining answerable. A hash pinned from 0.3.x will differ.
- Policy is
relay:repo-policy:v2. v1 documents are read and upgraded in
memory, keeping their declared policy_id.
Fixed
community-alpha-durable-evidenceasserted the message of a check it was not
exercising — it tampered with artifact_file.path while naming the check that
reads original_temporary_path. Corrected, and a case added for the check
that was never actually tested.
0.3.1 — 2026-08-06
Bug fixes for the 0.3.0 upgrade path. No new features.
Fixed
- An upgrading repository's first verdict came back `UNRESOLVED`, naming
relay.policy.json— a file Relay had just written itself — as an uncovered changed surface. In CI that is exit 2 and a failed build.
0.3.0 excluded Relay's own metadata by adding it to the default .relayignore. But setup never overwrites an existing .relayignore, so the exclusion reached new repositories only; every repository set up under 0.2.0 kept an older file. The exclusion now lives in the surface collector itself, where a stale config file cannot defeat it and a user cannot opt back in. Covers relay.policy.json, .relayignore, and everything under .relay/.
- The installer banner claimed it clones a repository. It does not, and never did in 0.3.0 — it downloads a sha256-pinned tarball and verifies it before running anything. The text was inherited from the pre-tarball 0.2.0 installer.
- The installer said a signed binary was "planned for 0.3.0" while running as 0.3.0. Now states plainly that one is not yet available.
- The installer banner printed twice. The
curl | bashfix restages and re-execs the script, and the child reprinted it. The re-exec is now marked so the banner prints once.
0.3.0 — 2026-08-05
The verdict engine.
0.2.0 could produce signed evidence but could not judge it — eval always returned UNRESOLVED. 0.3.0 answers the question: given this task and this evidence, is the work done?
Added
- `relayevals verdict "<task>" --receipt <path>` — returns PASS / BLOCK / UNRESOLVED with typed reasons and runnable next actions. Exit codes 0 / 1 / 2 gate a pipeline with no glue code.
- `relay.policy.json` — a committed, reviewable verdict policy at the repository root: mandatory receipt kinds, snapshot-match requirement, and whether uncovered surfaces block. Written by
setup, defaults tocoverage_enforcement: "warn". - Changed-surface coverage — the union of committed divergence (against the merge base) and uncommitted work, mapped against the
--surfaceglobs each receipt declares. A receipt covers a file only if its signature verifies *and* its snapshot matches; a stale receipt covers nothing. - Signed failing-assertion claims — jest/vitest, pytest, mocha, go, and cargo output is parsed at
receipt runtime and written into the receipt, so a BLOCK names the failing assertions withfile:linefrom evidence inside the signature. An unrecognised framework still yields one assertion rather than a silent "tests failed". - Requirement extraction — deterministic and offline by default;
--extract-requirements llmis opt-in, discloses that the task string leaves the machine, and caches results.--requirements <path>supplies a manifest instead. - Interactive `setup` — prints what it will create and asks for confirmation, gated on a real TTY.
- Enhanced `receipt run` output — working directory, human-readable timeout, allowlisted env variable *names*, repository unchanged/mutated, trust status.
- `docs/MIGRATION_0_2_TO_0_3.md`, `docs/internal/DEFERRED_TO_0_3_1.md`.
Fixed
- `.relayignore` was never applied.
setupwrote the file and no code read it, despite its own header claiming its patterns were "excluded from Relay's changed-surface inventory". It is now enforced. Gitignore-style negation (!foo) is rejected with an error rather than silently ignored, since dropping it could hide a changed surface. - Git stderr leaked into user output. Probes that are expected to fail —
@{upstream}on a branch with no upstream — printedfatal: no upstream configured, making a handled condition look like a crash.
Changed
- `setup` prompts when stdin is a TTY. Scripted and CI use is unaffected: without a TTY it proceeds without prompting rather than blocking.
--yesforces the old behaviour. setupwrites and reportsrelay.policy.json;.relayignoredefaults now exclude Relay's own metadata, which is reviewed in pull requests rather than covered by a test receipt.- First runtime dependency:
@anthropic-ai/sdk. Loaded lazily, so the default offline path never parses it.
Not in this release
- Verdict artifacts are unsigned. Every artifact carries a literal
"signed": false;decision_hashis an integrity check, not a signature. Receipts are the transferable evidence — re-run the verdict rather than trusting one you were handed. Signing arrives in 0.3.1. evalstill dispatches to the 0.2.0 engine and still returnsUNRESOLVED. It is unchanged so existing scripts keep working; useverdict.- See
docs/internal/DEFERRED_TO_0_3_1.mdfor the full deferred list.
0.2.0 — 2026-08-05
A signed evidence channel for AI-generated code changes.
0.2.0 is the "receipt loop" release. Alpha 0.1.0 could only ever reach UNRESOLVED because the CLI had no way to produce, sign, or verify evidence. 0.2.0 gives you:
- A one-command bootstrap for local, CI, or attested (WebAuthn) identity.
- Signed Ed25519 receipts bound to a deterministic git snapshot.
- Offline
receipt replaythat re-verifies signatures and detects stale snapshots without re-executing. - A
checkcommand that emits typednext_actions[]for every gap.
What 0.2.0 does not yet include (roadmap for 0.3.0):
- A verdict engine that returns PASS from
relayevals eval. Today'sevalstill traces toUNRESOLVED— Sprint 9.5 is the work that turns receipts into a PASS/BLOCK verdict. Usereceipt run+receipt replayfor the trusted-evidence surface; treatevalas the alpha's exploratory command. - macOS Keychain integration for producer/authority private keys (currently file-backed at mode 0600 under
~/.local/share/relayevals/). - Natural-language task→requirement extraction, changed-surface coverage mapping, and artifact-level
inspect/replay/approve --touch-id.
0.2.0 — details (Sprints 1–10)
Turns the Alpha 0.1.0 UNRESOLVED-only surface into a working local + CI producer/receipt/replay loop.
Added
relayevals setup --profile local— bootstraps a local Ed25519 authority + producer and a repo-local.relay/config.json. Idempotent;--forcereplaces.relayevals setup --profile ci --non-interactive— imports a pre-provisioned authority public key and producer private key from env vars, files, or inline values. Refuses to run without--non-interactive.relayevals setup --profile attested— bootstraps the attested profile from an enrolled WebAuthn credential (coverage enroll). Stores a public-key-only attested-authority descriptor pinning the credential id, aaguid, attestation format, and attestation-object hash; generates a local producer keypair; writes config withassurance_profile: "attested". Rejects--non-interactive. Idempotent per credential.relayevals check— readiness summary with non-zero exit when unconfigured.--format jsonemits typednext_actions[](code, reason, resolution scope, runnable commands, policy rule).relayevals receipt run -- <cmd>— sandboxed child runner. Captures exit code, wall-clock duration, stdout/stderr 64 KiB excerpts + full-stream SHA-256, pre- and post-execution git snapshots, and signs the whole envelope with the producer's Ed25519 key.relayevals receipt inspect <path>— human-readable summary of a receipt.relayevals receipt verify <path> --public-key <pem>— cryptographic verification against a supplied key.relayevals receipt replay <path>— offline re-verification with typed reason taxonomy (SIGNATURE_INVALID,RECEIPT_ID_MISMATCH,KEY_ID_MISMATCH,STRUCTURAL_INVALID,PRODUCER_KEY_UNRESOLVED) and snapshot state (match/stale/no-repo/not-checked). Public-key resolution falls back from--public-keyto the local producer store.relayevals producer {init,list,export-public,revoke,rotate}— local producer lifecycle.config pathanddoctor env— surface the data home and environment posture.
Changed
- Native runtime boundary rejects a broader
DANGEROUS_ENV_NAMESset (NODE_OPTIONS,NODE_PATH,SSLKEYLOGFILE,LD_PRELOAD,DYLD_*, …) beforenode::Start, withRELAY_STRICT_ENV=1retained as a backward-compat shim. receiptschema is a locked v1 envelope (schemas/receipt-v1.schema.json), signed and canonicalized per RFC 8785.- Repository config is a locked v1 envelope (
relay:repo-config:v1).
Documented
docs/QUICKSTART.md,docs/CI.md,docs/TRUST_MODEL.md,docs/MIGRATION_FROM_ALPHA.md.
Fixed (alpha bugs)
- Launcher no longer requires an exact-path match on
execPath; canonical realpath comparison is sufficient. - Setup rejects untrusted
NODE_*/SSLKEYLOGFILE/LD_PRELOAD/DYLD_*at import time. - macOS AppleDouble (
._*) files no longer contaminate the changed-surface inventory (via.relayignoredefaults).
0.1.0 — Community Alpha (unresolved-only)
Initial ad-hoc-signed macOS arm64 SEA published via curl -fsSL https://relayevals.com/install.sh | bash. The binary loaded, but every code path terminated in UNRESOLVED because no receipts, producers, or authorities could be produced or verified. See [docs/COMMUNITY_ALPHA_ACCEPTANCE.md](docs/COMMUNITY_ALPHA_ACCEPTANCE.md).