Comparison
Relay vs just running CI
Your CI already runs the tests. The question it cannot answer is whether those tests executed the lines your agent just changed — and that gap is exactly where AI-generated code slips through.
Facts checked against official docs on 2026-08-18
What plain CI is
A standard CI pipeline runs the test suite on every push and reports pass or fail. It is necessary, cheap, and universal — and it measures the suite, not the change. A green run means the tests you already had still pass; it carries no information about whether anything executed the new lines.
Pricing: Whatever your CI already costs.
Side by side
| plain CI | Relay | |
|---|---|---|
| Runs your tests | Yes | Yes — and wraps the run in a signed receipt |
| Fails when existing tests break | Yes | Yes |
| Knows which changed lines executed | No — a suite can pass while 0% of the diff ran | Yes — that intersection is the core measurement |
| Tamper-evident evidence | No — logs can be re-run, edited, or cherry-picked | Yes — Ed25519 signatures + a snapshot hash binding evidence to the exact tree |
| Tells the agent what to do next | No — a red build is a message to a human | Yes — a machine-readable action, flags pre-resolved, until PASS |
| Gate on the measurement | Coverage thresholds, if configured, measure the project — not the diff | The verdict is specifically about the changed lines |
The honest take
This is not an argument against CI — Relay runs inside it (`relayevals verdict` exits with the decision, so branch protection acts on it). The argument is about what a green build means. Before agents, "the suite passed" and "the change was exercised" were usually the same fact, because the person who wrote the code also wrote a test that called it. Agents broke that correlation: they produce code and sincere-sounding claims at a rate that makes "green" a weaker signal than it used to be. Measuring the diff directly restores the signal.
See what Relay measures on your own repository — one command, no setup, writes nothing: relayevals try. Docs · The full guide to verifying AI-generated code