An agent-operated production codebase

More than eight in ten commits across the two systems written up here — AgendaProfe, carrying real bookings and real money, and a live transit system in pre-launch — were written or co-written by a coding agent. The figures in the header are counted from those repositories’ histories, not estimated: a commit counts as agent-attributed when the author or a co-author trailer names one.

That sentence tends to produce one of two wrong reactions. The first is that the codebase must be a mess. The second is that the engineering must be mostly automatic now. This case study is about why both are wrong, and about the part that turned out to be genuinely hard — which was not the generation.

What agent-operated actually means

It does not mean unattended. Every change is directed at something specific, reviewed, and gated before it reaches anyone real. What it means is that the typing is no longer the job. The job is deciding what to build, giving an agent the context it cannot infer, and verifying what comes back — and the last of those is where nearly all of the engineering effort now lives.

Concretely: development runs on a remote server, with between three and ten agent sessions working in parallel, each in its own git worktree on its own branch. Direction is short written instructions — sometimes dictated from a phone. A session ends in a pull request like anyone else’s, and the pull request meets the same gates a human’s would.

The economics of this are lopsided in a way that took a while to internalise. Generation is effectively unbounded — ten sessions produce diff faster than any one person can meaningfully read. Review attention is the fixed resource. So every structural decision in the practice is really the same decision: how to spend less attention per change without trusting any single change more.

The failure mode is plausible code

Bad code is not the risk. Bad code is caught by the same compilers, tests and reviews that always caught it.

The risk is plausible code: a change that reads correctly, passes a cursory review, and encodes a misunderstanding that surfaces weeks later as a production incident. Volume makes this strictly worse, because plausibility is exactly the property agent output optimises for, and volume is exactly what agents provide.

One real example. A mobile test flow tapped a submit button after typing a code — reasonable, explicit, correct when written. Then the screen gained auto-submit. The flow’s tap no longer failed; the framework computed the tap’s coordinates from a stale snapshot and dispatched it after the app had already navigated away, landing it on whatever now occupied those pixels on the next screen. Every artefact involved was individually plausible. The system lied only in combination.

That class of failure — individually plausible parts, wrong whole — is what the rest of this write-up is organised against.

Gates that do not rely on attention

The principle: a tired human reviewer is the least reliable component in the loop, so nothing important is allowed to depend on one.

Agent sessions

3–10 in parallel

Worktree per branch

pull request

Per-PR checks

Types · lint · unit · ratchets

+ context verification

merge

Preview

Disposable DB branch

promote — a decision, not a push

Full gate

Integration · E2E · native guard

Against the exact candidate

fast-forward

Production

Auto-deploys on advance

The individual mechanisms matter more than the shape. All four are AgendaProfe’s, and its own case study describes how each is built; what belongs here is why an agent-operated codebase needs them in particular.

Ratchets rather than thresholds. Anything that could have been a global number is enforced against the diff instead — coverage on changed lines, lint rules that fail only on new violations. The reason is volume. A threshold is a number someone has to defend every time it moves; a ratchet scales with the diff rather than with my patience, which is the only property that survives ten sessions merging in a day.

Guards for breakage no review can see. The mobile release check is the clearest case: a build is refused when native code changed without a matching runtime version bump, because the crash it prevents happens only on real devices, after release. Agents did not invent that failure mode. They raise the rate at which harmless-looking diffs arrive at it, and reading more carefully cannot catch something the diff does not contain.

Probes, because a missing signal is not a signal. Production is probed on a schedule, including a forged-signature request against the payment webhook to confirm it still rejects one. Signature verification is exactly the sort of code an agent will tidy while doing something else, and neither review nor CI reports a check that quietly stopped happening. Something has to keep asking.

A release gate, not a release script. The expensive suites run at promote time, against the exact candidate, rather than on every commit. The agent-specific part is what “the candidate” is: not a change I watched being written, but the accumulated output of several sessions, no one of which is trusted on its own merits however green it went in.

Where the agent is kept out

Not every part of the codebase gets the same treatment. Money arithmetic, authentication boundaries and database migrations are written and reviewed the slow way — not because an agent cannot produce them, but because reviewing something I did not write is not the same as knowing it, and those three are where I need the whole failure surface in my head rather than on a screen.

There is one hard block, too, and it is enforced by tooling rather than by instruction: agents in the main product repository cannot run the test suite locally. It exhausted the development machine’s memory often enough that asking politely stopped counting as a control. Every check runs server-side instead — the same principle as everything above, applied to the machine rather than to the reviewer.

Decision records as the context layer

Gates catch what is wrong. They do nothing about work that is correct but off-policy — the agent that helpfully reintroduces the ORM that was deliberately removed, or resurrects a rejected architecture because it looks like an improvement.

For that, both systems carry numbered decision records, counted in the header. They are not history; they are current policy. Reversing one requires reading it first, and an agent handed the relevant record argues from the same constraints I would. Most bad agent output traces to missing context rather than missing capability, and a decision log is context in the one form that survives being handed to something that was not there at the time.

This is the single highest-leverage practice in this whole write-up, and it is also where the story turns.

Where it rotted anyway

The context layer — the instruction files, rules and memory that agents read before touching the code — is prose. Nothing compiles it. Nothing tests it. And unlike a stale comment, which a human reads sceptically, an agent executes it.

Some documentation is already guarded against exactly this: a generated document describing what the test suite covers fails CI when it stops matching the suite, because documentation that cannot silently diverge is documentation still worth reading in six months. But a drift guard only reaches documents something can generate, and the instruction files are not among them.

An audit of the AgendaProfe repository — the most disciplined one I operate, the one everything above is true of — found four live instances of context drift in its instruction layer:

• Environment variables for two services that had been decommissioned months earlier, still declared as current in the build tooling.
• A leftover server configuration pointing at a database that no longer existed.
• A memory file directly contradicting the project’s own stated pull-request policy.

Every one of those statements was true when it was written. That is what makes it drift rather than carelessness — and drift is a property of time, not of discipline. The uncomfortable conclusion: the context layer was the single most load-bearing artefact in the practice, and the only one nothing verified. Code has a compiler, a test suite and a release gate. The instructions agents obey literally, at scale, every session, had nothing.

The tool that fell out of it

So the fix was to treat context as code: groundtruth parses an agent-context file, turns the claims it can verify into executable assertions — this path exists, this script is defined, this environment variable is genuinely gone — and fails CI when one goes false. Claims it cannot check are reported as unverifiable rather than silently skipped, because a green check that quietly ignores half the file is worse than no check at all. The design is written up separately, and the tool is open source.

It now runs as a gate on this site’s own repository: the pull request behind the page you are reading was checked against the agent-context file that directed it.

What it costs, honestly

Verification is most of the work. The commit share in the header overstates how automatic this is. Direction, review, and the design of the gates are the job now, and they are slower to build than the features they protect. The gates above were each built in response to something — a class of bug, a near miss — not installed from a checklist on day one.

The human is the bottleneck, permanently. Parallel sessions collapse build time, but decisions, product judgment and anything requiring a conversation stay serial. The practice makes one person dramatically wider, not redundant — and it makes the quality of that person’s written context the limiting factor on everything downstream.

Stale context is worse than absent context. The audit finding is general. A team adopting agents accumulates standing instructions that nothing on earth verifies, and agents follow them with confidence, at scale, every session. My best-kept repository had four false claims in the layer agents trust most. I no longer believe any repository with real history has zero.

Where it stands

The practice runs everything I ship: the payments platform, the transit system from the other case study, the infrastructure, and this site — same loop, same gates, same context discipline, now with the context layer itself under verification.

It has also become transferable. Auditing an agent-context layer against its repository — every claim proven or failed, a gate left behind so the lies cannot return — is a repeatable piece of work, and one I now offer to teams as a fixed-price audit — for anyone whose agents have been reading instructions nobody has checked in months.