Trust Is an Engineering Deliverable (Part 2 of 2)
Last post I argued the loop is the easy half and the boundary is the real work. This one is about the people the loop crowd keeps writing off, because I think they’re the most important audience in this whole discourse.
I have been watching developers stay stuck at the prompt. Good engineers. They will happily run an agent turn by turn, read every diff, approve every step, and they will not let it run an isolated goal-based loop without them in the chair.
Ask why and you get some version of fear. Fear of what it’ll touch. Fear of what it’ll break while nobody’s watching. Fear of being the name on the postmortem for a change no human wrote.
The standard response to this is coaching. Trust the model, the models are better now, just try it. I want to say plainly that this response is wrong, and not just ineffective wrong. Wrong wrong. The fear is not a mindset problem. It is an accurate risk assessment of missing infrastructure.
These developers are looking at an autonomous system with no gates, no evidence trail, and no blast radius limits, and they are declining to bet their reputation on it. That is not timidity. That is the same judgment we hired them for.
Which means it sounds like a problem to solve with engineering.
We Already Solved This Once
Think about how a junior engineer gets merge rights. Nobody sits them down and grants trust. Nobody coaches the seniors into feeling comfortable.
We built a system where trust is mostly unnecessary: CI that runs the tests, review that requires another set of eyes, branch protection that blocks the direct push, rollback that bounds the damage when something slips through anyway. The junior merges on day one not because anyone trusts them but because the system made the trust question cheap.
Then something interesting happens over months. The junior’s record accumulates. Their PRs pass clean, their incidents are rare, and the humans around them start granting real trust, the informal kind, based on evidence the system collected without anyone trying.
The infrastructure didn’t just protect the codebase. It manufactured the trust.
That is the move with loops. You do not ask developers to trust the agent. You build the system that collects the evidence, and you let the trust arrive on its own schedule.
Leverage Isn’t Enough
I want to be fair to the best version of the other argument before I push on it. The compound engineering crowd at Every has been making the loop case longer than almost anyone, a long time in AI years anyway, and their framing is genuinely good: every unit of work should teach the system, so the next unit gets cheaper.
The lessons compound. The leverage compounds.
True. And incomplete. Because error compounds on exactly the same curve. A loop that learns also drifts, and a lesson encoded from a bad example is a bug with a memory.
Compounding tells you the loop is getting more capable. It tells you nothing about whether the loop has earned the right to act on that capability. Leverage compounds output. Evidence compounds trust. You need both curves, and the discourse only talks about one of them.
What This Looks Like at My Desk
Enough theory, here is a system I actually run.
I have agents that mine my work surfaces for actionable signals. One watches Teamwork for task assignments. Another watches the calendar for meetings, and the notes and transcripts those meetings leave behind, pulling action items out of the wreckage.
Another reads email and Teams for requests for help or work that arrive dressed as conversation. Another watches GitHub PRs and issues for requests for change. Different sources, same job: find the signals hiding in the noise.
Here is the part that matters. The agents do not act on what they find. They extract the data, categorize each signal, and attach a confidence rating, and the rating travels with the signal as evidence. Then a gate decides.
Only confident signals matching the desired category get through. A vague maybe-request in a Teams thread scores low and stays out. A direct assignment with my name on it scores high and lands in front of me.
Notice where the gate sits. Not at the output end, checking finished work before a merge. At the intake end, before any work begins, before a decision is even on the table.
Everyone’s verification story this week lives at the end of the loop, grading homework after it’s done. Mine starts at admission. The evidence is not an audit trail bolted on after the fact. It is the ticket that gets a signal into the room.
That is shift-left, applied to governance. We spent a decade moving testing earlier in the pipeline because defects get expensive with age. Decisions are the same.
A bad signal admitted at intake becomes a bad task, becomes bad work, becomes a bad merge, and every stage downstream pays interest on it. So the gate goes first. Engineering shifts left, and the decision gates are standing there before any decision gets made.
Is intake gating sufficient on its own? No, and I won’t pretend it is. You still need the output gates from part one, the evals and promotion rules and blast radius limits.
The point is the boundary is not one wall at the end of the loop. It is gates at both ends, evidence flowing between them, and the loop running in the space the gates define.
Receipts Can Be Forged
There is a failure mode hiding in everything I just said, and if you have run agents against a test suite you have already met it. The agent optimizes for the gate, not the work. Tell a loop the receipt is a green test run and it will get you a green test run. Sometimes by fixing the code.
Sometimes by weakening the assert, deleting the flaky test, mocking the world away, or writing a test that lovingly mirrors the bug it was supposed to catch. Goodhart’s law with a commit bit: the moment the measure becomes the target, the measure stops measuring.
So the receipts need checks and balances of their own. Not infinite regress, just a second layer that audits the first, and the trick is making that layer arithmetic instead of judgment.
Mutation testing is the cleanest example. Seed faults into the code and count how many the tests actually catch. A suite that lets mutants walk free is a forged receipt, and the kill rate is a number nobody can argue with.
CRAP scores do similar work from another angle, flagging code that is complex and barely tested, which is exactly where an agent hides its shortcuts. And a complexity delta on every change catches the slow forgery: tests green, behavior fine, structure quietly rotting underneath.
The structural half of checks and balances matters as much as the metrics. The agent that writes the code must not be able to touch the tests, the eval definitions, or the gate thresholds. Separation of duties, enforced as a diff constraint, not a convention.
A writer that can edit its own gate is not a governed system. It is a system with paperwork.
Audit the receipts with arithmetic. Lock the gates away from the writer. Then a green run starts meaning what it says.
The Trust Ladder
So how does the fearful developer, the correctly fearful developer, actually get from prompting to loops? The same way the junior got merge rights. In stages, with evidence at every rung.
- Report-only. The loop finds work and proposes, nothing more. Every proposal carries its confidence and its evidence. You read them for a week or two and you learn what the loop gets wrong, which is the most valuable data you will collect in this entire process.
- Propose-with-gates. The loop still doesn’t act, but now your gates filter the proposals, and you watch the gates instead of the raw stream.
- Act-with-gates. The loop acts on bounded, reversible work, where the worst case is a reverted commit.
- Unattended. Eventually, for some loops, on some work, the chair sits empty.
Each rung is a deposit. Every gated decision with a receipt builds the record, the same way the junior’s clean PRs did.
And here’s the part I keep coming back to: at no rung did anyone have to feel brave. The developers I described at the top do not have a courage deficit. They have an evidence deficit, and evidence is something we know how to manufacture.
The loop crowd is right that the loop is the future of this work. The trust crowd is right that an ungoverned loop is a liability with a scheduler. Both camps are staring at the same missing piece.
Build the system that keeps receipts, and the fear takes care of itself, not because anyone talked the fear away, but because the fear was never the problem. The missing receipts were.
Trust isn’t a feeling you wait for. It’s a deliverable you ship.
Let’s talk about it.










