Governing AI-DLC, Part 3: How to Know Your AI-Assisted Delivery Is Actually Safe

Quick context. AI-DLC is a way of running delivery where AI agents do the building inside a defined process, with people at fixed decision points. This part is about the instruments: what do you actually measure, and what does it take to move the needle.

What story points were actually for

Worth being precise about the two instruments the Agile era gave us, because their purpose explains why they stopped working.

A story point was a unit of account for capacity. A burndown was a forecast of that capacity being used up against a promise. Both existed for one reason: capacity was the thing you ran out of.

Capacity is not the thing you run out of any more. You can add agents. So both instruments now measure something you have plenty of — which is exactly why teams report excellent numbers while feeling less in control. The instruments are not lying. They are carefully answering a question nobody needs answered.

Two things are genuinely scarce now: people’s attention, and justified confidence that the guardrails work. Anything worth measuring measures one of those.

The two that matter most

Everything else on that board is a diagnostic for these two.

Human minutes per intent replaces the story point. How many minutes of human judgement did one piece of work consume — answering questions, approving gates, actually reading things. It is the unit of account for the only resource still rationed. And it is the number that tells you whether the programme is working at all, because if it does not fall as the system matures, your automation is moving work around rather than removing it. In my experience that is the most common way one of these efforts quietly fails while every dashboard stays green.

Recall replaces the burndown as the chart on the wall. Of the problems that reached production, what share should have been caught by a check you already had?

Now check whether any of it is wired up

Look again at the status column, because it tells you something the numbers themselves do not.

The pattern that runs through every one of these platforms: they measure consumption exhaustively and effectiveness not at all. How long a step took, how many tokens, what it cost at today’s prices. Not one number about whether the guardrails are catching anything.

That is the same instinct that builds a finance dashboard before a security one. Tokens have a price. Confidence does not, so nobody built the meter.

But the more useful split is between the amber and the red. Time spent before approving is merely not wired up — it needs the interface to record how long somebody had the thing open, which is maybe a fortnight of work, and it would tell you straight away whether your approvals are real. Whereas reversal rate, recall and the time it takes to turn a miss into a check cannot be produced at all, and not because nobody got round to them.

Recall is a fraction and these platforms collect only the top of it. You know what the checks caught. You cannot know what got past them, because knowing that means still watching after the merge — and the lifecycle ends when the code is handed over.

So this is not a new problem. It is the missing operations phase wearing a different hat. Close that one loop and recall, learning time and reversal rate all arrive together. Which makes it the single highest-leverage fix in this whole series, and I did not expect to arrive there from a question about metrics.

So how do you actually move recall?

You add checks. Which sounds trivial and is not, because “write more checks” is a direction, not a craft. The craft has a name.

In the first post I argued for a role nobody staffs: the sensor author, the person whose job is turning a review somebody keeps doing by hand into an automatic check. Here is what that person actually writes.

An invariant, not a test

The unit of work is the invariant: something that must be true in every state the system can get into. Not something that was true once, on a Tuesday, for the input somebody happened to think of.

That distinction is the whole thing:

• This login rejects a bad password is a test. It samples. It tells you about the cases you wrote down.
• No part of this system accepts a password over an unencrypted connection is an invariant. It covers cases you never enumerated.

When forty agent runs touched your code last night, only the second kind of statement is worth anything.

The structure underneath, which earns its keep

This is well-worn ground in formal verification, and it transfers exactly. For a property to be an invariant, three things have to hold: it is true at the start; every step keeps it true; and being true actually means you are safe.

Now look at what AI-DLC is. A sequence of steps. Each step is a change of state.

A check at a step boundary is the “every step keeps it true” condition. That is not an analogy — it is the same obligation, done by a machine instead of a mathematician.

It also corrects something I was too quick about in the platform post. I criticised checks that only fire at step boundaries rather than continuously. Under this framing the boundary is exactly the right place, as long as the step is the unit of change. The criticism only bites when an agent can break the property in the middle of a step — and then the boundary is too coarse and the check belongs closer to the work. Both are true, and I would rather say so than keep the tidier version.

Where invariants come from: your policy

Here is the actual work, and it is more interesting than it sounds. Take your security policy and try to restate each line as an invariant. Exactly three things happen.

It converts cleanly, and you write the check. Or it converts only once you invent a definition you did not previously have — and that definition is the valuable output, because “sensitive data” now means a list rather than a shared feeling. Or it does not convert at all, because nothing could ever prove it false, in which case you have learned that a line you have been auditing against for years was a sentiment with a policy number on it.

What makes a good one, borrowing from the people who do this most seriously — the smart-contract world, where getting it wrong costs money the same afternoon:

• Write the plain sentence first. If you cannot say it in one sentence, you do not have an invariant yet, and writing code will not give you one.
• Say it as a “never”. “No”, “never”, “always”. Positive requirements pile up; negative ones combine cleanly.
• One counterexample should kill it, and checking it should be cheap enough that you never think about the cost.
• No judgement required. If deciding needs an opinion, it is a review, not an invariant. That is fine — just do not put it in a gate.
• Start with a handful. The mature version of this is a few load-bearing rules, not a wall of them.

This is not a fringe technique. AWS’s own automated reasoning work does exactly this at scale: Zelkova turns access policies into logic and checks them with a solver, and the same family of tools is used internally to verify that services hold to their security invariants. The idea that a security property should be a machine-checkable claim rather than a paragraph is, by now, the mainstream position wherever the stakes are high.

Why this matters more with agents than without

Two reasons, and the second surprised me.

The first is volume. An agent producing forty pieces of work overnight is forty chances to break a property nobody wrote down. Human-paced development left you time to notice. This does not.

The second is that the model cannot hold the rule for you. Veracode’s 2026 analysis of AI-generated code lands exactly here: tracking where data came from, what changed it, and whether each step preserved a security property is the kind of sustained reasoning current models are not built for. So the rule cannot live in the prompt. It has to sit outside, be checked by a machine, and be rechecked at every step.

And this is what closes the loop

Here is the part that made the whole thing click for me.

If your checks are written as invariants, then recall measures how complete your set of invariants is. And the question you ask after an incident sharpens from the vague “should a check have caught this?” to the answerable “which invariant was missing, or too weak, or checked in the wrong place?” That question has an artefact as its answer.

So it closes in one vocabulary: policy becomes invariants, invariants become checks, checks produce recall, and every gap in recall names the next invariant. The sensor author is the person who runs that loop, and “raising the bar” stops being a motivational phrase and becomes a diff.

Where this costs you

Recall will look appalling the first time you compute it. It is also a lagging indicator, so it will keep looking appalling for a quarter after you fix the cause. Introduce it somewhere that punishes bad numbers and it will be quietly redefined until it flatters. Fund the review hour and leave the first six months untargeted.

Somebody has to do that review hour. For every problem found in production, a person decides whether an existing check should have caught it. Roughly an hour each, the most valuable hour in the system, and invariably the first thing cut when the quarter gets busy.

Do not aim for perfect recall. That is how you end up with a blocking check for every mistake anyone has ever made, a process nobody can get through, and engineers routing around you inside a month.

And do not make the number of invariants a target. It will be gamed within a quarter and you will get fifty rules nobody could possibly break. Coverage of your policy and recall are the numbers; a count is vanity.

One more, and it is a real cost: adding an invariant retroactively makes everything approved without it unverified. Your set of rules is versioned, and merged work is only as checked as the version it was checked against. That is a re-verification problem you now own.

What I would do on Monday

1. Work out one recall number by hand. Take the last ten problems found in production and ask, for each: should an existing check have caught this? The fraction is your recall. It will be worse than you expect and more useful than any dashboard.
2. Try to convert five lines of your security policy into “never” sentences. Notice which ones will not convert. That list is more interesting than the four that did.
3. Measure how long people spend on an approval before clicking. If your tooling cannot tell you, that is itself the finding.

The short version

Agile’s instruments answered can we build enough? They were good answers, and machines have settled the question.

The question underneath — can we be confident enough, cheaply enough in human attention, about things we cannot take back? — has almost no instruments at all. Building them is the interesting work now, and it starts with the two numbers in this post and one honest hour a week.

Sources, and what each is good for

• How AWS uses automated reasoning to achieve security at scale — the clearest public description of treating a security property as a machine-checked claim rather than a paragraph.
• Security Invariants or GTFO — Alex Smolen. The argument for expressing a security programme as invariants whose only acceptable count is zero. Short, and it stuck with me.
• Spring 2026 GenAI Code Security Update — Veracode. Why the model cannot hold the rule for you, and therefore why it has to be enforced outside the model.

I write about this work as it develops. If you are trying to measure one of these programmes, I am reachable on LinkedIn, and the tooling I maintain lives under shenril and BlueSquadron.