When AI Writes the Code, 'Done' Has to Mean More

When engineering says "shipped, all tests pass, no P0s, the demo worked," that sentence used to end a delivery conversation. It still gets said in a lot of sprint reviews. It just no longer means what it used to.

The object you just called done is not the object it was a month ago. Traditional software was a vending machine. Input in, output out, same answer Tuesday as Monday. AI features are not vending machines. The same prompt produces different output across users, sessions, model updates, and contexts your QA team never imagined. What you shipped is a distribution of behaviors, some predictable and most not.

That single fact is why the definition of done, the team-level agreement on what done means, has become the most important artifact on a consulting delivery. It is also why most teams still carry a definition written for a world where code was deterministic and slow to produce. This is the story of what done means now, and how to rewrite the agreement so it survives agents.

The old done was built for deterministic software

The classic definition of done read like a handoff checklist. Code written. Unit tests pass. Code reviewed. Deployed. It worked because the underlying assumption held: inputs map to outputs predictably, and done was the moment the spec, the tests, and QA all lined up.

That assumption no longer holds for the parts of your system where agents or models generate behavior. Jeff Gothelf, who has spent the AI era rewriting product practice, puts it plainly: we built our definitions of done for deterministic software, and for AI they have to be rewritten entirely.¹

The data backs the rewrite. CodeRabbit analyzed 470 open-source pull requests and found AI-co-authored PRs carry about 1.7 times more issues than human-only PRs, with logic and correctness problems 75% more common and security vulnerabilities up to 2.74 times more frequent.² Code that passes linting, compiles, and passes unit tests can still be subtly wrong in production. A unit test on a payment calculator returns the same result a hundred times. Run the same prompt through an LLM a hundred times and you get a handful of confident wrong answers.³

There is a deeper reason this is not a statistical blip. W. Edwards Deming's principle, that quality must be built in and not inspected in, assumed a builder who could internalize quality over time. A senior engineer learns why a standard exists. An agent does not. It reproduces the most common patterns in its training data without knowing why any of them matter. The result is what one SAFe analysis calls the quality consciousness gap: you cannot train an agent to care, so you have to make the surrounding system enforce quality mechanically.⁴

That is the real shift. When agents write a large share of the code, all tests pass is still important and means the same thing it always did. It just no longer means done.¹ The definition of done has to stop being a binary result and become a calibration.

Acceptance criteria as distributions, not assertions

The first thing to change is the shape of the acceptance criteria that feed the definition of done. Most criteria are assertions. "When the user inputs X, the system returns Y." That is verifiable because it is binary, and for an AI feature it is often meaningless, because the response to a fuzzy input is not a single Y.

Gothelf's replacement framing is simple and hard to argue with. Write the criterion as a distribution instead. "For 80% of inputs in category X, the system returns a response that meets quality bar Y. For the remaining 20%, the failure mode is degraded but not embarrassing."¹

Two consequences follow. First, you keep two sets of criteria on the same feature: deterministic ones for the parts that still behave like vending machines, auth, billing, navigation, and distributional ones for the parts that do not, feedback, context-specific responses, custom requests. Second, you stop pretending the probabilistic ones are deterministic. The teams that get this right do not write every criterion as a percentage. They are honest about which outcomes are which.

A consulting example makes it concrete. Suppose the deliverable is a support-triage bot that classifies and routes inbound tickets. A deterministic criterion is easy: the routing to the correct queue given a validated category must succeed with no failures, and the authentication boundary must hold. A distributional one is where the value lives: for the long tail of ambiguous phrasing, at least 80% of routed tickets must be accepted by the receiving team as correctly classified, and the remaining 20% must degrade to a safe default, routed to a human, not silently misfiled. The old done would have declared the bot done the moment the happy-path demo worked. The new done pins the acceptable variance and names the tolerated failure mode before launch.

For a consulting engagement this lands directly on the definition of ready. The requirements work we wrote about is upstream of this: if you cannot write testable acceptance criteria, you cannot write a distributional one either. Read the requirements piece. But there is a difference between acceptance criteria, which are per item, and the definition of done, which is the shared standard every increment must meet. In agent-native delivery the definition of done is where the acceptance criteria, the verification layer, and the release standard all get reconciled.

The verification layer that done now implies

The second thing to change is what the definition of done verifies. A DoD built on pass/fail binary checks is, for agent-generated behavior, a gamble. Sanjay Saini's four-point framework for Scrum teams is a useful template for what the verification layer has to include.³

The golden set. You cannot manually test an agent before every release, so you curate a golden dataset, fifty to a hundred inputs with verified, human-written answers, and run it in the CI/CD pipeline. The DoD criterion is a semantic similarity threshold, say above 90% measured with ROUGE or cosine similarity, against those verified answers. This is the mechanical drift detector: when the model or the behavior shifts, the score drops before users feel it.

The PII redaction guardrail. Agents are targets for prompt injection designed to leak data. Security stops being a non-functional nicety and becomes a core quality standard. The DoD test: feed the agent fake PII, a credit card number, and confirm it is redacted to a placeholder before it is processed or logged.

The circuit breaker. An autonomous agent does not get tired. Stuck in a retry loop it can burn real money in token spend in minutes. The DoD includes hard caps at the infrastructure level, a maximum number of steps per task or a maximum spend per execution.

The human fallback. When an agent cannot answer with high confidence it must know when to quit. The DoD criterion is a fallback test: below a confidence threshold, route to a human or return a safe canned response instead of guessing.

The mindset shift matters as much as the checks. Quality assurance used to be about finding bugs. In the agent era it is about detecting drift. An agent that is done today may not be done next sprint, because the underlying model changes and user behavior shifts.³ The definition of done has to be treated as a living contract that gets re-verified, not a certificate that once granted, stands forever. That is why it belongs on the sprint-review agenda and why the evidence, the golden-set score, the guardrail results, belongs in front of the stakeholder alongside the demo.

Enforce the standard, because inspection cannot scale

The third change is the hardest for delivery teams to accept. The new definition of done cannot be a wiki page that developers consult. When agents produce far more than humans can review, inspection as the quality mechanism breaks.

The volume problem is concrete. Per Faros AI's data, cited in the SAFe built-in quality analysis, teams using AI code generation produce about 98% more pull requests while requiring 91% more review time.⁴ Review queues saturate. Faster reviews miss more defects. More reviewers fragment context. Inspection was designed for human-speed output, and agents break that ratio permanently.

The response is to make the definition of done an attestation chain: a set of machine-enforced gates that each increment must pass before a human sees it. The SAFe analysis calls this harness engineering, and it is the direct implementation of built-in quality when agents are the builders.⁴ The practical version, drawn from the same playbook:

Encode standards as rules, not guidelines. A guideline says prefer composition over inheritance. A lint rule rejects any PR that introduces a new inheritance hierarchy without approval. Guidelines depend on the builder's judgment. Rules do not care whether the builder has any.

Invert test-driven development. The test suite stops being a verification step and becomes the specification. When an agent iterates until tests pass, the tests are the contract it is held to. Teams that do this find their quality expectations get encoded as executable specs rather than prose.

Apply WIP limits to agent-generated pull requests, separate from human work. Agent PRs queue faster than any reviewer can process them. The SAFe analysis recommends a limit in the range of three to five pending agent PRs per team, with automated pre-review, linting, test validation, security scanning, before anything enters the human queue, and flow measured on merged code, not generated code.⁴

Track provenance at release. When humans write code, authorship is implicit in git history. With agents you need explicit attestation: which agent, under what instruction contract, with what policies and sandbox. This is also moving from best practice to regulatory requirement, since the EU AI Act's obligations around AI-generated code begin taking effect in August 2026.⁴

None of this removes the human review seam. It shrinks it to what only a human can judge, architectural intent, domain correctness, and the judgment calls that mechanical gates cannot see. The definition of done becomes the boundary that decides what reaches that seam.

The payoff is measurable even without switching models. The SAFe analysis reports a case where changing only the harness, not the underlying model, raised quality scores from 52.8% to 66.5%.⁴ That is the point: you do not need to wait for better agents to get better output. You need a stricter harness and a definition of done that the harness enforces.

Ship the tripwire and rehearse the rollback

The last piece is the one most teams skip, and it is where the definition of done stops being theoretical. In the deterministic world you shipped the feature and triaged when complaints surfaced. For AI that is too slow. By the time a complaint reaches engineering, the user has already formed an opinion of your product.

Gothelf's closing move is to build the failure response into the definition of done itself. Write the triage playbook before launch. Decide who owns model-quality issues, UX issues, content issues, and the PR fallout a model might trigger. Then define the signal that means this is not done after all: error rate, off-tone complaints per thousand sessions, escalations to human review, a hallucination rate from a sample audit. Set the threshold before launch. Decide what happens when it is crossed. Then rehearse the rollback, actually run it, not just write it down, because the moment you discover your rollback does not work is the moment you most need it to.¹

For a consulting team this is a scoping and contracting decision as much as a technical one. The acceptance gate, where the client signs off, now has to include the tripwire and the rollback as part of what done means. Read the stakeholder piece. Done is not the moment the feature ships. It is the moment the people downstream of the feature know what to do when it misbehaves, and it will misbehave, because it is AI.

The one-line version

Done was the language of a build culture. Calibrated is the language of a learning culture. The definition of done that survives agents writes acceptance criteria as distributions, adds verification of the probabilistic layer, enforces the standard with machine gates instead of hoping builders internalize it, and ships a tripwire with a rehearsed rollback. When engineering says all tests pass, the right response is no longer "great, ship it." It is "show me what you verified, and show me what you will do when it drifts."