When an AI agent says “done”, what should we check?

The agent says the fix is done. You read the explanation, open the application and press the button that started all this. Nothing happens. Now there are two jobs in front of you: the original repair and working out why the agent believed it had finished. That extra job is a useful place to begin talking about harness engineering.

Harness engineering begins in that gap. The term describes the software built around a model so it can use tools, keep track of work and check what happened. A better prompt helps, but the application still needs to open and the button still needs to work.

I find it easier to understand through an ordinary question: what would a colleague need to finish this job reliably? Relevant files, suitable tools, clear permissions and a way to test the outcome are a useful place to start.

Give the model a working environment

A model can reason about a task and propose the next action. The harness determines what it can observe, which tools it can call, what those calls are allowed to change and how their results become the next step’s context. The environment is where those actions meet reality: a repository, a database, a browser, a service or a person.

This is the framing in Hailin Zhong and Shengxin Zhu’s May 2026 preprint, AI Harness Engineering. Their proposed H0–H3 ladder adds tools, persistent context and verification in stages. The useful distinction is the evidence each stage leaves behind.

The paper includes a controlled validation task; it does not establish a general performance law. I read it as a useful architectural proposal. It makes the unit of analysis the complete system, and makes the evidence of a run as important as its final patch.

Make the responsibilities understandable

In practice, I find seven pieces worth naming. The names are less important than knowing where each job happens. The model should not have to reinvent the rules every time it answers.

Contract

Define the task, the evidence that counts as done and the conditions that end a run.

Context

Select the relevant sources, instructions and prior decisions for this step.

State and memory

Distinguish what was proposed, what was attempted and what actually completed.

Tools

Expose explicit operations with validated inputs and useful, bounded results.

Authority

Enforce access rules and require approval before an action crosses its permitted scope.

Verification

Check claims against tests, sources, transaction records or the running product.

Recovery

Record failures, bound retries and provide a route to repair or human handover.

Much of this is familiar software work: check inputs, remember state and record changes. The extra difficulty is that the model may choose an unexpected next step. The surrounding system needs a useful response when it does.

Check approval before the money moves

Consider a refund that requires approval. Writing “ask a person first” in the prompt expresses the rule. Checking an approval record at the execution boundary enforces it.

The distinction is concrete. The tool should validate that approval belongs to this action, this amount and this recipient before it sends anything to the payment service. The model can propose a refund; it cannot manufacture the authority to issue one.

Joongho Ahn and Moonsoo Kim examine this move in their July 2026 preprint, From Prompts to Contracts. They place source boundaries, routing rules, trace requirements and output checks in code around a replaceable model. Their code-owned checks held across 270 composition-boundary runs with three hosted models; deliberate faults activated the validators.

The scope matters: fixed scenarios over public data from five Korean corporate groups. Those results support the pattern within that evaluation. They do not mean all agent behaviour can be made correct by adding validators. A check can enforce only the property it was designed to inspect.

A useful loop knows when to stop

Prompts still matter. Clear tool descriptions reduce ambiguity; selected context gives the model a better basis for its next decision. Anthropic’s Writing effective tools for agents makes this practical through tool naming, descriptions and response design.

But useful instructions sit inside a larger loop. After the model proposes an action, something must establish whether it may run and whether it worked.

The check should look at the thing that changed. For code, try the feature. For a release, visit the running service. For a database action, inspect the transaction result. Asking the model to read its answer again can help with wording, but it cannot stand in for these observations.

Anthropic’s Building effective agents describes this as obtaining ground truth from the environment. Its later work on long-running agents makes the approach tangible: persistent progress records, a feature list and browser tests that exercise the actual application. These are ways to make the next step depend on observable state.

Read what happened, then inspect the result

A successful final answer can conceal a poor execution path: unnecessary retries, a wrong tool call, an approval requested too late. A trace makes that path inspectable. Outcome checks establish what the path actually achieved.

Anthropic’s Demystifying evals for AI agents treats evaluation as a test of the model, harness, tools and environment together. One practical lesson is to grade the state left behind: whether the file changed, the tests passed or the expected record exists. The transcript helps explain how the system got there.

That makes traces part of the product’s operating record. They should capture tool calls, policy decisions, approvals, results and enough context to investigate a failure, with appropriate handling of sensitive data. A polished completion message is useful to the reader; it is not a substitute for that record.

Where I meet this in my own work

I came to this problem through autonomy and the handover to a person. A machine recommends an action, but the evidence behind it is often lost before the person has to decide.

NODERIQ is concerned with carrying that evidence to the decision. NowFlow makes approval an explicit routing step. QFlow Studio keeps the brief, generated source, provider route and run evidence together so an experiment can be reconstructed. In the agentic software-testing work I lead, evaluation and oversight belong before the release decision.

The projects differ, but I keep meeting the same practical question: when did we decide an action was allowed, and can the next person see what happened afterwards?

Start the review with a failed attempt

Try opening a failed run with someone who was not there. Can you show what the agent saw and what it tried? Can you tell whether anything changed before it stopped? That conversation usually reveals what the system still needs to record or explain.

From there, the design work becomes concrete. Agree what a finished task looks like, put checks in the tools and keep enough information to resume after a problem. Then look at the actual result together.

The aim is an agent whose work a colleague can follow and rely on. The model brings ability; the tools and checks help that ability turn into something that actually works.