Harness engineering, or the layer around the model
Longer prompts and stronger models did not stop agents skipping steps, misusing tools and declaring work done. The fix is the layer around the model.
For the first two years of generative AI, doing better meant writing a better prompt. Longer instructions, a bigger context window, a stronger model whenever one shipped. Prompts that began as a paragraph grew to a few hundred lines, and for a chat interface that was often enough.
Then the systems started acting rather than answering, and a different problem appeared. An agent forgets a decision it made twenty steps earlier. It picks the wrong tool, or the right tool with the wrong arguments. It retries the same failing call until the budget runs out. It reports a task complete that it never finished, produces a figure it never checked, or runs something irreversible without anyone having agreed to it.
The reflex is to change the model or rewrite the prompt. Sometimes that helps. But a model that is intelligent is not the same thing as a system that is reliable, and most of what makes an agent reliable is not inside the model at all. It is in what has been built around it.
The model is not the system
A preprint from May this year puts this plainly. In AI Harness Engineering: A Runtime Substrate for Foundation-Model Software Agents, Hailin Zhong and Shengxin Zhu argue that the capability of a software-engineering agent is a property of the model, the harness and the environment together, not of the model on its own. The harness is the runtime layer between the other two. It decides how the agent observes a project, how it acts on it, what feedback comes back, and how completion is established.
The paper names eleven responsibilities for that layer: task specification, context selection, tool access, project memory, task state, observability, failure attribution, verification, permissions, entropy auditing and intervention recording. It arranges them on a four-level ladder, H0 to H3. At H0 the agent has a task description and the repository files, and an episode leaves behind a patch. H1 adds a tool registry, a test-command registry and a protocol for using them. H2 adds project memory, task state and a rule for selecting context. H3 adds deterministic checks, bug reproduction, failure attribution and a verification protocol, and an episode leaves behind reproduction logs, attribution logs and a report that maps each requirement to its evidence.
It is a framework paper rather than an empirical one, and it is a preprint, so it should be read as a well-argued proposal rather than a settled result. The proposal is the right shape. The model reasons and proposes. The harness decides what the model is shown, which tools exist, what state the system is in, which actions need permission, how success is checked and what happens on failure. That is an architectural move, from prompt-centred systems to execution-centred ones.
What the layer has to own
Across the systems I have worked on, and across what the vendors have published, the same responsibilities keep appearing. I group them into seven.
- Contract. What the task is, what counts as done, where the agent must stop, which lines it may not cross.
- Context. What the model needs to see right now, and how sources, rules and earlier decisions are selected for it.
- State and memory. Which stage the work is in, which decisions were taken, which actions were actually carried out rather than merely proposed.
- Tools. Which API, database, browser, terminal or enterprise service is available, and under which conditions.
- Permissions and policy. What the agent may read, what it may change, and which operations need a person.
- Evidence and verification. Whether there is a test, a source, a log or a record that shows what the agent claims is true.
- Recovery and observability. When something fails, whether the cause is known, and whether the right response is a retry, a different route, or a handover.
The point that matters is that none of this has to be left to probabilistic reasoning. Some of these are rules and should be code. Some are shapes and should be schemas. Some are decisions and belong in a policy engine. Some are claims and should be tested. And some are actions that should not happen until a person says so.
An instruction is not a guarantee
You can write “ask the user before making a payment” into a system prompt. That is an instruction, and a good model will usually follow it. If instead the payment tool cannot be called without an approval token, the model cannot make the payment without approval, whatever it decides. Those are two different guarantees, and only one of them survives a bad day.
A second preprint, From Prompts to Contracts by Joongho Ahn and Moonsoo Kim from July, examines exactly this move. They take an exploratory enterprise agent and rebuild it so that the deterministic behaviour lives in code, manifests, schemas and validators around a replaceable model. Five things become contracts rather than prompt expectations: source grounding, entity routing, trace completeness, output hygiene and the language used for recommendations. Across three hosted models the code-owned checks held on all 270 runs at the composition boundary, and a fault-injection control confirmed that the validators fire when a contract is deliberately broken. Remove the code-owned gate and the guarantee disappears.
The evaluation is narrow: fixed scenarios over public data on five Korean corporate groups, one domain, one case study. The authors say so themselves. What it demonstrates is not a universal number but an architectural fact. A control that lives in code stays put when the model is swapped, and a control that lives in a prompt has to be re-verified every time.
So the question a serious system asks is shifting. Less “can the model do this?”, which is increasingly yes, and more “how does the system know the model did it correctly?”.
The prompt does not go away
None of this retires prompt engineering. It demotes the prompt from being the system to being a component of one, and components still have to be good.
Tool descriptions matter. Anthropic’s note on writing tools for agents, from September 2025, reports that “even small refinements to tool descriptions can yield dramatic improvements”, and spends most of its length on naming, namespacing, what a tool returns and how many tokens it costs to return it. System instructions matter. Context selection matters. The instructions that let an agent plan matter.
What changes is the shape of the whole. Prompt, model, answer was a straight line. The systems that work now are a loop.
- Goal
- Context
- Agent
- Tool
- Environment
- Evidence
- Verification
- Feedback
- Next action
Andrew Ng’s reflection pattern, the first of the four agentic design patterns he described in March 2024, is the simplest version of this: produce an output, have a test or a second pass check it, and feed the result into the next iteration.
A loop needs ground truth
A loop is not a guarantee either. An agent grading its own output does not become reliable by doing it three times. A badly designed loop repeats the same mistake at three times the cost.
What makes the loop useful is something outside the model that can say no, and it is different for every kind of agent.
- Coding agent
- A test suite that runs against the change, not the agent's summary of it.
- Research agent
- A source that can be opened and read at the place it is cited.
- Database action
- The transaction result, committed or rolled back.
- Deployment agent
- A health check on the running service after the change.
- Payment or refund
- The record in the ledger and the confirmation that came back.
Anthropic’s Building effective agents, from December 2024, describes the need for agents “to gain ‘ground truth’ from the environment at each step”, and to be able to pause for human feedback at checkpoints. Their harness for long-running agents, from November 2025, is the same idea made concrete: a feature list the agent cannot mark done without a passing check, a progress file it must read before starting, and browser tests that exercise the product rather than the agent’s account of it.
OpenAI’s practical guide to building agents, from 2025, names two triggers for handing control to a person: an agent that exceeds a retry or failure threshold, and any action that is “sensitive, irreversible, or has high stakes”, with cancelling orders, authorising large refunds and making payments as the examples. Their current API documentation implements it as an interruption: “the model can still decide that an action is needed, but the run pauses until you approve or reject it”. That is the payment rule again, moved out of the prompt and into the runtime.
Traces are part of the product
Agents are not deterministic software. Two runs from the same starting state can take different paths, so looking only at the final answer says very little about whether the system is safe to run again. Which tool did it choose. Which sources did it use. How many times did it retry. Which policy fired. When it made a wrong call, did anything catch it.
Anthropic’s Demystifying evals for AI agents, from January 2026, treats an agent eval as a systems test. The thing under test is the model plus harness, tools, memory and environment, and the unit of record is the trace, “the complete record of a trial, including outputs, tool calls, reasoning, intermediate results”. Their strongest practical advice is to grade the state the agent left behind rather than the story it tells about it: the tests pass, the file changed, the refund record exists. Human judgement stays as the reference for calibrating any model-based grader.
Once that is the practice, the trace stops being a debugging artefact and becomes part of what you ship. A system that cannot show its trace cannot be evaluated, cannot be audited, and cannot be trusted in any setting where those words carry weight.
Where this meets my own work
I did not arrive at this from the agent literature. I arrived at it from autonomy, where the failure I keep describing is the handover: a machine produces a recommendation, a person has to decide whether to act on it, and the evidence for the recommendation was flattened somewhere upstream. NODERIQ exists to carry that evidence to the point of decision. NowFlow treats an approval gate as a first-class routing step rather than a line in a prompt, and records where an agent may act, where it must stop and ask, and what it actually did. QFlow Studio keeps the brief, the circuit, the generated source, the provider route and the run evidence in one record, because nobody reconstructs an experiment from memory. And in the agentic software-testing work I lead, the evaluation, oversight and reliability controls are defined before a release decision rather than after it.
Each of those is a harness decision, and none of them lives in a prompt.
The question that changed
The strongest models are available to anyone who can pay for them, and the gap between vendors narrows a little every quarter. Saying which model you use is no longer a technical differentiator. What differs is the system around it: what the model sees, what it can touch, how state is kept, how actions are verified, how failures are caught, where people intervene, and how much of all that can be inspected afterwards.
That is why harness engineering is starting to look like the next engineering discipline of agentic AI rather than a passing term. The model supplies intelligence. Architecture, state, policy, tools, evidence, evaluation and recovery are what turn it into a product that can be relied on.
The question used to be how to write the best prompt. The better question is how to turn a capable model into a system that works correctly, shows that it did, and stops when it should.
Sources
- AI Harness Engineering: A Runtime Substrate for Foundation-Model Software Agents (Zhong and Zhu, arXiv:2605.13357, May 2026, preprint) arxiv.org
- From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents (Ahn and Kim, arXiv:2607.08028, July 2026, preprint) arxiv.org
- Building effective agents (Anthropic, December 2024) anthropic.com
- Writing effective tools for agents (Anthropic, September 2025) anthropic.com
- Effective harnesses for long-running agents (Anthropic, November 2025) anthropic.com
- Demystifying evals for AI agents (Anthropic, January 2026) anthropic.com
- A practical guide to building agents (OpenAI, 2025) cdn.openai.com
- Guardrails and human review (OpenAI API documentation) developers.openai.com
- Agentic design patterns, part 1 (Andrew Ng, The Batch, March 2024) deeplearning.ai