Glossary/H/harness
Agent architecture Also called: agent harness, scaffold

What is a harness?

A harness is the program that wraps a model: it owns the conversation, the tool definitions, the filesystem access, the loop and the stopping rules. Because the model keeps nothing at all between calls, the harness is what makes a session feel continuous, and it does that by reassembling and resending the entire relevant state on every single request.

Written by Derek Wonk · last updated August 2026

What does a harness actually do?

Four jobs, and every agent framework you have used is an opinionated version of them. It assembles the request: system prompt, whatever history it has decided to keep, any retrieved material, and the definitions of the tools it is willing to expose. It executes tool calls the model asks for and decides how to report the results, including failures. It enforces budgets: token, step and wall-clock limits, plus any guard that has to approve an action before it happens. And it decides when the run is over.

Notice what is not on that list. The harness does not think, and the model does not act. Every side effect in an agent run is performed by harness code, which is also why the harness is where security lives.

The model is stateless, so the harness resends everything

This is the single most useful fact to internalise, because so much confusing behaviour follows from it. A model call is a pure function of its input. There is no session on the provider's side that remembers your previous turn, no hidden handle, nothing accumulating server side. When a conversation appears to have continuity, it is because the harness sent turn one, then turns one and two, then turns one, two and three, and so on.

Three consequences fall straight out of that. Token usage per request grows through a session even if your messages stay short, so the last step of a long run can cost many times the first. Anything the harness drops is genuinely gone from the model's point of view, no matter how important it was, which is why a long agent run can quietly stop honouring an instruction from the top of the session. And the shape of a conversation is entirely the harness's choice: it can summarise, prune, reorder or retrieve, and the model has no way to know the difference.

Prompt caching does not change the mechanism, it changes the price. Providers let you mark a stable prefix so repeated tokens are cheaper to process on subsequent calls, but you are still sending them. Statelessness is the design, not an optimisation gap.

Harness vs framework vs agent: what is the difference?

The agent is the behaviour: a model choosing its own next step in a loop. The harness is the code that makes that loop possible in your process. A framework is a reusable harness someone else wrote, with conventions for tools, tracing and retries attached. You can have a harness with no framework, which is a good first exercise, and a framework you use in a workflow with no agentic step at all.

The Model Context Protocol sits at the same layer from a different angle: rather than each harness inventing its own way to expose tools and data, MCP standardises that interface so a tool server can be reused across harnesses.

What separates a good harness from a working one

Manage the context budget explicitly rather than appending forever: summarise old turns, keep the tool surface narrow, and know what your request costs before you send it. Report tool failures back to the model as data, not as exceptions that kill the run, because a model handed a clear error message will usually correct itself. Make tools idempotent or gate the destructive ones behind an approval, since the loop will retry. And log the full request and reply for every step, because after a bad run the trace is the only thing that tells you whether the model reasoned badly or the harness fed it something wrong.

These are the operational concerns of running untrusted-ish work repeatedly and reproducibly, which is why harnesses tend to end up wanting an isolated environment per run, scoped credentials and a durable log. If that list sounds like a continuous delivery pipeline, that is because it is one.

The same session, seen as three separate stateless requests

request 1  ->  [system] [user: "summarise ci.log"]
               tools: read_file, search_repo
               ~1.2 K tokens

request 2  ->  [system] [user] [assistant: call read_file] [tool result: 4 KB]
               tools: read_file, search_repo
               ~5.6 K tokens        <- turn 1 resent verbatim

request 3  ->  [system] [user] [assistant] [tool result]
               [assistant: call search_repo] [tool result]
               tools: read_file, search_repo
               ~8.9 K tokens        <- turns 1 and 2 resent verbatim

the provider stored nothing between these three calls

Continuity is an illusion the harness maintains at its own cost. This is also why a step cap and a token cap are not optional.

Common questions

Agent harnesses: frequently asked

Is the harness the same thing as the agent?

No. The agent is the pattern of a model choosing its own next action in a loop. The harness is the program that runs that loop: it assembles each request, executes the tool calls, enforces the budgets and decides when to stop. You can build a harness that never behaves agentically, and the agentic behaviour is impossible without a harness underneath it.

Why does my token usage grow every turn if my messages are short?

Because the model keeps nothing between calls, so the harness resends the whole conversation each time, including every earlier tool result. A short user message appended to a long history is still a long request. Prompt caching can make the repeated prefix cheaper, but the tokens are still being sent, so summarising or pruning history is the only thing that actually shrinks the request.

Do I need a framework to write a harness?

No, and writing one by hand first is worth the afternoon. A minimal harness is a loop that sends a request, checks the reply for a tool call, runs it, appends the result and repeats until a stop condition. Frameworks add value once you need many tools, tracing, retries and evaluation, but they also hide exactly where your tokens go, which is the thing you most need to see early.

Does the harness decide what the model remembers?

Entirely. The model can only see what arrived in the current request, so whatever the harness chose to include is the model's whole world for that call. That is why a long session can stop honouring an instruction from the beginning: the harness dropped it to stay inside the context window, and nothing in the model's reply will tell you that happened.

Sources

Where these facts come from

agent session context window tool calling