What is a session?
A session is a fiction your code maintains. The model call keeps nothing: Anthropic's documentation states that the Messages API is stateless and that a multi-turn conversation is sent in a single request. A session is therefore whatever history, budget and tool state your harness chooses to reassemble and resend every time.
Two meanings, both in circulation
The first is provider side. Some APIs will store the conversation for you and let you refer to it by identifier on the next call, and OpenAI's documentation frames this as one of several ways to manage conversation state, which matters for preserving information across turns. Convenient, and it does not change the underlying mechanics: the stored history is still assembled into the request the model reads.
The second is harness side, and it is the one that matters when you build agents. Here a session is a run: one goal, its own message history, its own token and step budgets, its own set of permitted tools, its own identifier in your logs. Anthropic's Messages API documentation is explicit that the API is stateless and that turns are added to send a multi-turn conversation in a single request, so on this side continuity is entirely your construction.
What a session is actually made of
Six things, in most working systems. The system prompt, which is constant. The kept turns, meaning the subset of history your harness decided to resend. Tool results, which are usually the bulk of the bytes. Retrieved passages, injected per request rather than carried. Counters, for steps taken and tokens spent, so a runaway run can be stopped. And identifiers, so a complaint about one run can be traced to the exact requests it produced.
Note what is absent: anything the model holds. Every item on that list lives in your process or your database, which is why the same conversation can be resumed from another machine, and why a bug in your pruning logic looks exactly like the model forgetting.
Why long sessions fail
They grow, and growth is compounding rather than linear in effect. Each request resends the accumulated history, so the twentieth turn costs several times the first for the same amount of new information. Meanwhile the context window fills, and once it is close to full the harness starts dropping messages: usually the oldest, which is often where the instructions were.
The symptom sequence is recognisable. Costs climb, then latency climbs, then the model starts ignoring rules it followed earlier, then it repeats work it already did. None of that is degradation in the model. It is a session that outgrew what the harness could carry, and the fix is in the session design, not in the prompt.
Designing a session you can debug
Four habits pay for themselves. Give every session an identifier and put it on every log line and every stored request. Store the exact serialised request body, at least for failures, because your template is not what the model saw. Make truncation explicit, logging when messages were dropped and which ones, rather than letting the window silently do it. And end sessions deliberately, with a step cap and a token cap, so a bad run fails instead of running until someone notices.
The alternative to short sessions is not one long session; it is summarising deliberately. Write a compact statement of what was established, start a fresh session with it plus the current goal, and keep the transcript on disk for the audit trail. That converts an unbounded conversation into a bounded one you chose the contents of.
One session, three requests, growing payload
session 7f3a: goal "triage the failing deploy"
request 1 system + user ~ 1.2 K tokens
request 2 system + user + tool_call + 4 KB log ~ 3.1 K tokens
request 3 system + user + all of the above
+ 3 search hits + plan ~ 6.8 K tokens
# request 3 contains everything in requests 1 and 2, verbatim.
# the model did not keep it. your harness resent it.
session record you keep on your side
id : 7f3a
steps : 3 / 12 <- hard cap, ends the run
tokens : 11.1 K / 60 K <- hard cap, ends the run
dropped : none yet <- log it the moment it happens
request bodies : stored per stepThe growth is the design, not a leak: statelessness means continuity costs tokens. The caps are what turn a runaway session into a failed one you can look at.
Common questions
Session: frequently asked
Does the provider remember my session between calls?
The model does not. Some APIs will store the conversation server side and let you reference it by id, which saves you from resending it yourself, but the history is still assembled into the request the model reads. Anthropic's Messages API is documented as stateless, with multi-turn conversations sent in a single request.
Is a session the same as memory?
No. A session is state inside one run: this conversation, its budgets, its tools. Memory is what deliberately survives across runs, such as a user's preferences or facts about a project, and it is normally stored and retrieved rather than carried. Conflating them produces systems that either forget everything or resend everything.
Why does a long session get more expensive per turn?
Because each request resends the whole kept history plus every tool result so far. Input tokens grow with the length of the run while the new content per turn stays roughly constant, so cost per unit of progress rises steadily. That, not the model, is what makes long agent runs expensive.
What happens when a session exceeds the context window?
Either the request is rejected or your harness drops messages to fit, and the second is more dangerous because it fails silently. The model appears to forget instructions and repeat work. Log every truncation, keep must-survive rules in the system position, and prefer a deliberate summary and a fresh session over letting the window decide what to discard.
Sources
Where these facts come from
- Anthropic docs: Messages API: states the API is stateless and that multi-turn conversations are sent in a single request
- OpenAI docs: conversation state: the provider side options for preserving information across turns
- Anthropic docs: context editing: provider tooling for pruning accumulated tool results in a long run