What is a context window?

A context window is the maximum number of tokens a model can take into account in a single request. The system prompt, the entire conversation so far, any documents you pasted in, the definitions of every tool you exposed and the reply the model is about to write all compete for that one budget. It is a per-request ceiling, not a per-account one, and it is published per model.

Written by Derek Wonk · last updated August 2026

What is a context window and how does it work?

Models do not read characters or words, they read tokens: short chunks produced by a tokeniser, where a common English word is often one token and a long identifier or a non-Latin script can be several. The context window is counted in those tokens, and everything you send counts, including the parts you did not write by hand. Tool definitions in particular surprise people, because a dozen JSON schemas can occupy more of the budget than the user's actual question.

The reply shares the same budget. If a model documents a 200,000 token window and your request already occupies 199,000, there is no room left to answer, regardless of how short the answer would be. This is why providers document a maximum output separately: it reserves headroom inside the window rather than adding to it.

Because the limit is per request and per model, it is not something you can raise by upgrading a plan. Anthropic and other providers publish the window on each model's page, and the number moves when models are released, not when your usage changes.

What happens when you exceed it?

Three different behaviours, and knowing which one you are getting matters. A raw API call generally fails outright with an error naming the limit, which is the honest outcome. A chat product will often silently drop the oldest messages so the conversation continues, which looks like the model forgetting the start of your session. A harness written with a sliding window does the same thing on purpose, ideally by summarising what it drops rather than deleting it.

The failure mode to watch for is the middle case combined with an agent loop, because the run keeps going while quietly losing the instructions it was given at the top. If your agent starts ignoring a constraint it obeyed five steps ago, look at what fell out of the window before you look at the prompt.

A context window is not memory

These get used interchangeably and they are different mechanisms. The window is a limit on one request. Memory is state your program stores between requests and chooses to resend. A model with a very large window still starts every request from nothing: if the earlier conversation is present, it is present because your harness put it there.

That distinction is the whole reason retrieval exists. Rather than resending everything and paying for it, you store the material outside the window and send only the passages a query actually needs. See memory for the state side of this and harness for the code that does the assembling.

Is a bigger window always better?

No, for three separate reasons. Cost and latency scale with what you actually send, so filling a large window on every request is a decision with a monthly invoice attached. Attention is not uniform across a long input: the well-known lost in the middle result showed that models retrieve facts placed at the start and end of a long context far more reliably than facts buried in the middle. And a large window makes it easy to defer the work of deciding what is relevant, which is usually the work that most improves the output.

The practical stance is to treat the window as a budget you spend deliberately: put instructions and the most relevant material at the edges, keep tool definitions lean, and summarise history rather than accumulating it.

Where the budget actually goes on a modest agent request

system prompt and house rules      ~   900 tokens
11 tool definitions (JSON schema)  ~ 3 400 tokens   <- usually the surprise
conversation so far (14 messages)  ~ 6 200 tokens
retrieved passages (4 chunks)      ~ 2 100 tokens
-------------------------------------------------
sent                               ~12 600 tokens
room left for the reply            = window - 12 600

Tool definitions and accumulated history usually dominate. Trimming the tool surface is often a bigger win than trimming the prompt.

Common questions

Context windows: frequently asked

Does the context window include the model's answer?

Yes. The window covers the whole request plus the completion, so a request that nearly fills the window leaves no room to reply. Providers document a separate maximum output length, which reserves headroom inside the window rather than extending it. If you get a truncated answer on a long request, the window is the first thing to check.

How many words is a token?

There is no fixed ratio, which is why counting words is unreliable. For ordinary English prose a token is often a short word or a word fragment, so a page of text is usually several hundred tokens. Code, long identifiers, JSON and non-Latin scripts all tokenise less efficiently than prose. If a number matters to you, count it with the provider's own tokeniser rather than estimating.

Is the context window the same as memory?

No. The context window is a per-request limit on how much the model can consider at once. Memory is state stored outside the request that your program chooses to resend. A large window does not give a model memory, because the model still begins each request with nothing but what you sent.

Sources

Where these facts come from

Anthropic docs: context windows: how the window is counted and how output length interacts with it
tiktoken, OpenAI's tokeniser: the tokeniser itself: run it to see why token counts do not map onto words
Lost in the Middle: How Language Models Use Long Contexts: measured drop in retrieval accuracy for material placed mid-context