What is a stop sequence?

What is a stop sequence and how does it work?

Text generation is a token by token loop. After each new token the serving code checks whether the tail of the output matches any string in your stop list. If it does, generation halts, the matched text is stripped from the returned completion, and the response reports which entry fired. If none do, the loop runs until the model emits its own end-of-turn token or the request hits max_tokens.

Anthropic's Messages API takes stop_sequences as an array of strings and reports the match in a stop_sequence field alongside stop_reason: "stop_sequence". OpenAI's Chat Completions API takes a stop parameter that is a string or an array of up to four strings, and reports finish_reason: "stop" when one of them halted the run. Both providers strip the matched string before returning the text, so the completion never appears to end with the trigger.

The check is purely a decoding-time filter. The model does not see your list and cannot reason about it, so a stop that is a substring the model produces naturally, such as a newline or a common word, will cut coherent answers off mid-sentence the first time that substring appears.

Stop reasons: which limit actually ended the run?

Every response names the reason generation stopped, and it is the first thing to inspect when output looks short or truncated. Anthropic's stop_reason currently takes values including end_turn (the model finished on its own), max_tokens (the output cap was hit), stop_sequence (one of yours fired, with stop_sequence naming which), tool_use (the model requested a tool), pause_turn, refusal, and model_context_window_exceeded. OpenAI's per-choice finish_reason takes stop, length, tool_calls, content_filter, and the deprecated function_call.

Treating a truncated reply as a complete one is the classic bug this field exists to prevent. A max_tokens or length stop means the model was still writing when the cap fired, so the text is the start of an answer and often ends mid-sentence or mid-JSON. A stop_sequence stop on ordinary prose usually means the sequence was too short or too generic to belong in the list. A tool_use or tool_calls stop is not an answer at all: it is a request for your code to run a tool and send the result back.

Where stop sequences quietly go wrong

Three failure modes show up repeatedly. Picking a substring the model writes naturally, such as "\n" or "User:", halts answers the first time the model emits a newline or quotes a role name. Using stop sequences to fence structured output almost always loses to structured outputs or a tool call, because the decoder can still produce malformed JSON right up to the stop. And forgetting that the matched text is stripped means a downstream parser expects a closing marker that is no longer present in the response.

Stop sequences also interact with streaming. To detect a multi-token stop the server has to hold back the last few tokens until it knows they are not the start of a match, so a chunk you would expect to arrive immediately can sit briefly while the check completes. The effect is invisible on short one-token stops and noticeable on longer ones. If first-token or inter-token latency matters, keep stops short and few.

The provider's own end-of-turn token is the stop you should rely on first. A well-behaved prompt with a clear task usually lets the model close on end_turn without any custom sequences at all, and custom stops become a last resort for framing markers your prompt introduces on purpose, such as a delimiter between a completion and a ground-truth answer during an eval.

POST /v1/messages
{
  "model": "claude-sonnet-4-5",
  "max_tokens": 200,
  "stop_sequences": ["\n\nHuman:", "</answer>"],
  "messages": [
    {"role": "user",
     "content": "List two primes, then write </answer>."}
  ]
}

response:
{
  "content": [{"type": "text", "text": "2, 3"}],
  "stop_reason": "stop_sequence",
  "stop_sequence": "</answer>",
  "usage": {"input_tokens": 24, "output_tokens": 8}
}