What is a prompt?

What a real prompt is made of

In a toy example the prompt is a question. In a running system it is an assembled document, and the parts arrive from different places in your code. A system instruction sets the role, the constraints and the tone. The message list carries the turns your harness chose to keep. Tool definitions describe, in schema form, everything the model is allowed to call. Retrieved passages are pasted in by a search step. And somewhere near the end sit your rules about output shape: JSON, a heading structure, a maximum length.

Adding those up changes where you debug. The text a user typed is often the smallest part of the request, so when output goes wrong the first question is not how to reword the instruction but what the full request actually contained. Log the serialised body once and the answer is usually visible immediately.

What actually changes behaviour

The provider guides converge on unglamorous advice. Be explicit about the task and about the output format, because the model cannot infer conventions you never stated. Show examples of a correct answer, since one worked example usually outperforms a paragraph of description. Say what to do when information is missing, so that abstaining is a permitted path rather than something the model has to invent around. And put the material the model must use where it cannot be missed, keeping instructions close to the content they govern.

What tends not to help: politeness, threats, insistence on accuracy, and long lists of prohibitions. A negative instruction still names the thing you do not want, and a rule stated once clearly beats the same rule restated five ways. Anthropic's guidance leads with being clear and direct for exactly this reason.

Prompts drift in long runs

A prompt that works in isolation can quietly stop working inside an agent loop, because the loop keeps changing what surrounds it. Tool results accumulate, the harness starts pruning older messages to stay inside the context window, and the careful instructions you wrote at the top can be among the things trimmed. Nothing errors. The model simply drifts back towards default behaviour, and the run looks like a model regression.

The defence is structural rather than verbal. Keep the rules that must survive in the system position, restate the output contract in the final instruction rather than only at the start, and make pruning explicit in your own logs so you can see when something load bearing was dropped.

Prompts are not code, but version them like code

A prompt is the interface to a probabilistic component, which makes it the least reviewed and most load bearing string in most codebases. Treat it accordingly: keep it in the repository rather than in a dashboard, diff changes in review, and keep a set of examples with expected outcomes so that a rewrite can be judged rather than admired.

The reason is economic. Prompt changes are cheap to make and expensive to evaluate, so without a small evaluation set every edit is a guess that ships. With one, you can answer the only question that matters after a rewrite: did the results get better, and on which inputs did they get worse.

the request the model sees             where it came from
-----------------------------------   -----------------------------
system: role, constraints, format     your repo (versioned)
tools:  [search, read_file, ...]      tool registry, JSON schemas
messages[0..n]: kept history          harness, after pruning
  + tool_result blocks                real tool output, verbatim
context: 6 retrieved passages         retrieval step
user: "summarise the outage"          the person (smallest part)

BAD  "Summarise this nicely, be accurate, don't make things up."

GOOD "Summarise the outage in <= 120 words for an on-call engineer.
      Use ONLY the passages. Cite as [n] after each claim.
      If a cause is not stated in the passages, write: cause not stated."