What are logprobs?

What does a logprob measure?

At every step of generation the model produces a probability for each token in its vocabulary. A logprob is the natural log of the probability of one of those tokens. OpenAI's cookbook defines it as log(p), where p is the probability of a token occurring at a specific position based on the previous tokens in the context. A logprob of 0.0 means the model put all of its probability on that token; -0.69 is about 50 percent; -4.6 is about 1 percent. To get the probability back, exponentiate: p = exp(logprob).

APIs return logs instead of raw probabilities because the numbers get very small very fast, and logs add where probabilities multiply. The probability of a whole sequence is the product of its token probabilities, so its log is the sum of the token logprobs. That sum is the quantity you compare when you rank candidate answers or compute perplexity.

A logprob describes one token in one context. It says how likely the model found the next piece of text. It does not say whether the claim in the text is true, so a fluent, confident hallucination can carry high logprobs on every token.

How do you request logprobs from an API?

Each provider names the parameters differently, and the requests look much the same. OpenAI's Chat Completions API takes logprobs: true to return the logprob of each output token, and top_logprobs, an integer from 0 to 20, to also return that many most likely alternatives at each position. The response puts them under choices[].logprobs.content, one entry per token with token, logprob, bytes and a top_logprobs list.

The Gemini API uses responseLogprobs: true in the generation config, and logprobs, also in the range 0 to 20, for the number of top tokens including the chosen one. Each candidate then carries a logprobsResult and an avgLogprobs score. vLLM's SamplingParams takes logprobs as an integer and follows the OpenAI convention of always including the sampled token, so you can get up to logprobs + 1 entries per position. vLLM also offers prompt_logprobs, which scores the tokens of your input as well as the output.

Support is per model and per endpoint, so check the reference for the exact model you call. The Gemini reference lists responseLogprobs and logprobs among the generation config fields that some calls do not support.

What developers use logprobs for

• Classification confidence: ask for a one-word label and read the logprob of the label token. OpenAI's cookbook works through this with news article categories.
• Ranking fixed choices: with top_logprobs set, one call returns the model's probability for each candidate label at the first position, which is cheaper than sampling many times and counting.
• Perplexity: exponentiate the negative mean logprob of a sequence. Lower perplexity means the model found the text more predictable.
• Debugging sampling: when a run produces an odd token, the top alternatives at that position show whether the model was split between options or whether temperature or top-p let a low-probability token through.

Treat any threshold you set on these numbers as something to tune. The cookbook's own advice for its retrieval example is to validate the classifications and thresholds with task-specific evals before you use them to restrict answers.

Where logprobs mislead

Logprobs are per token, and one word can be several tokens. A label like negative may be one token in one tokenizer and two in another, so compare the first token of each label only when every label starts with a distinct token. OpenAI returns bytes for each token because a single character can span several tokens, and you need the bytes to rebuild the text exactly.

The top list is truncated. OpenAI's reference notes that the returned list can hold fewer entries than you asked for, and that a token outside the top 20 is reported with a logprob of -9999.0 to mark it as very unlikely. Do not average that sentinel into a confidence score.

Each model and model version assigns its own probabilities, so a threshold tuned on one may not hold on the next. Re-tune it when you change models.

POST /v1/chat/completions
{
  "model": "gpt-4.1-mini",
  "messages": [
    {"role": "system",
     "content": "Label the review positive, neutral or negative. Reply with one word."},
    {"role": "user",
     "content": "Setup took two minutes and it has not crashed once."}
  ],
  "max_completion_tokens": 1,
  "logprobs": true,
  "top_logprobs": 3
}

response, choices[0]:
{
  "message": {"role": "assistant", "content": "positive"},
  "logprobs": {
    "content": [
      {"token": "positive", "logprob": -0.0143,
       "bytes": [112, 111, 115, 105, 116, 105, 118, 101],
       "top_logprobs": [
         {"token": "positive", "logprob": -0.0143},
         {"token": "neutral",  "logprob": -4.6052},
         {"token": "negative", "logprob": -6.0}
       ]}
    ]
  },
  "finish_reason": "length"
}

exp(-0.0143) = 0.9858   positive
exp(-4.6052) = 0.0100   neutral
exp(-6.0)    = 0.0025   negative