What are structured outputs?

What does the API actually do?

The provider compiles your JSON Schema into a grammar and uses it at sample time. At each decoding step the token distribution is masked so only tokens that continue a legal partial parse can be picked, which is called grammar-constrained sampling. When a required field would be missed, closing the object early is not a legal continuation; when an enum lists three allowed values, no fourth string can be produced. The reply is valid JSON that validates against the schema on the first try, so your code can call json.loads or an equivalent without a retry loop.

This is a decoding constraint, not a repair step. The model still chooses the content inside the schema, so a required customer_email field can come out empty, wrong or fabricated even though the shape is guaranteed. If accuracy matters you still need an eval on the values, not just a schema on the envelope.

How you turn it on: OpenAI, Anthropic, Gemini

OpenAI takes the schema on the request. On the Responses API you set text.format to {"type": "json_schema", "name": "...", "strict": true, "schema": {...}}; on Chat Completions the same object goes in response_format. With strict: true the schema is enforced by grammar-constrained sampling, every object must set additionalProperties: false, and every property must appear in required. The feature is supported from GPT-4o onwards.

Anthropic exposes two paths that share the same compiler. For a direct JSON reply you pass output_config.format with type: json_schema and a schema to the Messages API on Claude Opus 4 and newer, Sonnet 4 and newer, and Haiku 4.5. For a tool call whose arguments must match your schema, add strict: true to the tool definition alongside input_schema: the returned tool_use.input is then guaranteed to conform.

Gemini takes the schema on generationConfig. Set responseMimeType to application/json and pass responseSchema with the shape you want; the SDKs let you hand in a Pydantic model or a Zod schema and convert it for you. Gemini accepts a subset of JSON Schema (the OpenAPI 3.0 subset) rather than the fuller draft that OpenAI and Anthropic take.

Where the JSON Schema subset bites

Every provider accepts a subset of JSON Schema, and the parts they drop are usually the parts you reach for. Anthropic's structured outputs and strict tool use compile the same grammar and reject numeric constraints (minimum, maximum, multipleOf), string constraints (minLength, maxLength, pattern in most places), recursive schemas through $ref, and additionalProperties set to anything other than false. OpenAI's strict mode requires additionalProperties: false on every object and every property listed in required, and only a limited set of keywords is honoured. Gemini follows the OpenAPI 3.0 subset, so number and string constraints are limited and some combinators are not accepted.

The failure mode is a compile error at request time, not a bad response. Tighten the schema until the provider accepts it, then validate the numeric ranges, regex patterns and cross-field rules yourself in the calling code.

Structured outputs vs. tool calling vs. JSON mode

Three features get called "JSON mode" in casual writing and mean different things. Plain JSON mode, on OpenAI's older response_format: {"type": "json_object"} and similar options elsewhere, tells the model to return a JSON value but does not enforce a schema, so keys, types and required fields are still your problem. Structured outputs enforce a schema you supply and guarantee the shape at generation time. Tool calling asks the model to fill a schema for a named function; with strict mode on, the argument object is enforced the same way as a structured output.

Pick the tool-call form when the model's next action is to invoke a function and you want the arguments validated. Pick the direct structured-output form when you want a typed value back and no function to dispatch, such as extracting an invoice from an email or classifying a support ticket. Do not stack them: forcing a schema on top of a tool call that already has strict: true splits your validation across two places.

POST /v1/responses
{
  "model": "gpt-4o-2024-08-06",
  "input": [
    {"role": "user", "content": "Extract the invoice fields from this email: ..."}
  ],
  "text": {
    "format": {
      "type": "json_schema",
      "name": "invoice",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "invoice_number": {"type": "string"},
          "total_cents":    {"type": "integer"},
          "currency":       {"type": "string", "enum": ["USD", "EUR", "GBP"]},
          "line_items": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "sku":      {"type": "string"},
                "quantity": {"type": "integer"}
              },
              "required": ["sku", "quantity"],
              "additionalProperties": false
            }
          }
        },
        "required": ["invoice_number", "total_cents", "currency", "line_items"],
        "additionalProperties": false
      }
    }
  }
}

response.output_text:
{
  "invoice_number": "INV-2026-0918",
  "total_cents": 24990,
  "currency": "USD",
  "line_items": [
    {"sku": "A-100", "quantity": 2},
    {"sku": "B-220", "quantity": 1}
  ]
}