What are structured outputs?

Structure changes the interface

Free-form text is useful when the output is for a person. It becomes fragile when code must recover fields by searching for headings, removing Markdown fences, or guessing whether “yes” means true.

A structured-output mode supplies a schema through the model API. The provider constrains generation so a successful response follows the supported shape. The application can then parse a typed object instead of extracting data from prose.

flowchart LR
  I[Task + input] --> M[Model generation]
  S[Supported schema] --> M
  M --> R{Response state}
  R -->|complete| J[Schema-valid data]
  R -->|refusal| F[Explicit refusal]
  R -->|truncated or failed| E[Incomplete result]
  J --> V[Domain validation]

The response envelope still matters. A refusal, token limit, content filter, interrupted stream, or provider error is not an empty instance of the requested object.

Structured response or function call?

Use a structured response when the model's answer is data for the caller: extracted invoice fields, a classification with reasons, a lesson outline, or a set of UI properties.

Use function calling when the model should choose or prepare a capability owned by the application: search_documents, create_issue, or run_test. The generated arguments describe a proposed call. The harness validates and decides whether to execute it, then returns an observation to the model if the loop continues.

The wire formats may look similar, but the ownership differs:

• structured output: parse and use a generated result;
• function call: evaluate a proposed operation before any effect;
• ordinary text: present language to a person or another language-processing step.

Do not invent a fake function solely to obtain JSON when the API provides a response-schema feature. Do not treat a response object as an action simply because it contains a field named command.

JSON Schema is a vocabulary, not one universal implementation

JSON Schema defines a broad language for describing JSON documents. Model providers normally support a documented subset and may impose additional rules on required fields, optional values, recursion, property counts, nesting, or initial schema compilation.

Treat the provider, model, API version, schema, and SDK as one compatibility unit. Validate schemas at startup or deployment instead of discovering unsupported keywords during user traffic. Keep generated clients and server validators aligned with the same canonical schema.

Strict schemas should still allow legitimate uncertainty. If a field may be unknown, model that state explicitly with null, a tagged union, or a status field. Forcing the model to invent a string because every field is required produces valid fiction.

Parseable is not correct

Schema constraints can establish properties such as:

• the output is an object;
• a required field exists;
• an enum contains one of the declared strings;
• a value has a numeric JSON type;
• no undeclared properties appear.

They do not establish that an email belongs to the current user, a date exists, two totals reconcile, a cited document supports a claim, or a refund is authorized. Those are domain and policy checks.

Perform deterministic validation after parsing. Resolve identifiers against authoritative data. Check cross-field invariants, freshness, access, quantity limits, and evidence. Preserve the raw response envelope and validation errors in the trace without logging secrets.

Failure handling belongs to the contract

Define what the caller does when generation refuses, stops early, returns a transport error, passes the schema but fails domain validation, or produces an unsupported case. Retrying the identical request is not automatically useful, and an automatic “repair” model call can change meaning while hiding the original defect.

For extraction, an explicit unknown result may be safer than a guessed value. For batch work, quarantine invalid records with their versioned input and response metadata. For an agent action, reject invalid arguments without executing and return a bounded error observation so the model can choose another path.

The useful guarantee is narrow and powerful: structured generation moves syntax from prompt folklore into an API contract. Keep truth, authorization, and product semantics in ordinary software.