Awesome Testing

API lesson 03 · Constrain structured output

What does structured output guarantee—and what must your code still check?

Shared system · deterministic provider fixtures

Harbor Support Drafting API. One bounded request moves through typed items, tools, state, budgets, and evidence without live provider calls.

01 · Smallest useful mechanism

Separate syntax, schema, semantic rules, and authority.

Structured-output modes constrain generated shapes. Applications still validate provider-supported schema subsets, domain invariants, source evidence, permissions, and safe handling of refusals or incomplete generations.

A valid object is a candidate for business logic, not proof that the business claim is correct.

02 · Experiment

Test the prediction

Deterministic protocol fixture

This workbench uses inspectable provider-event fixtures and injected failures. It does not make live calls or execute external effects.

Protocol workbench · harbor-schema-v1

Question to consider

Which validation pipeline can safely display a return decision?

Complication

The candidate is schema-valid but references another customer and an expired policy.

What would you do?

Choose the approach you would be prepared to defend in a design review.

Consider the trade-offs first.

Choose an answer, then run the scenario. You’ll see what the system checks, what happens under the complication, and where the control boundary holds or breaks.

03 · Implementation brief

Build it, break it, prove the outcome.

Implement
Define a versioned output schema and a separate semantic validator.
Break it on purpose
Supply impossible dates, a cross-tenant ID, an unknown evidence ID, and a refusal.
Completion evidence
Each branch produces a stable error category and no invalid candidate reaches downstream code.

Your learning artifact

Write the argument you would defend

Stored only in this browser. No account required; course reset does not delete it.

04 · Check your understanding

Which guarantee can strict structured output provide?

Primary sources · verified 2026-08-02

What the fixture does not prove

Teams often remove defensive validation after enabling a strict mode, then execute plausible but unauthorized or internally inconsistent values.

Common mistake: Strict structured output makes hallucinations impossible. It can guarantee supported structural constraints, not truth, freshness, authorization, or correspondence to external evidence.

Transfer exercise: Generate a schema-valid return decision whose deadline predates the purchase and whose customer ID belongs to another account.

Optional referenceDeep dive / reference chapterOpen the complete essay, diagrams, mathematics, exercises, glossary, and sources when you want more depth.

Next: Tool calling uses the same structured boundary, then adds the possibility of a real external effect.