Awesome Testing

API lesson 01 · Read the API contract

What must your application know before it can trust an API response?

Shared system · deterministic provider fixtures

Harbor Support Drafting API. One bounded request moves through typed items, tools, state, budgets, and evidence without live provider calls.

Prior knowledge
HTTP and JSON basics help. The LLM and agent courses provide useful context, but no provider SDK experience is required.
Study time
3–4 hours guided · about 8 hours of optional deep dives
By the end
Build a provider-portable LLM API boundary with typed events, structured output, bounded tools, explicit state, reliable recovery, and evaluation-ready traces.

01 · Smallest useful mechanism

Model the exchange as typed items and explicit terminal states.

Modern LLM APIs can return text, refusals, tool proposals, tool results, usage, and intermediate events. Your application should preserve and interpret those items before rendering a final answer.

The final text is one projection of a richer protocol trace.

02 · Experiment

Test the prediction

Deterministic protocol fixture

This workbench uses inspectable provider-event fixtures and injected failures. It does not make live calls or execute external effects.

Protocol workbench · harbor-request-v1

Choose the adapter behavior that preserves enough evidence for a product decision.

Injected failure

The response contains a draft and a tool call, then ends as incomplete.

Choose the implementation

The trace has not run.

Commit to an implementation, then inject the failure and inspect the actual state transitions.

03 · Implementation brief

Build it, break it, prove the outcome.

Implement
Define an internal request, item, usage, error, and terminal-state vocabulary; retain a redacted native trace reference.
Break it on purpose
Add an unknown output item and an incomplete status to an otherwise plausible draft.
Completion evidence
The UI does not claim success, the unknown item is preserved, and the trace explains the decision.

Your learning artifact

Write the argument you would defend

Stored only in this browser. No account required; course reset does not delete it.

04 · Check your understanding

Which signal determines whether the product operation completed?

Primary sources · verified 2026-08-02

What the fixture does not prove

Flattening a response into text discards the evidence needed for safe tools, retries, debugging, cost attribution, and provider portability.

Common mistake: If response text is non-empty, the request succeeded. Useful text can accompany truncation, refusal, tool use, or an incomplete operation. Success depends on the terminal state and product contract.

Transfer exercise: Compare the same request as raw OpenAI, Anthropic, and Gemini fixtures; map each item to one internal event vocabulary.

Optional referenceDeep dive / reference chapterOpen the complete essay, diagrams, mathematics, exercises, glossary, and sources when you want more depth.

Next: Once the item contract is explicit, streaming becomes a sequence of typed state transitions rather than animated text.