What does an LLM API provide, and what must your application still own?

Shared system · deterministic provider fixtures

Harbor Support Drafting API. One bounded request moves through typed items, tools, state, budgets, and evidence without live provider calls.

Prior knowledge: HTTP and JSON basics help. The LLM and agent courses provide useful context, but no provider SDK experience is required.
Study time: 3–4 hours guided · about 8 hours of optional deep dives
By the end: Build a provider-portable LLM API boundary with typed events, structured output, bounded tools, explicit state, reliable recovery, and evaluation-ready traces.

Start here

What it is, when it helps, and where we are going.

An LLM API is a network boundary through which an application sends typed inputs and receives model-generated items, events, usage, errors, and tool proposals under a provider contract.

Good use cases

Add language or multimodal generation to a product without operating the model-serving stack yourself.
Combine model output with application-owned retrieval, validation, tools, state, and user interfaces.
Switch or compare providers behind a stable internal contract while preserving important semantic differences.

What it does not solve

The provider API does not own your product policy, semantic validation, tool authorization, durable state, retry safety, or evidence that the resulting feature is fit to release.

Learning path

01One complete request
02Streaming
03Structured output and tools
04State and recovery
05Security and release

01 · Smallest useful mechanism

Begin with the job of the API boundary and one complete request.

An LLM API lets application code send instructions and typed inputs to a hosted model and receive generated items, usage, errors, and terminal status. Teams use it for drafting, transformation, extraction, classification, multimodal interpretation, and tool-assisted workflows. The API does not decide whether an answer is true, an action is authorized, or the resulting feature is ready to ship.

The provider generates model output; your application still owns the product contract around it.

02 · Experiment

Test the prediction

Deterministic protocol fixture

This workbench uses inspectable provider-event fixtures and injected failures. It does not make live calls or execute external effects.

Protocol workbench · harbor-request-v1

Question to consider

Which adapter behavior preserves enough evidence for a product decision?

Complication

The response contains a draft and a tool call, then ends as incomplete.

What would you do?

Choose the approach you would be prepared to defend in a design review.

Return output text onlyDiscard item types and treat non-empty text as success.Expose raw provider JSON everywherePreserve evidence but couple product code directly to one provider.Normalize and retain native evidenceMap semantic events while keeping the original trace for diagnosis.

Consider the trade-offs first.

Choose an answer, then run the scenario. You’ll see what the system checks, what happens under the complication, and where the control boundary holds or breaks.

03 · Implementation brief

Build it, break it, prove the outcome.

Implement: Define an internal request, item, usage, error, and terminal-state vocabulary; retain a redacted native trace reference.
Break it on purpose: Add an unknown output item and an incomplete status to an otherwise plausible draft.
Completion evidence: The UI does not claim success, the unknown item is preserved, and the trace explains the decision.

Your learning artifact

Write the argument you would defend

Stored only in this browser. No account required; course reset does not delete it.

Your response

04 · Check your understanding

Which signal determines whether the product operation completed?

Primary sources · verified 2026-08-02

What the fixture does not prove

Starting from the complete exchange makes streaming, schemas, tools, state, retries, cost, and traces extensions of one understandable boundary. Flattening that exchange into text discards the evidence those responsibilities require.

Common mistake: If response text is non-empty, the request succeeded. Useful text can accompany truncation, refusal, tool use, or an incomplete operation. Success depends on the terminal state and product contract.

Transfer exercise: Name one product use case, write the smallest complete request and response, then mark which decisions belong to the provider and which must remain in your application.

OpenAI — Migrate to the Responses APIOfficial guidance for the item-based Responses API, built-in tools, stateful continuation, and migration from Chat Completions.Anthropic — Messages APIOfficial stateless message request contract and typed multimodal content representation.Google — Function calling with the Gemini APIOfficial function-calling contract, including sequential, parallel, streamed, and stateless interaction patterns.

Optional referenceDeep dive / reference chapterOpen the complete essay, diagrams, mathematics, exercises, glossary, and sources when you want more depth.on demand

Next: Once the item contract is explicit, streaming becomes a sequence of typed state transitions rather than animated text.