Awesome Testing

API lesson 04 · Run a bounded tool loop

What must happen between a model tool proposal and a real operation?

Shared system · deterministic provider fixtures

Harbor Support Drafting API. One bounded request moves through typed items, tools, state, budgets, and evidence without live provider calls.

01 · Smallest useful mechanism

Build the loop around complete mediation and correlated results.

Tool loops require exact call/result correlation, schema and semantic validation, current authorization, approval for risky operations, bounded execution, sanitized results, and explicit stop conditions.

The model chooses from descriptions; the application decides what may happen.

02 · Experiment

Test the prediction

Deterministic protocol fixture

This workbench uses inspectable provider-event fixtures and injected failures. It does not make live calls or execute external effects.

Protocol workbench · harbor-tools-v3

Choose the harness for a policy lookup that may later expand to effectful tools.

Injected failure

The first response is lost after execution; an untrusted result contains instructions.

Choose the implementation

The trace has not run.

Commit to an implementation, then inject the failure and inspect the actual state transitions.

03 · Implementation brief

Build it, break it, prove the outcome.

Implement
Build a tool acceptance matrix and a bounded loop with immutable call IDs.
Break it on purpose
Mismap parallel results, revoke authority between calls, inject instructions in a result, and lose a write response.
Completion evidence
Every unsafe path produces an auditable zero-effect denial or a reconciled outcome-unknown state.

Your learning artifact

Write the argument you would defend

Stored only in this browser. No account required; course reset does not delete it.

04 · Check your understanding

What grants permission for a tool effect?

Primary sources · verified 2026-08-02

What the fixture does not prove

Function calling changes an LLM from a text component into a participant in workflows that can read private data or cause side effects.

Common mistake: The tool schema is also an authorization policy. A schema describes shape. Authorization depends on the current principal, resource, action, environment, and policy.

Transfer exercise: Send two parallel policy lookups, return their results out of order, and prove the final response still cites the correct call.

Optional referenceDeep dive / reference chapterOpen the complete essay, diagrams, mathematics, exercises, glossary, and sources when you want more depth.

Next: A tool loop needs context across turns, so the next lesson makes conversation and canonical state ownership explicit.