How should agent tool calls be controlled?

Proposal, decision, effect

A model may return a tool name and JSON arguments. That output is a proposal. The harness parses it, chooses the registered tool, validates the arguments, resolves referenced resources, checks current authorization, and only then permits an effect.

Only the allowed route reaches execution. A model proposal and a schema-valid payload remain on the non-authoritative side of the gate.

flowchart LR
  M[Model tool proposal] --> S[Schema validation]
  S --> V[Semantic validation]
  V --> P[Policy with current identity]
  P -->|ask| A[Exact approval]
  P -->|deny| D[Denial observation]
  P -->|allow| X[Bounded execution]
  A --> X
  X --> R[Result and effect record]
  R --> M

A JSON Schema can require fields, types, enumerated values, and structural constraints. It cannot decide whether the current user owns account 123, whether a file path escapes an allowed root after symlink resolution, or whether sending this email is appropriate now. Those are semantic and authorization checks owned by application code.

Function calling is an interface, not execution

During generation, the model may choose a tool name from the descriptions in its context and produce arguments shaped like the declared schema. The provider can return that proposal in a structured response rather than ordinary prose. Application code still decides whether the structure is valid, whether that tool exists now, and what observation is sent back after the attempt.

The model does not cross the network, click the button, or call the function by naming it. A host or harness maps the proposal to executable code. This distinction also applies to MCP: the protocol can standardize discovery and messages while the host remains responsible for exposure, consent, policy, credentials, and effects.

Permission belongs at the execution boundary

Instructions such as “ask before deleting” can influence model behaviour but cannot enforce it. The permission decision belongs between proposal and execution, where the harness has the real user identity, resolved target, current environment, policy, and consequence level.

Least privilege limits both functionality and reach. Prefer a scoped read_issue operation over an unrestricted shell when that is the required capability. Constrain credentials, paths, hosts, methods, resource IDs, data volume, time, and number of calls. A sandbox limits some effects but does not decide business authorization.

Approval must bind to an exact action

An approval should show what will happen and bind to immutable arguments, identity, target, and expiry. “Allow the agent to manage email” is not equivalent to “send this draft to these two recipients once.” If the model changes an argument after approval, the action requires a new decision.

Denial and tool errors should return structured observations the loop can handle. The model may choose a safer alternative or ask a clarifying question; it must not reinterpret denial as permission to route around the control.

Side-effecting tools need idempotency and effect identifiers. Retries after timeouts must distinguish “did not happen” from “happened but response was lost.”

Schemas do not authorize effects

A valid schema is not authorization. A model-generated explanation is not proof of user intent. A confirmation dialog is not useful if it hides the exact target, appears too often to read, or arrives after the effect.

Read-only is not universally harmless: reading secrets, private messages, or large datasets can be consequential. Conversely, a broad deny list cannot enumerate every dangerous command or encoded variation.

Exercise policy, approval, and recovery

Test the tool registry and policy without a model. Cover unknown tools, missing and extra fields, type confusion, encoded paths, symlinks, Unicode confusables, oversized values, injection strings, stale identity, revoked access, cross-tenant IDs, and unsupported methods.

For approvals, prove fail-closed behaviour on timeout, disconnect, malformed response, non-interactive execution, changed arguments, expired grants, and replay. Verify that the UI displays the resolved target and that audit records connect proposal, decision, approver, execution, and effect.

Inject tool timeouts, partial success, duplicated requests, lost responses, and cancellation races. Assert idempotency where required and ensure an error result cannot be mistaken for success. In end-to-end agent trials, include tempting paths that would complete the task faster by exceeding authority; success requires both the outcome and the permitted trajectory.