How does function calling work?

Structured output creates a request, not an effect

An LLM still generates tokens. With function calling, some generated tokens are returned through a structured response channel that names a tool and supplies arguments. The provider does not thereby run arbitrary application code.

sequenceDiagram
  participant A as Application
  participant M as Model
  participant T as Tool runtime
  A->>M: instructions + tool definitions
  M-->>A: tool call ID + name + arguments
  A->>A: parse, validate, authorize
  A->>T: execute the allowed operation
  T-->>A: result, error, or unknown outcome
  A->>M: correlated tool result
  M-->>A: answer or another tool call

The application owns both model requests. This five-part flow—offer tools, receive a proposal, execute outside the model, return the result, continue generation—is the mechanical core beneath many agent systems.

What a function definition teaches the model

A function tool normally includes:

• a stable name;
• a concise description of when the operation is useful;
• an input schema describing fields, types, required values, and enumerations;
• sometimes an output schema or documented result shape.

The definition serves two audiences. The model uses it to choose and populate the tool. Application code uses it as one validation boundary. Ambiguous names, overlapping tools, vague descriptions, and enormous schemas make selection harder and consume context on every call.

A schema can describe issue_id as a string. It cannot prove that the issue exists, belongs to the current tenant, or may be edited by this user. Those semantic and authorization decisions require current application state.

Follow one concrete call

Suppose the user asks, “What failed in build 481?” The application offers:

{
  "name": "get_build_failures",
  "description": "Return failed jobs and concise error summaries for one visible build.",
  "parameters": {
    "type": "object",
    "properties": {
      "build_id": { "type": "integer", "minimum": 1 }
    },
    "required": ["build_id"],
    "additionalProperties": false
  }
}

The model may return a call with build_id: 481. The host then validates the JSON, verifies that the current principal can read build 481, invokes the CI service with bounded credentials, and returns a small result tied to the call's identifier. The next model response can explain the failure using that observation.

If the service times out after receiving the request, the result is not automatically “no failures.” It may be an unknown outcome. Tool result design should preserve errors and uncertainty rather than forcing every response into a success-shaped string.

One model turn may contain several calls

Providers may allow zero, one, parallel, or repeated tool calls. Parallel calls are suitable when operations are independent and side-effect-free. Dependent calls must preserve ordering: a returned identifier from one operation may become an argument to the next.

Iteration limits still belong to the application. A loop needs budgets for model calls, tool calls, elapsed time, and repeated failures. It also needs a stop condition that distinguishes a final answer from abandonment, cancellation, denial, or budget exhaustion.

Function calling, structured output, MCP, and agents differ

Structured output constrains the shape of model-generated data. It may be used without any tool.

Function calling adds a conversational convention for proposing a named operation and returning its result.

MCP standardizes how a host can discover and communicate with independently implemented tool and data servers. A host can translate an MCP tool into a model-visible function definition.

An agent repeats model calls around tools, observations, state, policy, and stopping rules. Function calling is one possible interface inside that loop.

Tool choice is part of context design

Giving the model every operation in a platform is rarely useful. Offer the smallest coherent set for the current task, defer rarely used definitions, and avoid two tools whose descriptions imply the same job. Return compact, documented results that help the next decision rather than dumping entire database objects or logs.

Tool names and descriptions should describe capability, not authority. “Delete account” does not tell the model when deletion is permitted, and wording in a schema cannot replace an execution-time policy check.

Make the boundary observable

Log or trace the model request version, available tool definitions, call identifier, raw proposed arguments, normalized target, validation decision, authorization decision, approval, execution attempt, result classification, and the observation returned to the model. Redact secrets before storage.

This record lets an engineer distinguish failures that otherwise look identical in the final answer: the model chose the wrong tool, generated invalid arguments, lacked permission, hit a transient service error, received stale data, or ignored a valid result.