Tool Calling: How a Model Calls Tools
Tool calling gives an LLM “hands”: calling APIs, databases and tools in an agent loop. The foundation of AI agents and a frequent interview question for engineers.
Intuition
Imagine a very erudite consultant locked in a room with no windows. He reasons brilliantly, but does not know the weather outside and cannot press any button. All he can do is write notes. Tool calling is an agreement: if the consultant writes a note in a strictly defined format (“call get_weather for Moscow”), then the assistant behind the door — your code — will execute the request and slip a sheet with the result back in. The consultant remains a text generator; only the communication protocol changed.
How it works
The tool-calling loop has four steps:
- The developer describes the tools. For each — a name, a human-readable description and a JSON Schema of parameters: which arguments, of which types, which are required. These descriptions go into the request together with the messages.
- The model responds with a structured tool call. Instead of ordinary text it generates JSON: tool name + arguments. Important: the model executes nothing — it only produced text in the agreed format.
- Your code executes the call. It parses the JSON, validates the arguments, hits the real API and puts the result back into the context as a message with role tool.
- The model continues generation — now seeing the results, and answers the user in words (or asks for another tool).
Modern models support parallel tool calls: several tool calls in one response, and your code can execute them at the same time. A classic example — “weather in Moscow and St. Petersburg”: one model response, two calls, two results.
Description quality = call quality
The model picks a tool and fills arguments looking only at the name, description and schema. A tool description is prompt engineering: “get_data” with an empty description almost guarantees junk calls, while “returns the current weather by city name; use for any weather questions” — sensible ones. Good practice: describe not only “what it does” but also “when to use it”, give argument formats and typical values.
A related mechanism is structured output / JSON mode: the model is forced to emit strictly valid JSON under a given schema, but without subsequent execution. In essence tool calling is structured output with an agreed execution step attached: both mechanisms rest on the same ability to generate text to a schema (often with constrained decoding — masking tokens that break the grammar).
Typical mistakes and MCP
- A tool dump. 40 tools with one-line descriptions: the model confuses similar tools and pulls the wrong one. Cure: shrink the set, write clear descriptions, group them.
- Invalid arguments. The model can emit JSON that fails the schema (wrong type, extra field). The right code reaction is to validate and return an error text to the model: it almost always fixes itself on the second try.
- Tool hallucination. The model calls a tool that is not on the list. The code should answer “no such tool, available: …”, not crash and not execute a “similar” one.
MCP (Model Context Protocol) is an open standard for connecting tools and data sources to models. Instead of writing an integration for every “app × tool” pair, a server describes its tools once under the protocol, and any MCP-compatible client can use them. It standardizes transport and descriptions; the call mechanics stay the same.
- “Does the model itself execute tools?” — no, it generates a structured request; execution is always on your code's side. This is the main trap question.
- “How do you improve call accuracy?” — fewer tools, detailed “what and when” descriptions, strict JSON schemas, returning validation errors to the model for a retry.
- “What is MCP and why?” — a standard for describing and connecting tools: write the server once, use it from any client.
- “How does tool calling differ from JSON mode?” — both are about generating to a schema; tool calling adds the loop “call → execute → result into context → continue”.
All interview questions in this section: Agents and Tools — interview questions