How to Optimize Tool Calling for AI Agents

Many teams discover the gap between having tools and using them reliably only after a successful demo. The agent works in controlled examples, then real users ask messier questions, combine multiple requests, omit context, or use phrasing the system was not tested against. The challenge is reliability: there is a wide gap between an agent that has access to tools and an agent that selects the right tool, supplies the right inputs, completes the task, and does so at a cost suitable for production.

To optimize tool calling, improve the full path from user request to tool selection, input generation, and task completion. Model choice has the largest measured effect in Paragon's own evals, but it is not the first lever most teams can pull: tool descriptions, schemas, atomic tool design, dynamic filtering, and ongoing evals are what a team controls once a model is chosen, and they determine how much of the gap a smaller, cheaper model can close.

What It Means to Optimize Tool Calling

Tool calling — also called function calling in some model providers' documentation — should be measured across four separate dimensions.

Tool correctness and task completion often diverge in practice: an agent can select the right tool and still fail the request if a downstream step breaks, or it can complete a task despite an early misstep it recovers from. Paragon's own evals showed this directly — the model that picked the right tool most often was not the model that finished the most tasks. Input accuracy failures compound the same way: a correctly selected tool called with the wrong customer ID or date range still produces a wrong result. Task efficiency is the cost lens on all three — a system that eventually gets the right answer by retrying, over-fetching, or loading unnecessary tools can still be too slow or expensive for production.

These metrics should be tracked separately. If you only measure whether the task worked, you will not know whether failures come from poor tool selection, bad argument extraction, weak tool design, missing permissions, or excessive context.

What the Benchmark Data Shows

LLM choice had the largest measured effect on tool-calling performance in Paragon's evals across six SaaS providers. Prompt wording, description depth, and routing moved the numbers too, but mostly on multi-tool tasks — on single-tool tasks their effect was close to flat. Tool-calling reliability is one layer of the larger problem of connecting agents to third-party systems; see the agent integration infrastructure overview for how it fits with authentication, sync, and orchestration.

We ran these benchmarks with DeepEval's tool-calling and task-completion metrics across 50 test cases spanning six third-party providers: Salesforce, HubSpot, Slack, Gmail, Google Drive, and Notion. Thirty-six cases used a single tool; fourteen chained multiple tools. Each case scored 0 to 1 on Tool Correctness (right tool picked) and Task Completion (task finished). The run was April 2025, against gpt-4o, o3-mini, Claude 3.5-Sonnet, and an o3-2025-04-16 snapshot.

One limit is worth stating plainly, and it is why the findings below are reported as directions rather than as scores. The prompt, description, and routing results rest on the fourteen multi-tool cases in that set, and each configuration was a single run with no confidence intervals or repeated trials. The models were also April 2025 vintage. Absolute numbers from a run that old would tell you more about those specific model snapshots than about the choices in front of you, so what follows is the shape of the result — which lever moved which metric, and in which direction.

Which lever moved which metric

Model choice moved the numbers most. Swapping the underlying LLM produced larger and more consistent swings than any prompt, schema, or routing change tested. It was also the clearest case of the two metrics pulling apart: the model that picked the right tool most often was not the model that completed the most tasks.

Prompt wording and description depth only mattered on multi-tool work. Across the full fifty cases, rewriting the system prompt or adding detail to tool descriptions barely moved either metric. On the fourteen tasks that chained several tools, both helped task completion meaningfully. Single-tool calls were close to flat — a useful signal about when prompt and description work is worth the effort.

Routing to a narrower tool set was model-dependent, and cut both ways. Narrowing the exposed set from roughly twenty tools to around five had almost no effect on one model. On another it raised tool correctness while lowering task completion in the same run. That is why the filtering guidance below is framed per model rather than as a blanket rule: fewer tools is not automatically better, and the metric you optimize for decides the answer.

The practical read: model choice produced the biggest swings, but tool design is the lever a team actually controls once a model is chosen, and it decides whether a smaller, cheaper model can close the gap with a frontier one. Treat tool design as the most controllable lever here, not the biggest swing.

Optimizing Tool Calling for AI Agents

Start With Tool Design

When tool calling fails, many teams first consider upgrading to a larger model. This can help in some cases, but it is usually the wrong first lever. Tool design is the most controllable lever within a given model, because the model can only reason over the tools and schemas it receives.

A tool description is part of the prompt. If the description is too short, the model may not understand when to use the tool. If the description is too long, it adds token cost every time the tool is loaded and may distract the model with unnecessary detail.

The goal is a concise but informative description. It should explain what the tool does, when to use it, what it returns, and any important constraints. It should not include generic marketing language, irrelevant implementation detail, or lengthy examples unless they materially improve tool selection.

For example, "Search Notion" is too vague. "Searches connected Notion workspaces for pages matching a natural-language query and returns page titles, IDs, URLs, and matching snippets" gives the model enough context to choose the tool correctly.

Input design carries just as much weight. Engineers often expose tools in a way mirroring backend function arguments or raw API endpoints. This can work well for code, but it is harder for a language model. The model has to infer every argument from natural language, and each nested field adds another chance for failure.

Flatter schemas are usually better. A tool with one well-described query parameter can outperform a tool with deeply nested parameters, even if both eventually call the same backend API. The model's job should be to express the user's intent clearly. The server-side tool implementation can handle the API-specific details.

Wrap Multi-Step Workflows Into Atomic Tools

Many useful agent actions require more than one API call. Searching for a document may require one request to find matching files and another request to retrieve the file content. Creating a CRM follow-up task may require finding the contact, validating the account, and then creating the task.

If each API endpoint is exposed as a separate tool, the agent has to orchestrate the sequence. This creates more failure points. It may call the first tool correctly, misunderstand the result, skip the next step, or pass the wrong ID into the second call.

A better approach is to wrap common multi-step sequences into atomic tools. Instead of exposing "search page" and "get page contents" separately, expose a tool searching for a page and return its contents. Instead of exposing separate low-level CRM endpoints, expose a tool finding a contact and create a note or task in one operation.

Atomic tools reduce reasoning burden. They also make evaluation easier because the expected behavior is clearer. The agent does not need to know how the external API is structured. It only needs to understand the user's task and choose the tool designed for the task.

The tradeoff is flexibility. Atomic tools are more opinionated than raw endpoint tools. They should be designed around common product workflows. For production agents, the tradeoff is usually worth it: reliability outweighs exposing every backend primitive to the model.

Load Only the Tools the Task Needs

Every tool placed in the model's context has a cost. It consumes tokens, increases prompt size, and gives the model another possible option to consider. A large tool set can make an agent less accurate because the model must distinguish between many similar tools.

Tool filtering is one of the highest-impact optimizations for production agents. Instead of loading every available tool, load only the tools relevant to the current task, integration, user permissions, or conversation state.

If a user is working inside Salesforce, the agent may not need tools for Slack, Notion, Google Drive, and Zendesk. If the user asks to summarize documents, the agent may need retrieval tools but not write-action tools. If the user has not authenticated a specific integration, those tools should not be available at all.

This improves both accuracy and cost. The model sees fewer choices, so tool selection becomes easier. The prompt also becomes smaller, reducing input token usage. In agent systems with dozens or hundreds of possible actions, dynamic tool filtering is not optional. It is part of the reliability layer.

That said, filtering is a correctness lever more than a guaranteed win. In Paragon's evals, routing to a narrower tool set raised tool correctness on one model while lowering its task completion in the same run, and made almost no difference on another. Test it against your own model before applying it uniformly.

Match the Model to the Harness

Model choice is one input among several; it should be evaluated together with the harness, tool schemas, descriptions, and task set. A frontier model is not automatically the best tool-calling model for every agent.

Model size and tool-calling reliability do not move in lockstep. A smaller, cost-efficient model paired with well-designed tools can often complete a task about as reliably as a frontier model while consuming meaningfully fewer input tokens, particularly on the single-tool tasks common in production agents. In some cases, that is accurate enough for production at a much lower cost. In other cases, a more capable model may be justified because the task requires complex planning, ambiguity resolution, or multi-step reasoning.

The key is to test combinations. A model can perform well with one provider's tools but may perform differently with another provider's schema style. A tool set with simple atomic actions may work well with a smaller model, while a raw API-style tool set may require more reasoning capacity.

Teams should evaluate model and harness combinations using their own tasks. Public benchmarks are useful for orientation, but production agents fail on the details of a specific workflow, user base, and integration set.

Build an Evaluation Loop

Tool-calling quality should be tested the way software behavior is tested. Without an evaluation suite, teams end up relying on demos, intuition, and manual spot checks. This is not enough for production agents.

A useful evaluation suite does not need to be large at first. Start with 15 to 30 representative tasks. Include straightforward prompts, vague prompts, edge cases, and prompts resembling real user behavior. Real users often omit context, use shorthand, combine requests, or ask for something the agent cannot safely complete.