MCP Tool Calling at Scale: Cutting Token Cost

TL;DR

• Connecting 16 MCP servers to one agent puts 508 tool definitions into the model's context on every request, which consumed 75.1 million input tokens across a 65-query benchmark run.
• MCP tool calling degrades as tool count grows because tool definitions are re-sent on every turn and every intermediate result passes back through the model.
• Bifrost Code Mode exposes four meta-tools instead of the full catalog and reduced input tokens by 92.8% and estimated cost by 92.2% at 508 tools across 16 servers.
• Tool filtering in Bifrost stacks at three levels, client configuration, request headers, and virtual key, and a tool must pass all applicable filters to reach the model.
• Bifrost adds 11 microseconds of overhead per request at 5,000 requests per second, so centralizing tool calling does not trade latency for control.

Connecting 16 MCP servers to a single agent puts 508 tool definitions into the model's context on every request. MCP tool calling at that scale spends most of the context budget on reading tool catalogs rather than on the task, and it raises cost and latency together. Bifrost, the open-source MCP gateway built in Go by Maxim AI, addresses this by filtering the tool set per credential and by letting the model write code against tools instead of calling them one at a time. This post covers why MCP tool calling breaks down as tool count grows, and what keeps it efficient at 500 tools.

What Is MCP Tool Calling?

MCP tool calling is the pattern where a language model discovers and invokes external tools at runtime through Model Context Protocol servers rather than through hardcoded integrations. The model receives tool definitions in its context, returns a structured call, and a runtime executes that call against the server that owns the tool.

Three components make up the flow:

• MCP server: the process that exposes tools, each with a name, description, and JSON schema for its parameters
• MCP client: the component that connects to servers, collects their tool definitions, and presents them to the model
• Execution runtime: the layer that receives the model's tool call, executes it, and returns the result into the conversation

Model Context Protocol is an open standard, so a tool written once is reachable by any compliant client. Bifrost operates as both an MCP client and an MCP server, connecting to external tool servers and exposing the combined set to clients such as Claude Desktop. Tool execution is explicit by default: a tool call returned by a model is treated as a suggestion, and running it requires a separate call unless agent mode is configured with an auto-execute list.

MCP vs Tool Calling: How the Two Relate

MCP and tool calling are not competing approaches. Tool calling is the model capability, the ability to emit a structured request to invoke a named function. MCP is a transport and discovery standard that supplies those function definitions from external servers at runtime instead of requiring them to be compiled into the application.

The final row is the trade-off that matters at scale. Direct function calling keeps the tool set small because a developer chose each one deliberately. MCP makes adding tools nearly free, and the context cost of that convenience arrives quietly. Teams evaluating the architectural boundaries here will find the distinctions laid out in MCP gateway vs MCP proxy vs MCP server.

Why Tool Calling Breaks Down at Scale

Tool calling breaks down at scale for two compounding reasons: tool definitions are re-sent on every turn of the conversation, and every intermediate result flows back through the model's context. Neither cost is visible when an agent has eight tools. Both dominate when it has five hundred.

Consider a workflow spanning five MCP servers with roughly 100 tools. In the classic flow, the run takes about six model turns, all 100 tool definitions are present in each of them, and every result the tools return is serialized into the conversation before the next step. The token bill scales with tool count multiplied by turn count, not with the work performed, a dynamic examined step by step in how tool calling executes inside AI agents.

Accuracy degrades alongside cost. In ToolScope, Liu et al. report tool selection accuracy gains of 8.38% to 38.6% from merging and context-aware filtering of the toolset, which is a measure of how much selection quality is lost when a model is asked to choose from an unfiltered catalog. A larger tool list is not a neutral addition; it makes the correct tool harder to find. The operational side of this is covered in scaling AI agents to 500 tools.

The Token Cost of Loading Every MCP Tool

The token cost of loading every MCP tool is measurable and steep. Bifrost benchmarked Code Mode against classic MCP across three rounds with increasing tool counts, running the same query set with the feature off and on, and the gap widened as the footprint grew.

Two details in that table matter more than the headline percentage. Classic MCP token usage grew roughly fourfold between 96 and 508 tools, while Code Mode usage fell over the same range, from 8.3 million to 5.4 million. And pass rates held at or above 98.5% in every round, so the reduction did not come from doing less work. At around 500 tools, average input tokens per query fell from 1.15 million to 83,000.

Anthropic's engineering team reported the same shape of result independently, describing a Google Drive to Salesforce workflow that fell from 150,000 tokens to 2,000 tokens, a 98.7% reduction, when the agent wrote code against MCP servers instead of calling their tools directly (Code execution with MCP). The full Bifrost numbers are published in the MCP gateway benchmark writeup.

Tool Filtering: Shrinking the Set Before It Reaches the Model

Tool filtering reduces the number of definitions a model sees on a given request, which lowers token cost and improves selection accuracy at the same time. Bifrost applies filtering at three levels that stack, so a tool must pass all applicable filters to be available to the model.

The default is restrictive rather than permissive. An MCP client with no tools_to_execute value exposes no tools, and a virtual key with no MCP configuration exposes none either. Tool filtering has to be granted deliberately, which is the correct default when the failure mode is an agent reaching a tool nobody meant to expose.

For fleets where the same bundle applies to many consumers, MCP tool groups define curated collections attachable across six dimensions: virtual keys, teams, customers, users, providers, and API keys. Matching runs against an in-process index, adding no request latency. The policy patterns are covered in MCP tool governance, filtering, and allowlisting.

Code Mode: Writing Code Instead of Calling 500 Tools

Code Mode replaces the full tool catalog with four meta-tools and lets the model write Python that orchestrates everything else inside a sandbox. Rather than reading 500 definitions to choose one, the model discovers what exists on demand and composes multiple calls in a single block of code.

The four meta-tools are:

• listToolFiles: discover which MCP servers are connected
• readToolFile: load Python stub signatures for a server on demand
• getToolDocs: fetch detailed documentation for one specific tool
• executeToolCode: run Python with full tool bindings in the sandbox

This changes the token math in two places. Tool definitions load only when the model asks for them, and intermediate results stay inside the sandbox instead of being serialized back into the conversation between every step. A workflow that took six model turns in the classic flow completes in three or four, with roughly 40% faster execution in large deployments.

Code Mode is worth enabling at three or more connected MCP servers, for multi-step workflows, and where tools feed into each other. With one or two small servers, classic MCP is simpler and the overhead of code generation is not repaid. The two can be mixed: heavy servers such as document stores and databases run through Code Mode while small utilities stay as direct tools. Practical results are documented in how the MCP gateway cuts token costs in Claude Code and Codex CLI and in Code Mode explained.

How an MCP Gateway Centralizes Tool Calling

An MCP gateway is a control layer that sits between agents and tool servers, centralizing connection management, authentication, filtering, and observability for every tool call. Without one, each application manages its own server connections and credentials, and no single place can answer which tools an agent actually reached.

Bifrost as an MCP gateway connects to external servers once and exposes the combined set through a single endpoint. Applications point at Bifrost rather than at individual servers, so adding or rotating a tool server is a configuration change rather than a redeploy across every consumer. Bifrost can also expose itself as an MCP server to clients such as Claude Desktop and Cursor, and it supports hosting custom tools registered directly on the gateway.

Centralization does not cost throughput. Bifrost adds 11 microseconds of overhead per request at 5,000 requests per second in published benchmarks, and clustering provides high availability with zero-downtime deployments for teams running this in production. The broader architecture is described in what an MCP gateway is and how it works.

MCP Authentication and Access Control at Scale

MCP authentication is the second problem that arrives with tool count. Every server has its own credential, and distributing those credentials to each application multiplies the number of places a secret can leak and makes rotation a coordination exercise across teams.

Bifrost handles MCP authentication at the gateway, supporting header-based auth, OAuth 2.0 with automatic token refresh and PKCE, and per-user credentials. Applications present a virtual key to Bifrost, and Bifrost presents the correct upstream credential to each tool server, so no application holds a raw MCP server secret.

Access control follows the same path. Inactive or expired virtual keys are rejected at MCP tool execution with a 403 regardless of their tool configuration, and token exchange swaps each caller's identity-provider token for a short-lived token scoped to the upstream server, per call, with no shared credential stored. For regulated deployments, Bifrost Enterprise runs these controls inside private cloud infrastructure. The credential-level detail is covered in securing agent tool access.

Scaling MCP Tool Calling: An Implementation Sequence

Scaling MCP tool calling works as an ordered sequence, because filtering decisions depend on knowing what is connected and Code Mode is worth enabling only once the footprint justifies it. The following order reflects how teams roll this out on the Bifrost gateway.

1. Centralize connections first. Point applications at one gateway rather than at individual servers, using connecting to servers to register each MCP server once.
2. Measure the tool count per request. Establish the baseline before optimizing; the token cost is a function of tool count multiplied by turn count.
3. Filter down to what each consumer needs. Set tools_to_execute per client, then narrow further per virtual key, starting from deny-by-default.
4. Enable Code Mode above three servers. Turn it on for document, database, and web servers where the definition payload is largest.
5. Move credentials to the gateway. Configure OAuth and header auth centrally so applications hold only a virtual key.
6. Add policy enforcement. Apply guardrails and per-key governance so tool access is auditable, not just configured.

Tool-level visibility is what makes the sequence verifiable rather than theoretical, and the measurement side is covered in observability and evaluation strategies for tool-calling agents and in tool calling execution efficiency.

Getting Started with MCP Tool Calling in Bifrost

MCP tool calling stops scaling when the tool catalog outgrows the context budget, and the fix is structural rather than incremental: filter what each consumer sees, and stop sending definitions the model has not asked for. The Bifrost platform combines both, with three-level tool filtering, Code Mode, centralized MCP authentication, and 11 microseconds of gateway overhead. The MCP documentation covers configuration for each.

To see MCP tool calling measured against your own servers and tool counts, book a demo with the Bifrost team.