MCP in one paragraph
MCP standardises the conversation between an AI application (the client) and a source of tools or data (the server). A server advertises tools with names, descriptions and argument schemas; the client lists them, the model picks one, the client calls it and gets a result back. Transports are STDIO for local servers and Streamable HTTP or SSE for remote ones. The protocol is published and governed openly, and most AI products and frameworks now speak it.
The success of the standard is also the problem. Once a tool is one JSON schema away from any model, the number of servers in an organisation grows quickly: hundreds of community servers exist and teams write their own. Each one is a credential, a capability and a potential blast radius.
What an MCP gateway adds
An MCP gateway is itself an MCP server to the clients and an MCP client to the upstream servers. Being in the middle lets it do five things nothing else in the stack can.
- A registry. Servers are registered once with their transport, command or endpoint and credentials. Clients get one URL instead of a configuration file per machine.
- Identity. Callers authenticate to the gateway, usually with a virtual key or an SSO token, so every tool call has a user, team or customer attached.
- Scoped tool access. Allowlists decide which tools each identity can see and call. A support agent can read tickets but not delete repositories, and the gateway enforces it rather than the prompt.
- Upstream authentication. The gateway holds the credentials for the real servers and can map a caller to a per-user upstream identity, so audit trails in the upstream system name the person, not a shared bot.
- Policy and audit. Rate limits, budgets, argument and result guardrails and an immutable log of every call, which is what security and compliance teams need before agents touch production systems.
Why not connect agents to servers directly?
For one developer on one laptop, direct connection is fine and is how MCP started. The failure modes appear at the organisation level. Credentials for the upstream systems end up in agent configuration on every machine. There is no inventory of which servers exist or who uses them. Any tool a server exposes is available to any model that can reach it, including destructive ones. And when something goes wrong there is no log that says which agent called which tool with which arguments.
These are the same problems API gateways solved for HTTP traffic twenty years ago, and the same problems AI gateways solve for model traffic. The MCP gateway applies the pattern to tool calls, which are the more dangerous half of what an agent does: a model call can produce a wrong answer, a tool call can delete the database.
Deployment patterns
Three patterns cover most deployments. A central gateway runs in the platform team’s cluster, fronts every approved server and is the only MCP endpoint developers are given; it suits regulated organisations. A per-environment gateway runs one instance per product or environment with its own registry, which keeps blast radius small at the cost of some duplication. A combined gateway runs the MCP gateway in the same process as the LLM gateway, so one identity, one set of budgets and one audit log cover both model calls and tool calls; this is the direction most vendors are converging on.
Endpoint coverage is the newest variable. Desktop AI apps, browser assistants and coding agents on employee laptops also speak MCP, and governing them means routing those clients through the gateway too, which is a device-management problem as much as a networking one. See Bifrost Edge for how we approach it.
Tool sprawl and token cost
A less obvious job of the gateway is keeping the model’s context small. Every tool a client lists costs tokens on every turn, and an agent with access to three hundred tools spends most of its context window on tool definitions. Gateways address this by scoping tool lists per identity, by grouping tools, and in some implementations by letting the model write small programs against the tool set instead of calling tools one by one, which reduces round trips and tokens. The naming differs by vendor; the problem is the same everywhere.
How to evaluate an MCP gateway
The checklist is short because the category is young. Weight the first three heavily.
- Transports and compatibility: STDIO, Streamable HTTP and SSE upstream; the current protocol version; works with the clients your people actually use.
- Identity model: how callers authenticate, whether per-user upstream credentials are supported, and whether it binds to your identity provider.
- Policy granularity: tool allowlists per user, team and customer; rate limits; argument-level guardrails.
- Audit: what is logged, where it is exported, and whether logs can be altered.
- Operations: self-hosted or managed, clustering, how servers are registered and rotated.
- Convergence: whether model calls and tool calls share identity, budgets and logs, or live in two products.
