MCP Gateway

An MCP gateway is a control layer that sits between AI agents and Model Context Protocol (MCP) servers, exposing many servers through one endpoint while it authenticates callers, decides which tools each caller may use, routes tool calls, and logs every invocation. An MCP gateway does for tool calls what an API gateway does for HTTP APIs, but it reads MCP's JSON-RPC messages and enforces policy per tool.

The Model Context Protocol ecosystem passed 10,000 active public MCP servers by December 2025. Each server brings its own credentials, tool list and logs, so wiring every agent directly to every server stops scaling quickly.

Bifrost, an open-source AI gateway, runs as an MCP gateway in front of those servers, in the same process that routes LLM traffic.

How does an MCP gateway work?

An MCP gateway works as an MCP server toward its clients and as an MCP client toward the servers behind it. An agent connects to one gateway endpoint; the gateway authenticates the caller, returns only the tools that caller may use, routes each tools/call to the owning server, and records the result.

Every call follows the same path:

1. 01

   Authenticate

   The client sends a JSON-RPC 2.0 request over Streamable HTTP with a credential; the gateway resolves it to a user, team or service.

2. 02

   Discover

   tools/list returns the merged catalog of every upstream server, filtered to what that identity may see.

3. 03

   Route

   tools/call is checked against policy again and sent to the owning server with the gateway's own credential for it.

4. 04

   Inspect and log

   The gateway scans arguments and results, then records caller, tool, server, outcome and latency.

The current specification revision, 2026-07-28, made MCP stateless: no initialize handshake, no session ID, and every request carries its protocol version and capabilities in _meta. Any gateway replica can therefore serve any request, while servers on earlier, session-based revisions still need a sticky upstream connection. The Model Context Protocol overview explains how hosts, clients and servers relate.

How an MCP gateway handles tool discovery and routing

An MCP gateway handles tool discovery by connecting to each upstream MCP server, calling tools/list, and merging the results into one catalog. It handles routing by mapping every tool name in that catalog back to the server that owns it, so clients see one tool list and the gateway decides where each call goes.

Three problems appear once several servers sit behind one gateway:

• Name collisions. Two servers can both expose search, so gateways usually prefix the server name: github-search and jira-search.
• Context cost. Every tool definition is sent to the model on every turn, so 300 tools add tokens, latency and wrong tool choices. Per-caller filtering, curated bundles and code-execution patterns keep the list short.
• Upstream health. Gateways health-check upstream connections and reconnect with exponential backoff, so one failing server does not take the catalog down.

A registry of approved servers, with their transport, credentials and owner, decides what the gateway may reach at all. A gateway can also bridge transports, spawning a local stdio (standard input and output) server as a subprocess and exposing its tools over HTTP. Bifrost connects to MCP servers over STDIO, HTTP or SSE.

MCP gateway authentication: OAuth, OIDC and JWT

MCP gateway authentication happens on two separate hops. Inbound, the agent or user proves its identity to the gateway, usually with an OAuth 2.1 access token or an API key. Outbound, the gateway presents its own credential, or a per-user one, to each MCP server. The specification forbids reusing the inbound token on the outbound hop.

OAuth2 (OAuth 2.0, and the OAuth 2.1 profile MCP builds on) issues scoped, expiring access tokens instead of shared passwords. OIDC (OpenID Connect) is the identity layer on OAuth 2.0 that tells the gateway who the user is, typically through single sign-on with Okta or Microsoft Entra ID, so each person maps to a tool policy. A JWT (JSON Web Token) is a signed token with claims such as sub, aud and exp; the gateway can verify it locally, with no network round trip per call.

Access control, rate limiting and data protection

After authentication, the gateway enforces what each identity may do: which servers and tools it can see, how often it can call them, and what data can pass through tool arguments and results. Individual MCP servers rarely implement these controls, and never consistently.

• Tool-level access control. Role-based access control (RBAC) maps roles, teams or keys to allowed tools, deny by default. Filter tools/list and re-check tools/call, because a client can request a tool it was never shown.
• Rate limiting. Rate limiting caps calls per identity and per tool over a time window. Downstream APIs limit per credential, so when twenty agents share one service token, one looping agent exhausts the quota for all of them.
• Data protection. Arguments can carry secrets or personal data, and results become model input that can carry a prompt injection, so gateways run guardrails on tool calls in both directions.
• Human approval. Destructive tools, such as deleting records, can require an explicit approval step.

MCP gateway observability and audit logs

MCP gateway observability means recording every tool invocation in one place: who called it, which tool and server, the arguments, the result or error, and the latency. Without a gateway those records sit in each server's own logs, if any, and cannot be joined to the model call that triggered them.

A useful log entry also carries the ID of the LLM request that produced the call. Teams then track p50 and p99 latency per tool, error rate per server and calls per identity, and export OpenTelemetry traces so one agent run shows every model and tool call.

Keep two records apart: tool-call logs cover data-plane traffic; audit logs cover control-plane changes, such as who registered a server. Compliance reviews need both, exported to a SIEM (security information and event management system). The requirements for MCP audit logging set out what each record must capture.

MCP gateway vs MCP server vs API gateway

An MCP server executes actions against one system; an MCP gateway governs access to many MCP servers; an API gateway governs HTTP traffic to many APIs. The two gateways play similar roles, but an API gateway routes by URL path and method, while an MCP gateway routes by tool name.

An MCP proxy forwards MCP traffic or bridges transports without identity, policy or logging.

Deploying an MCP gateway: open source, Docker and Kubernetes

Deploying an MCP gateway is mainly a choice of where it runs and how it scales. Most teams run a self-hosted gateway as a container next to their agents, with Docker on one node and Kubernetes in production, close to the MCP servers it calls, because every tool call pays the gateway's latency.

1. 01

   Self-hosted or managed

   An open source MCP gateway you host keeps upstream tokens and tool data inside your network; a managed one sends tool traffic through a third party.

2. 02

   Replicas

   Run stateless replicas behind a load balancer; legacy session-based servers and SSE streams still need sticky routing.

3. 03

   Stdio servers in Docker

   Spawned stdio servers need their runtimes, such as Node.js or Python, inside the gateway's image.

4. 04

   Kubernetes controls

   Use readiness probes, an autoscaler, secrets from a secret manager, and network policies that limit egress to registered MCP servers.

5. 05

   Latency budget

   Measure gateway overhead at p99 against tool calls that often take hundreds of milliseconds.

6. 06

   Laptops

   MCP servers configured in desktop apps never reach a central gateway, so endpoint inventory must cover them.

MCP gateway with Bifrost

Bifrost, the open-source AI gateway built by Maxim AI, runs as an MCP gateway and an LLM gateway in one process. As a client it connects to upstream servers over STDIO, HTTP or SSE; as an MCP server it exposes their tools at one /mcp endpoint that Claude Desktop, Cursor and custom agents connect to.

Inbound, gateway authentication accepts virtual keys in headers, OAuth 2.1 with Bifrost as the authorization server (Dynamic Client Registration, PKCE, and JWT access tokens bound to /mcp), or both. Outbound, each server uses one of six authentication types: none, headers, OAuth 2.0, per-user OAuth, per-user headers and token exchange, so upstream servers see the real user instead of a shared account.

Tool filtering stacks three levels: the client's tools_to_execute list (empty means no tools), request headers, and the virtual key. tools/list and tools/call check the same allow-list, and an inactive or expired key gets 403. Virtual MCPs bundle approved tools from several servers behind one /mcp/ slug endpoint. By default Bifrost does not execute tool calls the model returns; the application approves each one.

Tool executions appear in the built-in logs beside LLM requests. Code Mode replaces direct tool definitions with four meta-tools and runs model-written Python (Starlark) in a sandbox; with 508 tools across 16 servers it cut input tokens by 92.8% at the same 100% pass rate.

The same gateway routes LLM traffic to 25+ providers and 10,000+ models with 11 microseconds of overhead at 5,000 RPS.

For laptops, AI Gateway + Bifrost Edge (in alpha) inventories the MCP servers configured in desktop apps and enforces allow or deny decisions on each device.

To put your first MCP servers behind one endpoint, start with the gateway quickstart or see Bifrost as an MCP gateway.

FAQ