One open source LLM gateway to securely route and govern every AI request across 1,000+ models, with full control over cost, policy, and compliance.
One gateway between your apps and every provider
Edge + Gateway
[ PERFORMANCE AT A GLANCE ]
[ OVER 1,000+ TEAMS USE BIFROST ]
[ PROVIDERS AND MODELS ]
Bifrost supports 20+ AI providers and 1,000+ models through a single unified API. Configure providers once and Bifrost handles routing, failover, and load balancing across them.
Drop-in compatible SDKs
[ GATEWAY CAPABILITIES ]
Route, cache, monitor, and govern every model call from a single layer that adds 11µs of overhead.
[ FAILOVER IN 240ms · ZERO DROPPED REQUESTS ]
[ DROP-IN REPLACEMENT ]
Bifrost exposes 100% compatible endpoints for the SDKs you already use. Point the base URL at the gateway and every feature above turns on.
Start Bifrost as a single binary with npx or Docker. No additional dependencies needed.
npx -y @maximhq/bifrost
# or
docker run -p 8080:8080 maximhq/bifrostChange the base URL to the gateway and keep your existing code. Provider keys are held by Bifrost, not your app.
# Before
client = openai.OpenAI(api_key="your-openai-key")
# After
client = openai.OpenAI(
base_url="http://localhost:8080/openai",
api_key="dummy-key", # keys are held by the gateway
)Declare a fallback and Bifrost fails over automatically when the primary provider errors out.
curl -X POST http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "openai/gpt-4o-mini",
"messages": [{"role": "user", "content": "Hello"}],
"fallbacks": ["anthropic/claude-3-5-sonnet-20241022"]
}'The same base URL works for Claude Code, Cursor, Codex, and Gemini CLI, so coding-agent traffic is routed, logged, and governed like every other AI request.
[ USE CASES ]
Fail over between providers so an OpenAI or Anthropic outage never reaches your users.
An AI agent gateway gives agent frameworks a single endpoint for models and MCP tools, with tool allow-lists per key.
Give every team its own virtual key with a budget, and attribute cost per key, model, and provider.
Point Claude Code, Cursor, Codex, or Gemini CLI at the gateway so developer AI usage is logged, budgeted, and policy-checked like everything else.
Swap models behind a stable API to A/B quality and cost without touching application code.
Cache stable prompts and paraphrases so high-volume endpoints stop paying for the same answer twice.
[ DEPLOY ANYWHERE ]
Run native or third-party guardrails inside your own network, so sensitive data never leaves your environment.
[ COMPLIANCE FRAMEWORKS ]
Immutable audit trails and exportable logs across every request, for SOC 2, HIPAA, GDPR, and ISO 27001.
[ BIFROST FEATURES ]
Everything you need to run AI in production, from free open source to enterprise-grade features.
01 Governance
SAML support for SSO and Role-based access control and policy enforcement for team collaboration.
02 Adaptive Load Balancing
Automatically optimizes traffic distribution across provider keys and models based on real-time performance metrics.
03 Cluster Mode
High availability deployment with automatic failover and load balancing. Peer-to-peer clustering where every instance is equal.
04 Alerts
Real-time notifications for budget limits, failures, and performance issues on Email, Slack, PagerDuty, Teams, Webhook and more.
05 Log Exports
Export and analyze request logs, traces, and telemetry data from Bifrost with enterprise-grade data export capabilities for compliance, monitoring, and analytics.
06 Audit Logs
Comprehensive logging and audit trails for compliance and debugging.
07 Vault Support
Secure API key management with HashiCorp Vault, AWS Secrets Manager, Google Secret Manager, and Azure Key Vault integration.
08 VPC Deployment
Deploy Bifrost within your private cloud infrastructure with VPC isolation, custom networking, and enhanced security controls.
09 Guardrails
Automatically detect and block unsafe model outputs with real-time policy enforcement and content moderation across all agents.
[ SHIP RELIABLE AI ]
Change just one line of code. Works with OpenAI, Anthropic, Vercel AI SDK, LangChain, and more.
[ FAQ ]
Common questions about running an open source LLM Gateway in production.
An LLM Gateway is a single control point between your applications and every LLM provider you use. It handles routing, retries, failover, caching, cost tracking, and policy enforcement so individual services do not have to.
To know more, read the provider configuration guide.
An API gateway routes HTTP traffic and applies generic concerns like auth and rate limiting. An LLM Gateway understands model requests. It knows about tokens, model names, provider-specific formats, streaming, cost per call, and tool calls, so it can do things a generic API gateway cannot: fail over from GPT-4o to Claude mid-outage, cache semantically similar prompts, or block a request that exceeds a team’s token budget.
An LLM Gateway governs the model calls your applications make. An MCP gateway governs the tools models can reach. Bifrost is both: it routes LLM requests across 20+ providers, and it acts as an MCP client and server so models can discover and execute external tools under tool-level allow-lists.
See the MCP overview or the MCP gateway product page.
Bifrost is open source under Apache 2.0. You can self-host the full gateway, read the source on GitHub, and run it in your own VPC or air-gapped network with no outbound calls.
For production layouts, see the enterprise deployment guides.
The open source gateway is free under Apache 2.0. Enterprise features such as adaptive load balancing, clustering, guardrails, RBAC, SSO, audit logs, and in-VPC support are licensed separately.
Yes. Bifrost is a drop-in replacement. Change the base URL to the gateway and your existing OpenAI, Anthropic, Bedrock, GenAI, LiteLLM, LangChain, or PydanticAI code keeps working.
Bifrost adds 11µs per request in sustained benchmarks at 5,000 requests per second, with a 100% success rate. Routing, fallbacks, and policy checks stay in the hot path without a meaningful latency tax.
Yes. Virtual keys carry hierarchical budgets and rate limits at the key, team, and customer level, and every request is logged with its cost, model, and provider.
For the full control plane, see AI governance.