---
title: Multi-Provider Routing and Custom Providers in Bifrost
description: "Configure multi-provider routing and custom providers in Bifrost with governance rules, weighted distribution, and request-type controls for production AI.\n\nProduction AI applications routinely encounter provider-side issues: regional outages, rate-limit rejections at peak load, model deprecations, and unpredictable tail latency. Multi-provider routing addresses these problems by"
image: https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w1200/2026/07/multi-provider-routing-and-custom-providers-in-bifrost-bifrost-paper-waves.optimized.png
---

Try Bifrost Enterprise free for 14 days. [Request access](https://www.getmaxim.ai/#enterprise-trial)

*Configure multi-provider routing and custom providers in Bifrost with governance rules, weighted distribution, and request-type controls for production AI.*

Production AI applications routinely encounter provider-side issues: regional outages, rate-limit rejections at peak load, model deprecations, and unpredictable tail latency. Multi-provider routing addresses these problems by directing a request for `gpt-4o` to OpenAI, Azure OpenAI, or any other configured provider based on real-time conditions. [Bifrost](https://www.getmaxim.ai) is an open-source [AI gateway](https://www.getmaxim.ai/articles/top-5-llm-gateways-in-2026-a-production-ready-comparison/) designed for this pattern, with multi-provider routing that operates across three coordinated layers and supports custom provider definitions for self-hosted models, OpenAI-compatible endpoints, and environment-scoped configurations. The source code is available on [Bifrost's GitHub](https://github.com/maximhq/bifrost), and the [documentation](https://docs.getbifrost.ai/overview) covers a full setup in under five minutes.

## Why Multi-Provider Routing Matters for Production AI

Single-provider AI architectures fail in predictable ways. A regional outage at one provider takes the application down. A spike in usage exhausts the API key's rate limits. A new model release at a competing provider creates pressure to migrate, which requires SDK changes scattered across the codebase. [Industry analyses of multi-provider LLM strategies](https://dev.to/ash_dubai/multi-provider-llm-orchestration-in-production-a-2026-guide-1g10) point to the same set of pressures: cost variance, reliability gaps, and the operational cost of model-specific code paths.

Multi-provider routing solves the problem at the infrastructure layer instead of the application layer. Instead of every service maintaining its own retry logic, provider SDKs, and fallback rules, the gateway centralizes those decisions. A single configuration update changes how the entire application talks to its model providers, and engineering teams stop rewriting failover logic in every new service.

Bifrost provides this capability with three routing methods that can be used independently or combined: governance-based routing, dynamic routing rules, and adaptive load balancing. All three operate on top of a unified [model catalog](https://docs.getbifrost.ai/providers/provider-routing) that tracks every model available across [20+ supported providers](https://docs.getbifrost.ai/providers/supported-providers/overview).

## How Bifrost's Multi-Provider Routing Works

[Bifrost](https://www.getmaxim.ai) processes routing decisions in a defined order. Routing rules evaluate first and can override everything else. Governance rules apply next, using weighted distributions defined per virtual key. Load balancing then optimizes provider and key selection based on live performance metrics. The three layers run in sequence on every request.

The system is built around two core entities:

- **Virtual keys**: governance entities that authenticate consumers and define which providers, models, and budgets they can access.
- **Provider configs**: per-virtual-key definitions that map each consumer to one or more upstream providers, with weights, allowed models, and optional budget or rate-limit constraints.

Provider configs are where multi-provider routing is actually expressed. Each entry tells the gateway which provider to consider, which models that provider may serve for this consumer, and how much weight to assign in the selection process. The result is precise control over which traffic flows to which provider, without modifying any application code.

## Configuring Governance-Based Routing with Virtual Keys

[Governance routing](https://docs.getbifrost.ai/features/governance/routing) is the most explicit of the three methods. It works through provider configs attached to a [virtual key](https://docs.getbifrost.ai/features/governance/virtual-keys), and is the right choice when an organization needs deterministic control over how traffic is distributed.

A minimal two-provider configuration looks like this:

```json
{
  "provider_configs": [
    {
      "provider": "openai",
      "allowed_models": ["gpt-4o", "gpt-4o-mini"],
      "weight": 0.3,
      "budget": {
        "max_limit": 100.0,
        "current_usage": 45.0
      }
    },
    {
      "provider": "azure",
      "allowed_models": ["gpt-4o"],
      "weight": 0.7,
      "rate_limit": {
        "token_max_limit": 100000,
        "token_reset_duration": "1m"
      }
    }
  ]
}
```

When a request arrives with this virtual key, Bifrost runs the following sequence:

1. Validate that the requested model is allowed for at least one configured provider.
2. Filter out providers that have exceeded their budget or rate limit.
3. Apply weighted random selection across the remaining providers. With weights of 0.3 and 0.7, 70% of eligible requests go to Azure, 30% to OpenAI.
4. Transform the model identifier to a `provider/model` form, for example `azure/gpt-4o`.
5. Generate a fallback chain from the remaining providers, sorted by weight in descending order.

The `allowed_models` field controls what each provider can serve. Setting it to `["*"]` permits every model the provider supports, validated against the model catalog. Setting it to an explicit list restricts the provider to those exact models. Setting it to `[]` denies all models for that provider. This last behavior is intentional and signals a deny-by-default posture, which matters for [enterprise governance](https://www.getmaxim.ai/resources/governance) where unconfigured providers should not silently serve traffic.

## Adding Routing Rules for Dynamic Conditions

Static weights cover most cases, but production AI often needs decisions based on runtime context: the user's tier, the requesting team, current budget consumption, or a specific header. [Routing rules](https://docs.getbifrost.ai/providers/routing-rules) handle this through [Common Expression Language (CEL)](https://github.com/google/cel-spec) expressions, evaluated before governance rules apply.

Example rule expressions Bifrost supports:

- `headers["x-tier"] == "premium"` routes premium users to a specific provider and model.
- `budget_used > 85` redirects traffic to a cheaper provider when monthly spend approaches the cap.
- `team_name == "ml-research"` routes research workloads to a different model than production workloads.
- `headers["x-environment"] == "production" && tokens_used < 75` combines multiple conditions for sophisticated routing.

Rules are evaluated in scope precedence: virtual key, then team, then customer, then global. Within a scope, lower priority numbers evaluate first. The first matching rule determines the routing decision, and the remaining rules are skipped. When no rule matches, governance routing takes over.

This separation matters because routing rules are not a replacement for governance, they are a layer that sits on top of it. The same virtual key can have static provider weights and a few CEL-based overrides for special cases, with no conflict between the two.

## Defining Custom Providers in Bifrost

[Custom providers](https://docs.getbifrost.ai/providers/custom-providers) extend [Bifrost](https://www.getmaxim.ai) beyond its 20+ built-in providers. The use cases generally fall into three categories: connecting to an OpenAI-compatible endpoint that is not a built-in provider, creating multiple scoped instances of the same base provider, or routing different request types to different underlying endpoints.

A custom provider is configured through the `custom_provider_config` field. It declares a base provider type (which determines the API contract Bifrost uses), an optional list of allowed request types, and optional path overrides for individual endpoints.

Example: an OpenAI-compatible internal endpoint that should serve chat completions only.

```json
{
  "providers": {
    "internal-llm": {
      "keys": [
        {
          "name": "internal-llm-key-1",
          "value": "env.INTERNAL_API_KEY",
          "models": ["*"],
          "weight": 1.0
        }
      ],
      "network_config": {
        "base_url": "<https://internal-llm.example.com>"
      },
      "custom_provider_config": {
        "base_provider_type": "openai",
        "allowed_requests": {
          "chat_completion": true,
          "chat_completion_stream": true
        },
        "request_path_overrides": {
          "chat_completion": "/api/v2/chat",
          "chat_completion_stream": "/api/v2/chat"
        }
      }
    }
  }
}
```

The `allowed_requests` block is more than documentation. When set, only the request types marked `true` are permitted; every other operation returns an access-control error. This makes it possible to expose an embeddings-only instance of OpenAI for an analytics team, while a separate chat-only instance serves a customer-facing application. Both share the same upstream account, but each can only perform the operations its scope allows.

Bifrost supports custom providers built on these base types:

- `openai` (and any OpenAI-compatible endpoint)
- `anthropic`
- `bedrock` (AWS Bedrock)
- `cohere`
- `gemini`
- `replicate`

For air-gapped or self-hosted deployments, custom providers also accept TLS configuration: either a `ca_cert_pem` to trust a private CA, or `insecure_skip_verify` for trusted internal environments. These options enable the gateway to operate against internal endpoints that do not use publicly trusted certificates, which matters for [in-VPC and on-prem deployments](https://www.getmaxim.ai/enterprise) common in regulated industries.

## Common Configuration Patterns

A few patterns recur across Bifrost deployments:

- **Environment separation**: define `openai-dev`, `openai-staging`, and `openai-prod` as custom providers, each with a different `allowed_requests` scope. Issue dev-only virtual keys that can only access `openai-dev`, and the access boundary becomes structural rather than convention.
- **Cost-aware fallback**: in `provider_configs`, set the primary provider with weight 0.9 and a cheaper secondary provider with weight 0.1. Combine with a routing rule that pushes 100% to the cheaper provider when `budget_used > 85`, and the gateway shifts cost behavior automatically as spend climbs.
- **Cross-provider failover**: configure two providers that both serve the same model (for example, `openai` and `azure` both serving `gpt-4o`). When the primary fails, Bifrost's [automatic fallback chain](https://docs.getbifrost.ai/features/fallbacks) retries on the next provider with no application-side code changes.
- **Regional routing**: for data residency, define separate custom providers per region with TLS pinned to internal certificates. Use a routing rule like `headers["x-region"] == "eu"` to direct EU traffic to EU-hosted models, satisfying GDPR data-locality requirements without application logic.
- **Tool-scoped MCP traffic**: pair custom providers with [Bifrost's MCP gateway](https://www.getmaxim.ai/resources/mcp-gateway) to enforce different model and tool combinations per consumer.

Each of these patterns is configuration-only. No application code changes, no SDK swaps, no per-service retry logic. The gateway becomes the single point where multi-provider strategy is expressed and enforced.

## Operating Bifrost in Production

Multi-provider routing is a runtime decision, which means visibility into routing behavior matters as much as the configuration itself. Bifrost exposes the selected provider, the selected key, and any applied routing rule in the dashboard and via Prometheus metrics. The same data flows into traces conforming to the [OpenTelemetry specification](https://opentelemetry.io/docs/specs/otel/), so teams running distributed tracing can correlate routing decisions with downstream latency and error rates.

For enterprises evaluating Bifrost's fit, the [LLM Gateway Buyer's Guide](https://www.getmaxim.ai/resources/buyers-guide) provides a structured capability matrix covering routing, governance, observability, and deployment options. The [governance resource hub](https://www.getmaxim.ai/resources/governance) covers virtual key design patterns and cost-attribution strategies in detail.

## Get Started with Bifrost Multi-Provider Routing

Multi-provider routing turns LLM infrastructure from a single point of failure into a resilient, controllable layer. Bifrost's combination of governance routing, dynamic routing rules, and custom providers gives platform teams the configuration surface they need without requiring application-side changes. To see how Bifrost can simplify your multi-provider AI infrastructure, [book a demo](https://www.getmaxim.ai/book-a-demo) with the Bifrost team.

## Read next

[![API Key Management for LLM Apps: How to Manage Client Access](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w720/2026/10/how-to-enforce-ai-budgets-per-team-project-and-key-bifrost-1920x1080--1-.png) API key management for LLM applications means issuing every app, service, and environment its own revocable credential instead of sharing raw provider keys. This guide shows how to do it with Bifrost virtual keys, covering scoping, rate limits, rotation, revocation, and log attribution.](https://www.getmaxim.ai/articles/api-key-management-for-llm-applications/)

[![Claude Agent SDK Through a Gateway: Routing, Budgets, MCP](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w720/2026/10/claude-agent-sdk-through-a-gateway-routing-budgets-mcp-bifrost-1920x1080.png) The Claude Agent SDK packages Claude Code's agent loop, tools, and context management as a Python and TypeScript library. This guide shows how to route Claude Agent SDK traffic through Bifrost for provider routing and failover, per-tenant budgets, and governed MCP tool access.](https://www.getmaxim.ai/articles/claude-agent-sdk-through-a-gateway-routing-budgets-mcp/)

[![How to Enforce AI Budgets Per Team, Project and Key](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w720/2026/10/how-to-enforce-ai-budgets-per-team-project-and-key-bifrost-1920x1080.png) An AI budget is a hard spending ceiling enforced in the LLM request path, not a monthly report. This guide covers budget hierarchies, project mapping, reset windows, rate limits, exhausted-budget behavior, and a worked $10,000 example in Bifrost.](https://www.getmaxim.ai/articles/enforce-ai-budgets/)

## More in AI Gateway

[![Implementing LLM Guardrails with the Bifrost Enterprise AI Gateway](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w720/2026/07/implementing-llm-guardrails-with-the-bifrost-enterprise-ai-g-bifrost-paper-waves.optimized.png) Implement LLM guardrails at the gateway layer with Bifrost: content safety, PII detection, prompt injection blocking, and MCP tool governance in one platform. LLM guardrails are the runtime controls that validate every prompt and response flowing through an AI application, blocking harmful content, redacting sensitive data, and enforcing policy before](https://www.getmaxim.ai/articles/implementing-llm-guardrails-with-the-bifrost-enterprise-ai-gateway/)

[![Best LLM Gateways for Healthcare, Financial Services, and Government in 2026](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w720/2026/07/best-llm-gateways-for-healthcare-financial-services-and-gove-bifrost-paper-waves.optimized.png) The best LLM gateway for a regulated industry routes, logs, and redacts AI traffic inside your own network boundary. This guide ranks Bifrost, LiteLLM, Kong AI Gateway, F5 AI Gateway, and Cloudflare AI Gateway, and maps HIPAA safeguards to gateway controls.](https://www.getmaxim.ai/articles/best-llm-gateways-for-healthcare-financial-services-and-government-in-2026/)

```json
{
    "@context": "https://schema.org",
    "@type": "Article",
    "publisher": {
        "@type": "Organization",
        "name": "Maxim Articles",
        "url": "https://www.getmaxim.ai/articles/",
        "logo": {
            "@type": "ImageObject",
            "url": "https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w256h256/2025/08/thumbnail.png",
            "width": 60,
            "height": 60
        }
    },
    "author": {
        "@type": "Person",
        "name": "Kuldeep Paul",
        "image": {
            "@type": "ImageObject",
            "url": "https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/2025/08/1727978381919.jpeg",
            "width": 800,
            "height": 800
        },
        "url": "https://www.getmaxim.ai/articles/author/kuldeep/",
        "sameAs": []
    },
    "headline": "Multi-Provider Routing and Custom Providers in Bifrost",
    "url": "https://www.getmaxim.ai/articles/multi-provider-routing-and-custom-providers-in-bifrost/",
    "datePublished": "2026-05-22T04:33:00.000Z",
    "dateModified": "2026-10-13T11:02:48.000Z",
    "image": {
        "@type": "ImageObject",
        "url": "https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w1200/2026/07/multi-provider-routing-and-custom-providers-in-bifrost-bifrost-paper-waves.optimized.png",
        "width": 1200,
        "height": 630
    },
    "keywords": "AI Gateway",
    "description": "Configure multi-provider routing and custom providers in Bifrost with governance rules, weighted distribution, and request-type controls for production AI.\n\nProduction AI applications routinely encounter provider-side issues: regional outages, rate-limit rejections at peak load, model deprecations, and unpredictable tail latency. Multi-provider routing addresses these problems by directing a request for gpt-4o to OpenAI, Azure OpenAI, or any other configured provider based on real-time condition",
    "mainEntityOfPage": "https://www.getmaxim.ai/articles/multi-provider-routing-and-custom-providers-in-bifrost/"
}
```