​
Multi-Provider Setup

Configure multiple providers to seamlessly switch between them. This example shows how to configure OpenAI, Anthropic, and Mistral providers.
Provider Configuration Interface
  1. Go to http://localhost:8080
  2. Navigate to “Providers” in the sidebar
  3. Click “Add Provider”
  4. Select provider and configure keys
  5. Save configuration

​
Making Requests

Once providers are configured, you can make requests to any specific provider. This example shows how to send a request directly to OpenAI’s GPT-4o Mini model. Bifrost handles the provider-specific API formatting automatically. 
curl --location 'http://localhost:8080/v1/chat/completions' \
--header 'Content-Type: application/json' \
--data '{
    "model": "openai/gpt-4o-mini",
    "messages": [
        {"role": "user", "content": "Hello!"}
    ]
}'

​
Environment Variables

Set up your API keys for the providers you want to use. Bifrost supports both direct key values and environment variable references with the env. prefix: 
export OPENAI_API_KEY="your-openai-api-key"
export ANTHROPIC_API_KEY="your-anthropic-api-key"
export MISTRAL_API_KEY="your-mistral-api-key"
export GROQ_API_KEY="your-groq-api-key"
export COHERE_API_KEY="your-cohere-api-key"
Environment Variable Handling:
  • Use "value": "env.VARIABLE_NAME" to reference environment variables
  • Use "value": "sk-proj-xxxxxxxxx" to pass keys directly
  • All sensitive data is automatically redacted in GET requests and UI responses for security

​
Advanced Configuration

​
Weighted Load Balancing

Distribute requests across multiple API keys or providers based on custom weights. This example shows how to split traffic 70/30 between two OpenAI keys, useful for managing rate limits or costs across different accounts.
Weighted Load Balancing Interface
  1. Navigate to “Providers” → “OpenAI”
  2. Click “Add Key” to add multiple keys
  3. Set weight values (0.7 and 0.3)
  4. Save configuration

​
Model-Specific Keys

Use different API keys for specific models, allowing you to manage access controls and billing separately. This example uses a premium key for advanced reasoning models (o1-preview, o1-mini) and a standard key for regular GPT models.
Model-Specific Keys Interface
  1. Navigate to “Providers” → “OpenAI”
  2. Add first key with models: ["gpt-4o", "gpt-4o-mini"]
  3. Add premium key with models: ["o1-preview", "o1-mini"]
  4. Save configuration

​
Custom Network Settings

Customize the network configuration for each provider, including custom base URLs, extra headers, and timeout settings. This example shows how to use a local OpenAI-compatible server with custom headers for user identification.
Network Configuration Interface
  1. Navigate to “Providers” → “OpenAI” → “Advanced”
  2. Set Base URL: http://localhost:8000/v1
  3. Set Timeout: 30 seconds
  4. Save configuration

​
Managing Retries

Configure retry behavior for handling temporary failures and rate limits. This example sets up exponential backoff with up to 5 retries, starting with 1ms delay and capping at 10 seconds - ideal for handling transient network issues.
Retry Configuration Interface
  1. Navigate to “Providers” → “OpenAI” → “Advanced”
  2. Set Max Retries: 5
  3. Set Initial Backoff: 1 ms
  4. Set Max Backoff: 10000 ms
  5. Save configuration

​
Custom Concurrency and Buffer Size

Fine-tune performance by adjusting worker concurrency and queue sizes per provider. This example gives OpenAI higher limits (100 workers, 500 queue) for high throughput, while Anthropic gets conservative limits to respect their rate limits.
Concurrency Configuration Interface
  1. Navigate to “Providers” → Provider → “Performance”
  2. Set Concurrency: Worker count (100 for OpenAI, 25 for Anthropic)
  3. Set Buffer Size: Queue size (500 for OpenAI, 100 for Anthropic)
  4. Save configuration

​
Setting Up a Proxy

Route requests through proxies for compliance, security, or geographic requirements. This example shows both HTTP proxy for OpenAI and authenticated SOCKS5 proxy for Anthropic, useful for corporate environments or regional access.
Proxy Configuration Interface
  1. Navigate to “Providers” → Provider → “Proxy”
  2. Select Proxy Type: HTTP or SOCKS5
  3. Set Proxy URL: http://localhost:8000
  4. Add credentials if needed (username/password)
  5. Save configuration

​
Send Back Raw Response

Include the original provider response alongside Bifrost’s standardized response format. Useful for debugging and accessing provider-specific metadata.
Raw Response Configuration Interface
  1. Navigate to “Providers” → Provider → “Advanced”
  2. Toggle “Include Raw Response” to enabled
  3. Save configuration
When enabled, the raw provider response appears in extra_fields.raw_response: 
{
    "choices": [...],
    "usage": {...},
    "extra_fields": {
        "provider": "openai",
        "raw_response": {
            // Original OpenAI response here
        }
    }
}

​
Provider-Specific Authentication

Enterprise cloud providers require additional configuration beyond API keys. Configure Azure OpenAI, AWS Bedrock, and Google Vertex with platform-specific authentication details.

​
Azure OpenAI

Azure OpenAI requires endpoint URLs, deployment mappings, and API version configuration:
Azure OpenAI Configuration Interface
  1. Navigate to “Providers” → “Azure OpenAI”
  2. Set API Key: Your Azure API key
  3. Set Endpoint: Your Azure endpoint URL
  4. Configure Deployments: Map model names to deployment names
  5. Set API Version: e.g., 2024-08-01-preview
  6. Save configuration

​
AWS Bedrock

AWS Bedrock supports both explicit credentials and IAM role authentication:
AWS Bedrock Configuration Interface
  1. Navigate to “Providers” → “AWS Bedrock”
  2. Set Access Key: AWS Access Key ID (or leave empty for IAM)
  3. Set Secret Key: AWS Secret Access Key (or leave empty for IAM)
  4. Set Region: e.g., us-east-1
  5. Configure Deployments: Map model names to inference profiles
  6. Set ARN: Required for deployments mapping
  7. Save configuration
Notes:
  • If both access_key and secret_key are empty, Bifrost uses IAM role authentication from environment
  • arn is required for URL formation - deployments mapping is ignored without it
  • When using arn + deployments, Bifrost uses model profiles; otherwise forms path with incoming model name directly

​
Google Vertex

Google Vertex requires project configuration and authentication credentials:
Google Vertex Configuration Interface
  1. Navigate to “Providers” → “Google Vertex”
  2. Set API Key: Your Vertex API key
  3. Set Project ID: Your Google Cloud project ID
  4. Set Region: e.g., us-central1
  5. Set Auth Credentials: Service account credentials JSON
  6. Save configuration

​
Next Steps

Now that you understand provider configuration, explore these related topics:

​
Essential Topics

​
Advanced Topics