Configuration Overview (File Based)
You can directly use the UI (http://localhost:{port}/providers) to configure the providers.
Provider configuration in config.json defines:
- API credentials and key management
- Supported models for each provider
- Network settings and retry behavior
- Concurrency controls and performance tuning
- Provider-specific metadata (regions, endpoints, etc.)
{
"providers": {
"openai": {
/* provider config */
},
"anthropic": {
/* provider config */
},
"bedrock": {
/* provider config */
}
}
}
Basic Provider Setup
OpenAI
{
"providers": {
"openai": {
"keys": [
{
"value": "env.OPENAI_API_KEY",
"models": [
"gpt-3.5-turbo",
"gpt-4",
"gpt-4o",
"gpt-4o-mini",
"gpt-4-turbo",
"gpt-4-vision-preview"
],
"weight": 1.0
}
],
"network_config": {
"default_request_timeout_in_seconds": 30,
"max_retries": 1,
"retry_backoff_initial_ms": 100,
"retry_backoff_max_ms": 2000
},
"concurrency_and_buffer_size": {
"concurrency": 3,
"buffer_size": 10
}
}
}
}
Anthropic
{
"providers": {
"anthropic": {
"keys": [
{
"value": "env.ANTHROPIC_API_KEY",
"models": [
"claude-2.1",
"claude-3-sonnet-20240229",
"claude-3-haiku-20240307",
"claude-3-opus-20240229",
"claude-3-5-sonnet-20240620"
],
"weight": 1.0
}
],
"network_config": {
"default_request_timeout_in_seconds": 30,
"max_retries": 1,
"retry_backoff_initial_ms": 100,
"retry_backoff_max_ms": 2000
},
"concurrency_and_buffer_size": {
"concurrency": 3,
"buffer_size": 10
}
}
}
}
Advanced Provider Configuration
AWS Bedrock
{
"providers": {
"bedrock": {
"keys": [
{
"value": "env.BEDROCK_API_KEY",
"models": [
"anthropic.claude-v2:1",
"mistral.mixtral-8x7b-instruct-v0:1",
"mistral.mistral-large-2402-v1:0",
"anthropic.claude-3-sonnet-20240229-v1:0"
],
"weight": 1.0
}
],
"network_config": {
"default_request_timeout_in_seconds": 30,
"max_retries": 1,
"retry_backoff_initial_ms": 100,
"retry_backoff_max_ms": 2000
},
"meta_config": {
"secret_access_key": "env.AWS_SECRET_ACCESS_KEY",
"region": "us-east-1"
},
"concurrency_and_buffer_size": {
"concurrency": 3,
"buffer_size": 10
}
}
}
}
Azure OpenAI
{
"providers": {
"azure": {
"keys": [
{
"value": "env.AZURE_API_KEY",
"models": ["gpt-4o"],
"weight": 1.0
}
],
"network_config": {
"default_request_timeout_in_seconds": 30,
"max_retries": 1,
"retry_backoff_initial_ms": 100,
"retry_backoff_max_ms": 2000
},
"meta_config": {
"endpoint": "env.AZURE_ENDPOINT",
"deployments": {
"gpt-4o": "gpt-4o-aug"
},
"api_version": "2024-08-01-preview"
},
"concurrency_and_buffer_size": {
"concurrency": 3,
"buffer_size": 10
}
}
}
}
Google Vertex AI
{
"providers": {
"vertex": {
"keys": [],
"meta_config": {
"project_id": "env.VERTEX_PROJECT_ID",
"region": "us-central1",
"auth_credentials": "env.VERTEX_CREDENTIALS"
},
"concurrency_and_buffer_size": {
"concurrency": 3,
"buffer_size": 10
}
}
}
}
Key Management
Multiple API Keys
Balance load across multiple keys:{
"providers": {
"openai": {
"keys": [
{
"value": "env.OPENAI_API_KEY_1",
"models": ["gpt-4o-mini", "gpt-4o"],
"weight": 0.7
},
{
"value": "env.OPENAI_API_KEY_2",
"models": ["gpt-4o-mini"],
"weight": 0.3
}
]
}
}
}
Model-Specific Keys
Different keys for different models:{
"providers": {
"openai": {
"keys": [
{
"value": "env.OPENAI_API_KEY_BASIC",
"models": ["gpt-3.5-turbo", "gpt-4o-mini"],
"weight": 1.0
},
{
"value": "env.OPENAI_API_KEY_PREMIUM",
"models": ["gpt-4o", "gpt-4-turbo"],
"weight": 1.0
}
]
}
}
}
Network Configuration
Custom Headers and Timeouts
{
"providers": {
"openai": {
"keys": [
/* ... */
],
"network_config": {
"extra_headers": {
"X-Organization-ID": "org-123",
"X-Environment": "production"
},
"default_request_timeout_in_seconds": 60,
"max_retries": 3,
"retry_backoff_initial_ms": 200,
"retry_backoff_max_ms": 5000
}
}
}
}
Proxy Configuration
{
"providers": {
"openai": {
"keys": [
/* ... */
],
"network_config": {
"proxy_url": "http://proxy.company.com:8080",
"proxy_auth": {
"username": "env.PROXY_USER",
"password": "env.PROXY_PASS"
}
}
}
}
}
Performance Tuning
Concurrency Controls
{
"providers": {
"openai": {
"keys": [
/* ... */
],
"concurrency_and_buffer_size": {
"concurrency": 10, // Number of concurrent requests
"buffer_size": 50 // Request queue buffer size
}
},
"anthropic": {
"keys": [
/* ... */
],
"concurrency_and_buffer_size": {
"concurrency": 5, // Lower concurrency for rate limits
"buffer_size": 20
}
}
}
}
High-Volume Configuration
For production workloads:{
"providers": {
"openai": {
"keys": [
{
"value": "env.OPENAI_API_KEY_1",
"models": ["gpt-4o-mini"],
"weight": 0.5
},
{
"value": "env.OPENAI_API_KEY_2",
"models": ["gpt-4o-mini"],
"weight": 0.5
}
],
"network_config": {
"default_request_timeout_in_seconds": 45,
"max_retries": 2,
"retry_backoff_initial_ms": 150,
"retry_backoff_max_ms": 3000
},
"concurrency_and_buffer_size": {
"concurrency": 20,
"buffer_size": 100
}
}
}
}
Multi-Provider Setup
Production Configuration
{
"providers": {
"openai": {
"keys": [
{
"value": "env.OPENAI_API_KEY",
"models": ["gpt-4o-mini", "gpt-4o", "gpt-4-turbo"],
"weight": 1.0
}
],
"concurrency_and_buffer_size": {
"concurrency": 15,
"buffer_size": 75
}
},
"anthropic": {
"keys": [
{
"value": "env.ANTHROPIC_API_KEY",
"models": ["claude-3-sonnet-20240229", "claude-3-haiku-20240307"],
"weight": 1.0
}
],
"concurrency_and_buffer_size": {
"concurrency": 10,
"buffer_size": 50
}
},
"bedrock": {
"keys": [
{
"value": "env.BEDROCK_API_KEY",
"models": ["anthropic.claude-3-sonnet-20240229-v1:0"],
"weight": 1.0
}
],
"meta_config": {
"secret_access_key": "env.AWS_SECRET_ACCESS_KEY",
"region": "us-east-1"
},
"concurrency_and_buffer_size": {
"concurrency": 8,
"buffer_size": 40
}
},
"cohere": {
"keys": [
{
"value": "env.COHERE_API_KEY",
"models": ["command-a-03-2025"],
"weight": 1.0
}
],
"concurrency_and_buffer_size": {
"concurrency": 5,
"buffer_size": 25
}
}
}
}
Environment Variables
Required Variables
Set these environment variables before starting Bifrost:# OpenAI
export OPENAI_API_KEY="sk-..."
# Anthropic
export ANTHROPIC_API_KEY="sk-ant-..."
# AWS Bedrock
export BEDROCK_API_KEY="your-access-key"
export AWS_SECRET_ACCESS_KEY="your-secret-key"
# Azure OpenAI
export AZURE_API_KEY="your-azure-key"
export AZURE_ENDPOINT="https://your-resource.openai.azure.com"
# Google Vertex AI
export VERTEX_PROJECT_ID="your-project-id"
export VERTEX_CREDENTIALS="/path/to/service-account.json"
# Cohere
export COHERE_API_KEY="your-cohere-key"
# Mistral
export MISTRAL_API_KEY="your-mistral-key"
Docker Environment
docker run -p 8080:8080 \
-v $(pwd)/config.json:/app/config/config.json \
-e OPENAI_API_KEY \
-e ANTHROPIC_API_KEY \
-e BEDROCK_API_KEY \
-e AWS_SECRET_ACCESS_KEY \
maximhq/bifrost
Testing Configuration
Validate Provider Setup
# Test OpenAI provider
curl -X POST http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "openai/gpt-4o-mini",
"messages": [{"role": "user", "content": "Test message"}]
}'
# Test with fallbacks
curl -X POST http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "openai/gpt-4o-mini",
"messages": [{"role": "user", "content": "Test message"}],
"fallbacks": [
"anthropic/claude-3-sonnet-20240229"
]
}'
Configuration Validation
# Start Bifrost with config validation
bifrost-http -config config.json -validate
# Check which providers are loaded
curl http://localhost:8080/metrics | grep bifrost_providers
Architecture: For provider selection algorithms and load balancing, see Architecture Documentation.