Networking

Overview

Networking Features:

Proxy Support - HTTP, SOCKS5, and environment-based proxy configuration
Connection Pooling - Optimize network resources and performance
Custom Headers - Add authentication, organization, or tracking headers
Timeout Control - Fine-grained timeout configuration per provider
Retry Logic - Exponential backoff with configurable retry policies
Base URL Override - Custom endpoints for enterprise deployments

Benefits:

Better Performance - Connection reuse and pooling
Enterprise Ready - Proxy and firewall compatibility
Fault Tolerance - Automatic retry with backoff strategies
Flexible Deployment - Custom endpoints and headers

Basic Network Configuration

Default Network Settings

Go Package Usage

func (a *MyAccount) GetConfigForProvider(provider schemas.ModelProvider) (*schemas.ProviderConfig, error) {
    return &schemas.ProviderConfig{
        NetworkConfig: schemas.NetworkConfig{
            // Custom endpoint (optional)
            BaseURL: "https://api.openai.com",

            // Custom headers
            ExtraHeaders: map[string]string{
                "X-Organization": "my-org-id",
                "X-Environment":  "production",
                "User-Agent":     "MyApp/1.0",
            },

            // Timeout configuration
            DefaultRequestTimeoutInSeconds: 60,  // 60 second timeout

            // Retry configuration
            MaxRetries:          3,                        // Retry up to 3 times
            RetryBackoffInitial: 500 * time.Millisecond,  // Start with 500ms
            RetryBackoffMax:     10 * time.Second,        // Max 10 seconds
        },
    }, nil
}

Network Configuration Options:

Field	Type	Description	Default
`BaseURL`	`string`	Custom provider endpoint	Provider default
`ExtraHeaders`	`map[string]string`	Additional HTTP headers	`{}`
`DefaultRequestTimeoutInSeconds`	`int`	Request timeout	`30`
`MaxRetries`	`int`	Retry attempts	`0`
`RetryBackoffInitial`	`time.Duration`	Initial retry delay	`500ms`
`RetryBackoffMax`	`time.Duration`	Maximum retry delay	`5s`

HTTP Transport Usage Configuration (config.json):

{
  "providers": {
    "openai": {
      "keys": [
        {
          "value": "env.OPENAI_API_KEY",
          "models": ["gpt-4o-mini"],
          "weight": 1.0
        }
      ],
      "network_config": {
        "base_url": "https://api.openai.com",
        "extra_headers": {
          "X-Organization-ID": "org-123",
          "X-Environment": "production",
          "User-Agent": "MyApp/1.0"
        },
        "default_request_timeout_in_seconds": 30,
        "max_retries": 1,
        "retry_backoff_initial_ms": 100,
        "retry_backoff_max_ms": 2000
      }
    }
  }
}

Proxy Configuration

HTTP Proxy

Go Package - HTTP Proxy

func (a *MyAccount) GetConfigForProvider(provider schemas.ModelProvider) (*schemas.ProviderConfig, error) {
    return &schemas.ProviderConfig{
        ProxyConfig: &schemas.ProxyConfig{
            Type:     schemas.HttpProxy,
            URL:      "http://proxy.company.com:8080",
            Username: "proxy-user",     // Optional authentication
            Password: "proxy-pass",     // Optional authentication
        },
        NetworkConfig: schemas.NetworkConfig{
            DefaultRequestTimeoutInSeconds: 45,  // Increase timeout for proxy
        },
    }, nil
}

Proxy Configuration Options:

Field	Type	Description	Required
`Type`	`ProxyType`	Proxy type (http/socks5/env)	✅
`URL`	`string`	Proxy server URL	✅
`Username`	`string`	Proxy authentication user	❌
`Password`	`string`	Proxy authentication pass	❌

HTTP Transport - HTTP Proxy

{
  "providers": {
    "openai": {
      "keys": [
        {
          "value": "env.OPENAI_API_KEY",
          "models": ["gpt-4o-mini"],
          "weight": 1.0
        }
      ],
      "proxy_config": {
        "type": "http",
        "url": "http://proxy.company.com:8080",
        "username": "proxy-user",
        "password": "proxy-pass"
      },
      "network_config": {
        "default_request_timeout_in_seconds": 45
      }
    }
  }
}

SOCKS5 Proxy

Go Package - SOCKS5 Proxy

func (a *MyAccount) GetConfigForProvider(provider schemas.ModelProvider) (*schemas.ProviderConfig, error) {
    return &schemas.ProviderConfig{
        ProxyConfig: &schemas.ProxyConfig{
            Type:     schemas.Socks5Proxy,
            URL:      "socks5://proxy.company.com:1080",
            Username: "socks-user",     // Optional
            Password: "socks-pass",     // Optional
        },
    }, nil
}

HTTP Transport - SOCKS5 Proxy

{
  "providers": {
    "openai": {
      "proxy_config": { 
        "type": "socks5",
        "url": "socks5://proxy.company.com:1080",
        "username": "socks-user",
        "password": "socks-pass"
      }
    }
  }
}

Environment-Based Proxy

Go Package - Environment Proxy

func (a *MyAccount) GetConfigForProvider(provider schemas.ModelProvider) (*schemas.ProviderConfig, error) {
    return &schemas.ProviderConfig{
        ProxyConfig: &schemas.ProxyConfig{
            Type: schemas.EnvProxy,
            // Automatically uses HTTP_PROXY, HTTPS_PROXY, NO_PROXY environment variables
        },
    }, nil
}

Environment Variables:

export HTTP_PROXY=http://proxy.company.com:8080
export HTTPS_PROXY=https://proxy.company.com:8443
export NO_PROXY=localhost,127.0.0.1,.company.com

HTTP Transport - Environment Proxy

{
  "providers": {
    "openai": {
      "proxy_config": {
        "type": "env"
      }
    }
  }
}

Environment Variables:

export HTTP_PROXY=http://proxy.company.com:8080
export HTTPS_PROXY=https://proxy.company.com:8443
export NO_PROXY=localhost,127.0.0.1,.company.com

Timeout & Retry Configuration

Basic Retry Logic

Go Package - Retry Configuration

func (a *MyAccount) GetConfigForProvider(provider schemas.ModelProvider) (*schemas.ProviderConfig, error) {
    return &schemas.ProviderConfig{
        NetworkConfig: schemas.NetworkConfig{
            // Timeout settings
            DefaultRequestTimeoutInSeconds: 30,

            // Retry settings with exponential backoff
            MaxRetries:          3,                         // Retry up to 3 times
            RetryBackoffInitial: 500 * time.Millisecond,   // Start with 500ms
            RetryBackoffMax:     10 * time.Second,         // Cap at 10 seconds
        },
    }, nil
}

Retry Logic:

Attempt 1: Request fails
Wait: 500ms (initial backoff)

Attempt 2: Request fails
Wait: 1000ms (2x backoff)

Attempt 3: Request fails
Wait: 2000ms (2x backoff)

Attempt 4: Request fails
Give up after 3 retries

HTTP Transport - Retry Configuration

{
  "providers": {
    "openai": {
      "network_config": {
        "default_request_timeout_in_seconds": 30,
        "max_retries": 3,
        "retry_backoff_initial_ms": 500,
        "retry_backoff_max_ms": 10000
      }
    }
  }
}

Provider-Specific Timeouts

Go Package - Provider-Specific Timeouts

func (a *MyAccount) GetConfigForProvider(provider schemas.ModelProvider) (*schemas.ProviderConfig, error) {
    switch provider {
    case schemas.OpenAI:
        return &schemas.ProviderConfig{
            NetworkConfig: schemas.NetworkConfig{
                DefaultRequestTimeoutInSeconds: 30,  // Fast timeout for OpenAI
                MaxRetries: 2,
            },
        }, nil
    case schemas.Anthropic:
        return &schemas.ProviderConfig{
            NetworkConfig: schemas.NetworkConfig{
                DefaultRequestTimeoutInSeconds: 60,  // Longer timeout for Claude
                MaxRetries: 3,
            },
        }, nil
    case schemas.Bedrock:
        return &schemas.ProviderConfig{
            NetworkConfig: schemas.NetworkConfig{
                DefaultRequestTimeoutInSeconds: 120, // Longest timeout for Bedrock
                MaxRetries: 1,                       // Fewer retries for AWS
            },
        }, nil
    }
    return &schemas.ProviderConfig{}, nil
}

HTTP Transport - Provider-Specific Timeouts

{
  "providers": {
    "openai": {
      "network_config": {
        "default_request_timeout_in_seconds": 30,
        "max_retries": 2
      }
    },
    "anthropic": {
      "network_config": {
        "default_request_timeout_in_seconds": 60,
        "max_retries": 3
      }
    },
    "bedrock": {
      "network_config": {
        "default_request_timeout_in_seconds": 120,
        "max_retries": 1
      }
    }
  }
}

Custom Headers

Authentication Headers

Go Package - Custom Headers

func (a *MyAccount) GetConfigForProvider(provider schemas.ModelProvider) (*schemas.ProviderConfig, error) {
    switch provider {
    case schemas.OpenAI:
        return &schemas.ProviderConfig{
            NetworkConfig: schemas.NetworkConfig{
                ExtraHeaders: map[string]string{
                    "OpenAI-Organization": os.Getenv("OPENAI_ORG_ID"),
                    "OpenAI-Project":      os.Getenv("OPENAI_PROJECT_ID"),
                    "User-Agent":          "MyApp/1.0.0",
                    "X-Request-ID":        generateRequestID(),
                },
            },
        }, nil
    case schemas.Anthropic:
        return &schemas.ProviderConfig{
            NetworkConfig: schemas.NetworkConfig{
                ExtraHeaders: map[string]string{
                    "User-Agent":       "MyApp/1.0.0",
                    "X-Source":         "bifrost-gateway",
                    "anthropic-version": "2023-06-01",
                },
            },
        }, nil
    }
    return &schemas.ProviderConfig{}, nil
}

func generateRequestID() string {
    return fmt.Sprintf("req_%d", time.Now().UnixNano())
}

HTTP Transport - Custom Headers

{
  "providers": {
    "openai": {
      "network_config": {
        "extra_headers": {
          "OpenAI-Organization": "org-your-org-id",
          "OpenAI-Project": "proj-your-project-id",
          "User-Agent": "MyApp/1.0.0",
          "X-Source": "bifrost-gateway"
        }
      }
    },
    "anthropic": {
      "network_config": {
        "extra_headers": {
          "User-Agent": "MyApp/1.0.0",
          "X-Source": "bifrost-gateway",
          "anthropic-version": "2023-06-01"
        }
      }
    }
  }
}

Tracking and Monitoring Headers

Go Package - Monitoring Headers

func (a *MyAccount) GetConfigForProvider(provider schemas.ModelProvider) (*schemas.ProviderConfig, error) {
    return &schemas.ProviderConfig{
        NetworkConfig: schemas.NetworkConfig{
            ExtraHeaders: map[string]string{
                // Tracking headers
                "X-Request-ID":     generateRequestID(),
                "X-Session-ID":     getSessionID(),
                "X-User-ID":        getUserID(),
                "X-Environment":    os.Getenv("ENVIRONMENT"),

                // Application metadata
                "X-App-Version":    "1.2.3",
                "X-Build-Hash":     getBuildHash(),
                "X-Deployment-ID":  getDeploymentID(),

                // Monitoring
                "X-Trace-ID":       getTraceID(),
                "X-Span-ID":        getSpanID(),
            },
        },
    }, nil
}

HTTP Transport - Monitoring Headers

{
  "providers": {
    "openai": {
      "network_config": {
        "extra_headers": {
          "X-Environment": "production",
          "X-App-Version": "1.2.3",
          "X-Build-Hash": "abc123def",
          "X-Deployment-ID": "deploy-456",
          "X-Source": "bifrost-gateway"
        }
      }
    }
  }
}

Enterprise Configuration

Corporate Network Setup

Enterprise Network Configuration Go Package - Enterprise Setup:

func (a *MyAccount) GetConfigForProvider(provider schemas.ModelProvider) (*schemas.ProviderConfig, error) {
    return &schemas.ProviderConfig{
        // Corporate proxy
        ProxyConfig: &schemas.ProxyConfig{
            Type:     schemas.HttpProxy,
            URL:      "http://corporate-proxy.company.com:8080",
            Username: os.Getenv("PROXY_USER"),
            Password: os.Getenv("PROXY_PASS"),
        },

        NetworkConfig: schemas.NetworkConfig{
            // Conservative timeouts for corporate networks
            DefaultRequestTimeoutInSeconds: 90,

            // Corporate headers
            ExtraHeaders: map[string]string{
                "X-Corporate-ID":   os.Getenv("CORP_ID"),
                "X-Department":     "AI-Team",
                "X-Cost-Center":    "CC-123",
                "X-Compliance":     "SOC2-Type2",
            },

            // Aggressive retry for unreliable corporate networks
            MaxRetries:          5,
            RetryBackoffInitial: 1 * time.Second,
            RetryBackoffMax:     30 * time.Second,
        },
    }, nil
}

HTTP Transport - Enterprise Setup:

{
  "providers": {
    "openai": {
      "proxy_config": {
        "type": "http",
        "url": "http://corporate-proxy.company.com:8080",
        "username": "env.PROXY_USER",
        "password": "env.PROXY_PASS"
      },
      "network_config": {
        "default_request_timeout_in_seconds": 90,
        "extra_headers": {
          "X-Corporate-ID": "corp-123",
          "X-Department": "AI-Team",
          "X-Cost-Center": "CC-123",
          "X-Compliance": "SOC2-Type2"
        },
        "max_retries": 5,
        "retry_backoff_initial_ms": 1000,
        "retry_backoff_max_ms": 30000
      }
    }
  }
}

Multi-Region Configuration

🌍 Multi-Region Setup Go Package - Regional Endpoints:

func (a *MyAccount) GetConfigForProvider(provider schemas.ModelProvider) (*schemas.ProviderConfig, error) {
    region := os.Getenv("DEPLOYMENT_REGION")

    switch provider {
    case schemas.OpenAI:
        // Use regional endpoints for better latency
        baseURL := "https://api.openai.com"
        if region == "eu-west-1" {
            baseURL = "https://api.openai.com"  // OpenAI doesn't have regional endpoints
        }

        return &schemas.ProviderConfig{
            NetworkConfig: schemas.NetworkConfig{
                BaseURL: baseURL,
                ExtraHeaders: map[string]string{
                    "X-Region":           region,
                    "X-Preferred-Region": "eu-west-1",
                },
            },
        }, nil

    case schemas.Bedrock:
        // Use actual AWS regions
        bedrockRegion := "us-east-1"
        if region == "eu-west-1" {
            bedrockRegion = "eu-west-1"
        }

        return &schemas.ProviderConfig{
            MetaConfig: map[string]interface{}{
                "region": bedrockRegion,
            },
        }, nil
    }

    return &schemas.ProviderConfig{}, nil
}

Best Practices

Timeout Strategy

Recommended Timeout Values

Use Case	Timeout	Max Retries	Initial Backoff
Interactive Chat	30s	2	500ms
Batch Processing	120s	5	1s
Real-time API	15s	1	250ms
Background Jobs	300s	3	2s

// Example: Interactive chat configuration
func getInteractiveChatConfig() *schemas.ProviderConfig {
    return &schemas.ProviderConfig{
        NetworkConfig: schemas.NetworkConfig{
            DefaultRequestTimeoutInSeconds: 30,
            MaxRetries:                     2,
            RetryBackoffInitial:            500 * time.Millisecond,
            RetryBackoffMax:                5 * time.Second,
        },
    }
}

Proxy Best Practices

Proxy Configuration Tips Corporate Environment:

# Set proxy environment variables
export HTTP_PROXY=http://proxy.corp.com:8080
export HTTPS_PROXY=http://proxy.corp.com:8080
export NO_PROXY=localhost,127.0.0.1,*.corp.com

# Test proxy connectivity
curl -v --proxy $HTTP_PROXY https://api.openai.com/v1/models

Docker Environment:

# Pass proxy settings to container
ENV HTTP_PROXY=http://proxy.company.com:8080
ENV HTTPS_PROXY=http://proxy.company.com:8080
ENV NO_PROXY=localhost,127.0.0.1

Next Steps

Task	Documentation
Configure API keys	Key Management
Set up providers	Providers
Optimize performance	Memory Management
Handle network errors	Error Handling

Tip: Always test your proxy and timeout settings in a staging environment before deploying to production.

Introduction

Prompt Engineering

Offline Evals

Online Evals

Tracing

Simulations

Library

Dashboards

Integrations

Settings

CI/CD

Overview

Basic Network Configuration

Default Network Settings

Proxy Configuration

HTTP Proxy

SOCKS5 Proxy

Environment-Based Proxy

Timeout & Retry Configuration

Basic Retry Logic

Provider-Specific Timeouts

Custom Headers

Authentication Headers

Tracking and Monitoring Headers

Enterprise Configuration

Corporate Network Setup

Multi-Region Configuration

Best Practices

Timeout Strategy

Proxy Best Practices

Next Steps

​Overview

​Basic Network Configuration

​Default Network Settings

​Proxy Configuration

​HTTP Proxy

​SOCKS5 Proxy

​Environment-Based Proxy

​Timeout & Retry Configuration

​Basic Retry Logic

​Provider-Specific Timeouts

​Custom Headers

​Authentication Headers

​Tracking and Monitoring Headers

​Enterprise Configuration

​Corporate Network Setup

​Multi-Region Configuration

​Best Practices

​Timeout Strategy

​Proxy Best Practices

​Next Steps

Overview

Basic Network Configuration

Default Network Settings

Proxy Configuration

HTTP Proxy

SOCKS5 Proxy

Environment-Based Proxy

Timeout & Retry Configuration

Basic Retry Logic

Provider-Specific Timeouts

Custom Headers

Authentication Headers

Tracking and Monitoring Headers

Enterprise Configuration

Corporate Network Setup

Multi-Region Configuration

Best Practices

Timeout Strategy

Proxy Best Practices

Next Steps