--- title: "Enterprise AI Gateway Deployment | In-VPC, Air-Gapped, Multi-Cloud" description: "Deployment guide for running Bifrost in a VPC, on-premise, air-gapped, or multi-cloud environment with zero data egress and enterprise compliance controls." url: "https://www.getmaxim.ai/bifrost/resources/enterprise-deployment" markdown: "https://www.getmaxim.ai/bifrost/resources/enterprise-deployment.md" --- # Enterprise AI Gateway Deployment | In-VPC, Air-Gapped, Multi-Cloud > Deployment guide for running Bifrost in a VPC, on-premise, air-gapped, or multi-cloud environment with zero data egress and enterprise compliance controls. ## Important Links - [View MCP Gateway](https://www.getmaxim.ai/bifrost/resources/mcp-gateway.md) - [Features](https://www.getmaxim.ai/bifrost/#features) - [Enterprise](https://www.getmaxim.ai/bifrost/enterprise) - [Pricing](https://www.getmaxim.ai/bifrost/pricing.md) - [Docs](https://docs.getbifrost.ai) - [GitHub](https://github.com/maximhq/bifrost) - [Book a Demo](https://www.getmaxim.ai/bifrost/book-a-demo) ## Performance at a Glance - **11µs Gateway Overhead.** Per request at 5K RPS - **99.999% Uptime SLA.** Enterprise In-VPC deployments - **In-VPC Supported.** AWS, GCP, Azure, Cloudflare, Vercel - **0 Bytes Data Egress.** All processing inside your network ## Problem Cards - **No data sovereignty:** SaaS gateways route every prompt, completion, and API key through a third-party network. Sensitive data leaves your perimeter, and compliance teams block adoption before it starts. - **No air-gapped support:** Classified and regulated environments require zero external network access. SaaS gateways cannot operate offline, and most open-source proxies still phone home for updates or telemetry. - **Multi-cloud fragmentation:** When teams deploy a separate gateway in each cloud, policies drift between environments, configuration is duplicated manually, and rate limits or budgets cannot be enforced across providers from a single control plane. - **No production-grade HA:** Single-instance gateways are single points of failure. Without native clustering, automatic failover, or zero-downtime rolling updates, any outage takes your entire AI stack offline. ## Four Deployment Models, One Gateway - **In-VPC - In-VPC / Private Cloud:** Deploy entirely within your VPC on AWS, GCP, or Azure. Complete network isolation with native IAM integration, private endpoints, and no external dependencies. Platforms: AWS EKS/ECS, GCP GKE, Azure AKS. Proof points: 99.999% SLA, multi-zone HA, zero data egress. [In-VPC deployment docs](https://docs.getbifrost.ai/enterprise/invpc-deployments). - **On-Premise - On-Premise / Bare Metal:** Run on your own hardware via Kubernetes, Docker Compose, or a single binary on bare metal VMs. Full control over compute, storage, and networking with no cloud dependency. Platforms: Kubernetes v1.19+, Docker, single binary. Proof points: private registry mirror, credential rotation, Helm + Compose. [On-premise deployment docs](https://docs.getbifrost.ai/deployment-guides/enterprise/on-premise). - **Air-Gapped - Air-Gapped Environments:** For environments with zero internet access. Export the Bifrost image on a connected machine, transfer via tarball, load into your internal registry. No phone-home, no telemetry. Platforms: Docker save/load, internal registry. Proof points: fully offline operation, no telemetry leakage, image mirroring. [Air-gapped deployment docs](https://docs.getbifrost.ai/deployment-guides/enterprise/overview). - **Edge / Dev - Single-Node & Edge:** Run a single Bifrost instance for dev/test, branch offices, or edge deployments. Minimal footprint with instant setup via Docker Compose, fly.io, or a single Go binary. Platforms: Docker Compose, fly.io, Helm. Proof points: 2 vCPU / 4GB minimum, single binary deploy, 30-second setup. [Edge / single-node deployment docs](https://docs.getbifrost.ai/deployment-guides/enterprise/overview). ## Cloud Rows | Cloud | Targets | Auth | Registry | Iac | | --- | --- | --- | --- | --- | | AWS | EKS, ECS | IRSA | Artifact Registry | Terraform, Helm | | GCP | GKE, Cloud Run | Workload Identity | Artifact Registry | Terraform, Helm | | Azure | AKS | Azure WIF | Artifact Registry | Terraform, Helm | | On-Premise | K8s, Docker, Bare Metal | Basic Auth | Internal mirror | Helm, Compose, Single binary | ### One module, any cloud _main.tf_ ```hcl module "bifrost" { source = "github.com/maximhq/bifrost//terraform/modules/bifrost?ref=terraform/v0.1.0" cloud_provider = "aws" # "aws" | "gcp" | "azure" | "kubernetes" service = "eks" # AWS: "ecs" | "eks", GCP: "gke" | "cloud-run", Azure: "aks" region = "us-east-1" image_tag = "latest" } ``` - [Terraform module: maximhq/bifrost](https://github.com/maximhq/bifrost) ## Core Capabilities - **High Availability & Clustering:** Zero-downtime at scale. Peer-to-peer architecture, no single point of failure; 6 discovery methods: K8s, Consul, etcd, DNS, UDP, mDNS; Gossip-based state sync for rate limits and traffic; 3-node minimum; Zero-downtime rolling updates; Adaptive load balancing with health monitoring - **Security & Compliance:** Zero trust architecture, audit ready. Complete VPC isolation, no external dependencies; Vault: HashiCorp, AWS SM, GCP SM, Azure KV; TLS 1.3 in-transit, KMS encryption at-rest; RBAC + OIDC (Okta, Entra ID) + virtual keys; Audit trails for SOC 2 Type II, GDPR, HIPAA, ISO 27001; Container image signing + vulnerability scanning - **Observability:** Full stack monitoring. Built-in dashboard for real-time monitoring; Native Prometheus metrics (scrape or Push Gateway); OpenTelemetry / OTLP distributed tracing; Native Datadog/BigQuery connectors; Log exports to data lakes and storage; Health endpoints: /health, /ready, /cluster/status ## Setup Steps 1. **01 - Choose deployment model.** VPC, on-prem, air-gapped, or multi-cloud. Pick the model that matches your infrastructure and compliance requirements. ### Choose deployment model ``` # In-VPC, On-Premise, Air-Gapped, Multi-Cloud, or Edge ``` 2. **02 - Provision infrastructure.** Use the Terraform module or Helm chart. AWS, GCP, Azure, and generic Kubernetes all supported out of the box. ### Provision infrastructure ``` terraform apply ``` 3. **03 - Configure Bifrost.** Single JSON config file. Connect config store (Postgres or file), set up providers, define virtual keys and routing rules. ### Configure Bifrost ``` # config.json # providers, virtual keys, routing ``` 4. **04 - Deploy.** Single Go binary with minimal dependencies. Helm install, docker compose up, or terraform apply. ### Deploy ``` helm install bifrost maximhq/bifrost ``` 5. **05 - Integrate your SDKs.** Change one line in your existing OpenAI, Anthropic, or LiteLLM SDK. Point at your Bifrost endpoint. ### Integrate your SDKs ``` base_url = "https://bifrost.internal" ``` 6. **06 - Monitor & scale.** Enable clustering for HA, connect Prometheus or OpenTelemetry, set up auto-scaling. Production ready. ### Monitor & scale ``` curl http://bifrost.internal/cluster/status ``` ## Comparison Rows | Capability | SaaS | OSS | Bifrost | | --- | --- | --- | --- | | In-VPC deployment | No | Manual setup | Terraform + Helm | | Air-gapped support | No | No | Docker save/load | | Cloud-native auth | API key only | API key only | IRSA, Workload Identity, Azure WIF | | Vault integration | No | No | HashiCorp, AWS SM, GCP SM, Azure KV | | RBAC + SSO | Limited | No | Okta, Entra ID, OIDC | | Audit logs (SOC 2 Type II, HIPAA) | Limited | No | Immutable, compliance ready | | Data stays in your network | No | Yes | Yes, zero egress | | P99 latency at 500 RPS | ~50ms | ~90.72s | ~1.68s | | Uptime SLA | Varies | None | 99.999% | ## Use Cases - **Regulated Industries - Healthcare, Finance & Government:** Compliance requires all AI traffic to stay within your VPC. No data can transit third-party proxies. In-VPC deployment with complete network isolation, audit logs for HIPAA/SOC 2 Type II/GDPR, Vault for secrets, and zero data egress. All processing stays inside your controlled environment. - **Defense & Air-Gapped - Zero Internet Access Environments:** Your environment has no internet. You need a gateway that runs entirely offline with no phone-home. Docker image export/import workflow, internal registry mirroring, no telemetry leakage. On-prem credential management with offline operation and manual update cycles. - **Multi-Cloud Enterprise - AWS + GCP + Azure, Unified:** Your org runs AWS for production, GCP for ML, Azure for business units. You need one gateway with consistent governance. Clustered deployment across clouds with gossip-based state sync, cloud-native auth per environment (IRSA, Workload Identity, WIF), and unified rate limiting and budgets. - **Gateway Migration - Migrating from LiteLLM or Other Proxies:** You're hitting performance ceilings, missing enterprise features, or struggling with reliability on your current gateway. Drop-in LiteLLM compatibility, 54x faster P99 latency, 68% less memory. Plus enterprise features like RBAC, guardrails, clustering, and audit logs that alternatives don't offer. ## Compliance Badges - **Audited quarterly - SOC 2 Type II:** Compliance signal for audit-ready enterprise deployments. - **EU data residency - GDPR:** Data protection signal for privacy and residency reviews. - **BAA available - HIPAA:** Healthcare compliance signal for protected health information workflows. - **Certified - ISO 27001:** Security management signal for enterprise risk reviews. ## Features ### OSS Features - **01 - Model Catalog:** Access 8+ providers and 1000+ AI models through a unified interface. Also supports custom deployed models. - **02 - Budgeting:** Set spending limits and track costs across teams, projects, and models. - **03 - Provider Fallback:** Automatic failover between providers ensures 99.99% uptime for your applications. - **04 - MCP Gateway:** Centralize all MCP tool connections, governance, security, and auth. Your AI can safely use MCP tools with centralized policy enforcement. [MCP Gateway resource](https://www.getmaxim.ai/bifrost/resources/mcp-gateway.md). - **05 - Virtual Key Management:** Create different virtual keys for different use cases with independent budgets and access control. - **06 - Unified Interface:** One consistent API for all providers. Switch models without changing code. - **07 - Drop-in Replacement:** Replace your existing SDK with just one line change. Compatible with OpenAI, Anthropic, LiteLLM, Google GenAI, LangChain, and more. [Drop-in replacement docs](https://docs.getbifrost.ai/features/drop-in-replacement). - **08 - Built-in Observability:** Out-of-the-box OpenTelemetry support. Built-in dashboard for quick visibility without complex setup. - **09 - Community Support:** Active Discord community with responsive support and regular updates. ### Enterprise Features - **01 - Governance:** SAML support for SSO and role-based access control with policy enforcement for team collaboration. [Governance resource](https://www.getmaxim.ai/bifrost/resources/governance.md). - **02 - Adaptive Load Balancing:** Automatically optimizes traffic distribution across provider keys and models based on real-time performance metrics. - **03 - Cluster Mode:** High availability deployment with automatic failover and load balancing. Peer-to-peer clustering where every instance is equal. - **04 - Alerts:** Real-time notifications for budget limits, failures, and performance issues on Email, Slack, PagerDuty, Teams, Webhook, and more. - **05 - Log Exports:** Export and analyze request logs, traces, and telemetry data from Bifrost with enterprise-grade data export for compliance, monitoring, and analytics. - **06 - Audit Logs:** Comprehensive logging and audit trails for compliance and debugging. - **07 - Vault Support:** Secure API key management with HashiCorp Vault, AWS Secrets Manager, Google Secret Manager, and Azure Key Vault integration. - **08 - VPC Deployment:** Deploy Bifrost within your private cloud infrastructure with VPC isolation, custom networking, and enhanced security controls. [Enterprise deployment resource](https://www.getmaxim.ai/bifrost/resources/enterprise-deployment.md). - **09 - Guardrails:** Automatically detect and block unsafe model outputs with real-time policy enforcement and content moderation across all agents. [Guardrails resource](https://www.getmaxim.ai/bifrost/resources/guardrails.md). ## FAQ ### Can Bifrost be deployed entirely within our VPC with zero data egress? Yes. Bifrost deploys entirely within your VPC on AWS, GCP, or Azure with complete network isolation. All LLM requests, API keys, prompts, and completions stay within your network perimeter. Combined with vault support, no secrets leave your infrastructure. ### Does Bifrost support air-gapped environments with no internet? Yes. Export the Bifrost Docker image on a connected machine using docker save, transfer the tarball to your air-gapped environment, and load it into your internal registry. No phone-home, no telemetry, fully offline operation. ### What infrastructure-as-code options are available? Bifrost provides a single Terraform module that targets AWS (EKS/ECS), GCP (GKE/Cloud Run), Azure (AKS), and generic Kubernetes. Helm charts are also available for Kubernetes deployments. Docker Compose and single binary are provided for on-premise and bare metal. ### What is the minimum hardware requirement? A single Bifrost node requires 2 vCPU and 4GB RAM minimum. For production HA deployments, a 3-node cluster is recommended. The Go-native binary has a minimal footprint with 68% less memory usage than Python-based alternatives. ## Related Resources - [Source: Enterprise Deployment](https://www.getmaxim.ai/bifrost/resources/enterprise-deployment.md) - [Docs: Bifrost docs](https://docs.getbifrost.ai) - [GitHub: maximhq/bifrost](https://github.com/maximhq/bifrost) - [Pricing: Bifrost pricing](https://www.getmaxim.ai/bifrost/pricing.md) - [Enterprise: Bifrost enterprise](https://www.getmaxim.ai/bifrost/enterprise) - [Book a Demo: Bifrost demo](https://www.getmaxim.ai/bifrost/book-a-demo) - [Resources: Bifrost resources](https://www.getmaxim.ai/bifrost/resources.md) --- *This is a markdown version of [https://www.getmaxim.ai/bifrost/resources/enterprise-deployment](https://www.getmaxim.ai/bifrost/resources/enterprise-deployment) for AI/LLM consumption.*