Enterprise AI Gateway Deployment | In-VPC, Air-Gapped, Multi-Cloud

Performance at a Glance

Gateway Overhead

11µs Per request at 5K RPS

Uptime SLA

99.999% Enterprise In-VPC deployments

Supported

In-VPC AWS, GCP, Azure, Cloudflare, Vercel

Data Egress

0 Bytes All processing inside your network

Problem Cards

• No data sovereignty. SaaS gateways route every prompt, completion, and API key through a third-party network. Sensitive data leaves your perimeter, and compliance teams block adoption before it starts.
• No air-gapped support. Classified and regulated environments require zero external network access. SaaS gateways cannot operate offline, and most open-source proxies still phone home for updates or telemetry.
• Multi-cloud fragmentation. When teams deploy a separate gateway in each cloud, policies drift between environments, configuration is duplicated manually, and rate limits or budgets cannot be enforced across providers from a single control plane.
• No production-grade HA. Single-instance gateways are single points of failure. Without native clustering, automatic failover, or zero-downtime rolling updates, any outage takes your entire AI stack offline.

Four Deployment Models, One Gateway

• In-VPCIn-VPC / Private Cloud. Deploy entirely within your VPC on AWS, GCP, or Azure. Complete network isolation with native IAM integration, private endpoints, and no external dependencies. Platforms: AWS EKS/ECS, GCP GKE, Azure AKS. Proof points: 99.999% SLA, multi-zone HA, zero data egress. [In-VPC deployment docs]
• On-PremiseOn-Premise / Bare Metal. Run on your own hardware via Kubernetes, Docker Compose, or a single binary on bare metal VMs. Full control over compute, storage, and networking with no cloud dependency. Platforms: Kubernetes v1.19+, Docker, single binary. Proof points: private registry mirror, credential rotation, Helm + Compose. [On-premise deployment docs]
• Air-GappedAir-Gapped Environments. For environments with zero internet access. Export the Bifrost image on a connected machine, transfer via tarball, load into your internal registry. No phone-home, no telemetry. Platforms: Docker save/load, internal registry. Proof points: fully offline operation, no telemetry leakage, image mirroring. [Air-gapped deployment docs]
• Edge / DevSingle-Node & Edge. Run a single Bifrost instance for dev/test, branch offices, or edge deployments. Minimal footprint with instant setup via Docker Compose, fly.io, or a single Go binary. Platforms: Docker Compose, fly.io, Helm. Proof points: 2 vCPU / 4GB minimum, single binary deploy, 30-second setup. [Edge / single-node deployment docs]

Cloud Rows

module "bifrost" {
  source         = "github.com/maximhq/bifrost//terraform/modules/bifrost?ref=terraform/v0.1.0"
  cloud_provider = "aws"          # "aws" | "gcp" | "azure" | "kubernetes"
  service        = "eks"          # AWS: "ecs" | "eks", GCP: "gke" | "cloud-run", Azure: "aks"
  region         = "us-east-1"
  image_tag      = "latest"
}

• [Terraform module: maximhq/bifrost]

Core Capabilities

• High Availability & Clustering. Zero-downtime at scale. Peer-to-peer architecture, no single point of failure; 6 discovery methods: K8s, Consul, etcd, DNS, UDP, mDNS; Gossip-based state sync for rate limits and traffic; 3-node minimum; Zero-downtime rolling updates; Adaptive load balancing with health monitoring
• Security & Compliance. Zero trust architecture, audit ready. Complete VPC isolation, no external dependencies; Vault: HashiCorp, AWS SM, GCP SM, Azure KV; TLS 1.3 in-transit, KMS encryption at-rest; RBAC + OIDC (Okta, Entra ID) + virtual keys; Audit trails for SOC 2 Type II, GDPR, HIPAA, ISO 27001; Container image signing + vulnerability scanning
• Observability. Full stack monitoring. Built-in dashboard for real-time monitoring; Native Prometheus metrics (scrape or Push Gateway); OpenTelemetry / OTLP distributed tracing; Native Datadog/BigQuery connectors; Log exports to data lakes and storage; Health endpoints: /health, /ready, /cluster/status

Setup Steps

1. 01Choose deployment model. VPC, on-prem, air-gapped, or multi-cloud. Pick the model that matches your infrastructure and compliance requirements.

   # In-VPC, On-Premise, Air-Gapped, Multi-Cloud, or Edge

2. 02Provision infrastructure. Use the Terraform module or Helm chart. AWS, GCP, Azure, and generic Kubernetes all supported out of the box.

   terraform apply

3. 03Configure Bifrost. Single JSON config file. Connect config store (Postgres or file), set up providers, define virtual keys and routing rules.

   # config.json
   # providers, virtual keys, routing

4. 04Deploy. Single Go binary with minimal dependencies. Helm install, docker compose up, or terraform apply.

   helm install bifrost maximhq/bifrost

5. 05Integrate your SDKs. Change one line in your existing OpenAI, Anthropic, or LiteLLM SDK. Point at your Bifrost endpoint.

   base_url = "https://bifrost.internal"

6. 06Monitor & scale. Enable clustering for HA, connect Prometheus or OpenTelemetry, set up auto-scaling. Production ready.

   curl http://bifrost.internal/cluster/status

Comparison Rows

Use Cases

• Regulated IndustriesHealthcare, Finance & Government. Compliance requires all AI traffic to stay within your VPC. No data can transit third-party proxies. In-VPC deployment with complete network isolation, audit logs for HIPAA/SOC 2 Type II/GDPR, Vault for secrets, and zero data egress. All processing stays inside your controlled environment.
• Defense & Air-GappedZero Internet Access Environments. Your environment has no internet. You need a gateway that runs entirely offline with no phone-home. Docker image export/import workflow, internal registry mirroring, no telemetry leakage. On-prem credential management with offline operation and manual update cycles.
• Multi-Cloud EnterpriseAWS + GCP + Azure, Unified. Your org runs AWS for production, GCP for ML, Azure for business units. You need one gateway with consistent governance. Clustered deployment across clouds with gossip-based state sync, cloud-native auth per environment (IRSA, Workload Identity, WIF), and unified rate limiting and budgets.
• Gateway MigrationMigrating from LiteLLM or Other Proxies. You're hitting performance ceilings, missing enterprise features, or struggling with reliability on your current gateway. Drop-in LiteLLM compatibility, 54x faster P99 latency, 68% less memory. Plus enterprise features like RBAC, guardrails, clustering, and audit logs that alternatives don't offer.

Compliance Badges

• Audited quarterlySOC 2 Type II. Compliance signal for audit-ready enterprise deployments.
• EU data residencyGDPR. Data protection signal for privacy and residency reviews.
• BAA availableHIPAA. Healthcare compliance signal for protected health information workflows.
• CertifiedISO 27001. Security management signal for enterprise risk reviews.

Open Source & Enterprise

FAQ

Related resources

• [Source: Enterprise Deployment]
• [Docs: Bifrost docs]
• [GitHub: maximhq/bifrost]
• [Pricing: Bifrost pricing]
• [Enterprise: Bifrost enterprise]
• [Book a Demo: Bifrost demo]
• [Resources: Bifrost resources]