--- description: "Deploy an LLM stack in an air-gapped environment: move artifacts across the gap, serve self-hosted models, add a gateway, guardrails, identity, and logs." title: How to Deploy an LLM Stack in an Air-Gapped Environment image: https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w1200/2026/10/how-to-deploy-an-llm-stack-in-an-air-gapped-environment-bifrost-1920x1080.png --- Try Bifrost Enterprise free for 14 days. [Request access](https://www.getmaxim.ai/#enterprise-trial) **TL;DR** - An air-gapped environment has no physical or automated logical connection to outside networks, so model weights, container images, and reference data must cross by a controlled, verified transfer. - A production LLM stack in an air-gapped environment has six layers: artifact transfer, model serving, a gateway control plane, identity, guardrails, and observability. - Bifrost runs fully offline once its pricing datasheets load from local files and the MCP catalog sync is disabled; no prompts, responses, or telemetry leave the network. - Self-hosted model servers such as vLLM, SGLang, and Ollama sit behind Bifrost as providers, so applications call one OpenAI-compatible API with virtual keys, fallbacks, and guardrails. - Secrets detection and regex guardrails run inside the Bifrost process, and LLM-judge guardrails can use a self-hosted model, so content safety works without a cloud service. An air-gapped environment is a network with no physical or automated logical connection to the internet, so every model, container image, and update enters through a controlled manual transfer. Running LLMs inside one takes more than a model server: teams also need access control, guardrails, audit trails, and a repeatable update path. [Bifrost](https://www.getmaxim.ai/), the [open-source AI gateway](https://github.com/maximhq/bifrost) built by Maxim AI, runs as a self-hosted control plane inside that boundary and governs every model call made there. This guide walks through each layer of an air-gapped LLM stack in the order you deploy it. ## What Is an Air-Gapped Environment? An air-gapped environment is an isolated network whose systems are not physically connected to outside networks and whose data transfers are never automated. NIST defines an air gap as an interface where two systems [are not connected physically and any logical connection is not automated](https://csrc.nist.gov/glossary/term/air_gap), so data moves only manually, under human control. Air-gapped networks are common in defense, critical infrastructure, and parts of financial services and healthcare. For AI teams, that isolation breaks several assumptions most LLM tooling makes: - **Model downloads** from public hubs fail, so weights must be staged and transferred. - **Pricing lookups and telemetry** that call vendor endpoints fail or stall startup. - **Cloud guardrail and moderation APIs** are unreachable, so content safety must run locally. - **Updates** arrive in scheduled batches, not continuously. Teams evaluating platforms for this setting can compare options in our roundup of [AI governance platforms for air-gapped deployments](https://www.getmaxim.ai/articles/top-5-ai-governance-platforms-for-air-gapped-deployments/). ## The Layers of an Air-Gapped AI Stack Air-gapped AI is a deployment model in which models, the serving runtime, and every control around them run entirely inside an isolated network. A complete stack has six layers, and each one has to work without an outbound connection: artifact transfer, model serving, a gateway control plane, identity, guardrails, and observability. ![](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/2026/10/image-34.png) ***Figure 1: Every layer of the stack runs inside the boundary, and the internal registry is the only path by which new software and weights arrive.*** A model server alone has no notion of which team is calling, what they may send, or who changed a configuration. Those controls belong in the gateway layer. [The Bifrost AI gateway](https://www.getmaxim.ai/) is self-hosted, ships as a single binary or container, and is [air-gapped by default](https://www.getmaxim.ai/enterprise), with no prompts, responses, files, or telemetry leaving the customer's infrastructure. The [enterprise deployment guide](https://www.getmaxim.ai/resources/enterprise-deployment) compares in-VPC, on-prem, and disconnected patterns. | Layer | Job inside the air gap | Typical components | | --- | --- | --- | | Artifact transfer | Bring images, charts, weights, and reference data in, verified | Staging host, checksums, internal registry | | Model serving | Run open-weight models on local GPUs | vLLM, SGLang, Ollama | | Gateway control plane | One API, keys, routing, fallbacks, logs | Bifrost cluster | | Identity and secrets | Map people and services to permissions | Self-hosted OIDC IdP, Vault | | Guardrails | Block or redact unsafe inputs and outputs | In-process and self-hosted guardrails | | Observability | Metrics, traces, request and audit logs | Prometheus, OpenTelemetry collector, object storage | ## Approaches to On-Premise LLM Deployment An on-premise LLM deployment can expose model servers directly to applications, wrap them in a custom proxy, or place a gateway control plane in front of them. Direct access is fastest to stand up but leaves access control, audit, and failover to each application team; a gateway centralizes them. | Approach | Access control | Audit trail | Failover across model servers | Effort to maintain offline | | --- | --- | --- | --- | --- | | Applications call model servers directly | Per server, often none | Server logs only | None | Low at first, grows with each app | | Custom proxy built in-house | Whatever the team builds | Whatever the team builds | Custom code | High; every feature is owned internally | | Gateway control plane (Bifrost) | Virtual keys, RBAC, SSO | Request logs plus signed audit logs | Built-in fallbacks | One component to update per release | The trade-off is sharper offline, because every internally built component needs its own transfer, patching, and review process. For a broader comparison, see our guide to [self-hosted open-source LLM gateways for enterprise AI](https://www.getmaxim.ai/articles/best-self-hosted-open-source-llm-gateways-for-enterprise-ai-in-2026/). ## Step 1: Prepare an Air-Gapped Installation Bundle An air-gapped installation starts on a connected staging host, where every artifact is downloaded, pinned to an exact version, checksummed, and scanned before transfer. The bundle holds container images, Helm charts, model weights, and the reference data the gateway reads at startup. ![](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/2026/10/image-35.png) ***Figure 2: Verification happens on both sides of the gap, so a corrupted or tampered artifact is rejected before it reaches a running cluster.*** The joint guidance [Deploying AI Systems Securely](https://www.cisa.gov/news-events/alerts/2024/04/15/joint-guidance-deploying-ai-systems-securely), published by the NSA, CISA, FBI, and partner agencies in April 2024, recommends validating AI artifacts with cryptographic signatures and checksums and hardening model weight storage. In an air gap, the transfer point is the only place a bad artifact can be stopped. For Bifrost, the [on-premise and air-gapped deployment guide](https://docs.getbifrost.ai/deployment-guides/enterprise/on-premise) covers mirroring the Enterprise image into an internal registry. The bundle also needs two datasheets, because Bifrost [reads pricing and model parameter data at startup](https://docs.getbifrost.ai/deployment-guides/how-to/airgapped) and will not start without them. ```bash # Connected staging host docker pull /bifrost: docker save /bifrost: > bifrost.tar curl -o pricing.json https://getbifrost.ai/datasheet curl -o model-parameters.json https://getbifrost.ai/datasheet/model-parameters sha256sum bifrost.tar pricing.json model-parameters.json > SHA256SUMS # Inside the air gap sha256sum -c SHA256SUMS && docker load < bifrost.tar docker tag /bifrost: registry.internal.example/bifrost: docker push registry.internal.example/bifrost: ``` Pin a version or image digest rather than `latest` so an earlier release stays available for rollback. ## Step 2: Serve Self-Hosted LLM Models Inside the Network A self-hosted LLM in an air-gapped network runs on an inference server such as vLLM, SGLang, or Ollama, loading open-weight model files from local storage. Each server exposes an HTTP endpoint inside the network, and Bifrost connects to those endpoints as providers, so applications never address model servers directly. Stage model weights like images: download, hash, transfer, and copy them to a model store the inference servers mount. Inside the network, set `HF_HUB_OFFLINE=1` for runtimes that use Hugging Face libraries; with that variable set, [no HTTP calls are made to the Hugging Face Hub](https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables) and only cached files are read. Bifrost supports vLLM, SGLang, and Ollama as providers and configures each with a [self-hosted endpoint URL instead of an API key](https://docs.getbifrost.ai/deployment-guides/config-json/providers). vLLM instances are model-specific, so each served model gets its own key entry: ```json { "providers": { "vllm": { "keys": [ { "name": "vllm-llama3-70b", "value": "", "models": ["llama-3-70b"], "weight": 1.0, "vllm_key_config": { "url": "http://vllm-llama3.models.svc:8000", "model_name": "meta-llama/Meta-Llama-3-70B-Instruct" } } ] } } } ``` SGLang (`sgl_key_config`) and Ollama (`ollama_key_config`) follow the same pattern with a URL. For internal servers that use a private CA, per-provider network settings accept a PEM-encoded CA certificate, which avoids turning off TLS verification. The [vLLM provider guide](https://www.getmaxim.ai/bifrost/guides/providers/vllm) walks through setup, and our comparison of [LLM gateways for vLLM, SGLang, and Ollama](https://www.getmaxim.ai/articles/top-4-llm-gateways-for-self-hosted-models-vllm-sglang-and-ollama-2026/) covers the gateway side. ## Step 3: Put a Gateway in Front of Your Private LLM Fleet A private LLM fleet needs one entry point that authenticates callers, routes requests to healthy model servers, and records what happened. [Bifrost, the open-source AI gateway,](https://www.getmaxim.ai/) fills that role inside the air gap: it exposes an OpenAI-compatible API, routes to self-hosted providers, and runs as a cluster for high availability. ![](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/2026/10/image-36.png) ***Figure 3: Applications integrate once with the gateway, and every dependency the gateway needs, including identity and storage, sits inside the same network.*** Two settings make Bifrost fully offline: point the pricing and model parameter URLs at the transferred files with `file://` values, and set the MCP catalog sync interval to `0` so no requests go to `getbifrost.ai`. Set `file://` values in `config.json`; the web UI accepts only HTTP and HTTPS URLs. ```json { "framework": { "pricing": { "pricing_url": "file:///opt/bifrost/pricing.json", "model_parameters_url": "file:///opt/bifrost/model-parameters.json", "mcp_library_sync_interval": 0 } } } ``` Install with the Helm chart from your internal chart repository, setting `image.repository` to the internal registry. Bifrost Enterprise requires PostgreSQL 16 or later for its config store, which also runs inside the network. For high availability, [Bifrost clustering](https://docs.getbifrost.ai/enterprise/clustering) uses a peer-to-peer design with gossip-based membership, supports discovery through Kubernetes, DNS, Consul, etcd, UDP broadcast, or mDNS, and recommends three or more nodes, with rolling updates that avoid downtime. Applications change one base URL and keep their existing OpenAI-compatible SDKs, as described on the [LLM Gateway page](https://www.getmaxim.ai/llm-gateway). Bifrost adds about [11 microseconds of overhead per request at 5,000 RPS](https://www.getmaxim.ai/resources/benchmarks), so the gateway does not become the bottleneck in front of GPU-bound inference. The same control plane runs in private clouds through [in-VPC deployments](https://docs.getbifrost.ai/enterprise/invpc-deployments) when full disconnection is not required. ## Step 4: Wire Identity, Access, and AI Guardrails Offline AI guardrails and identity controls in an air-gapped environment must run on components inside the network: a self-hosted identity provider, scoped virtual keys, and guardrail checks that need no cloud API. Bifrost supports each of these, so access and content policy are enforced at the gateway for every model call. ![](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/2026/10/image-37.png) ***Figure 4: Policy checks run before any GPU time is spent, and a failed model server triggers a fallback instead of an error to the caller.*** **Identity.** [User provisioning with OIDC and SCIM](https://docs.getbifrost.ai/enterprise/user-provisioning) connects Bifrost Enterprise to identity providers including self-hosted Keycloak and Zitadel, and a `trusted_networks` CIDR allowlist lets Bifrost reach an IdP on a private address range. Role-based access control governs what each user can do, and data access control limits which teams, keys, and logs each person can see. **Access to models.** Every inference call presents a [virtual key](https://docs.getbifrost.ai/features/governance/virtual-keys) that resolves to specific providers, models, budgets, and rate limits. The [security hardening checklist](https://docs.getbifrost.ai/enterprise/moving-from-oss/security-hardening) sets `enforce_auth_on_inference` so no anonymous path to a model exists, and raw provider credentials never leave Bifrost. An internal HashiCorp Vault can hold those credentials through secret management. The [AI Governance page](https://www.getmaxim.ai/ai-governance) shows how these controls fit together. **Guardrails.** Bifrost [guardrails](https://docs.getbifrost.ai/enterprise/guardrails) validate inputs and outputs for LLM requests and MCP tool calls using CEL rules and reusable profiles. Several profiles work with no outside connection: - **Secrets detection** uses embedded Gitleaks rules and [runs entirely inside Bifrost](https://docs.getbifrost.ai/enterprise/guardrails/secrets-detection). - **Custom regex** runs in-process and includes a built-in PII detection template. - **Prompt guardrails** use an LLM or decision model as a judge, and [self-hosted models can serve as judges](https://docs.getbifrost.ai/enterprise/guardrails/prompt-guardrails) because the judge request needs no structured-output support. - **Microsoft Presidio** points at a privately deployed Presidio Analyzer service. Cloud-hosted guardrail services need a path to their provider's cloud, so a fully disconnected network relies on the in-process and self-hosted options above. Redaction can apply to the live request, to stored logs only, or to both with reversible placeholders. The [AI Guardrails page](https://www.getmaxim.ai/ai-guardrails) and our post on [what Bifrost ships and integrates for guardrails](https://www.getmaxim.ai/bifrost/blog/guardrails-at-the-gateway-what-bifrost-ships-and-what-it-integrates) cover the full set. ## Step 5: Keep LLM Observability and Audit Logs Inside the Network LLM observability in an air-gapped environment means metrics, traces, and logs flow only to systems inside the network. [Bifrost as a self-hosted gateway](https://www.getmaxim.ai/) captures request logs asynchronously, exposes Prometheus metrics, exports OpenTelemetry traces to a self-hosted collector, and writes signed audit logs, with no data sent to an outside service. - **Request logs.** [Built-in observability](https://docs.getbifrost.ai/features/observability/default) records inputs, outputs, tokens, cost, and latency asynchronously. Content logging can be turned off globally, per virtual key, or per connector where prompts must not be stored. - **Metrics.** Prometheus scrapes the `/metrics` endpoint, and multi-node clusters can use a Push Gateway so no node is missed. - **Traces.** The [OpenTelemetry integration](https://docs.getbifrost.ai/features/observability/otel) sends traces over HTTP or gRPC to any OTLP collector, including a self-hosted one. - **Payload storage.** [Log exports](https://docs.getbifrost.ai/enterprise/log-exports) offload request and response payloads to object storage, and an endpoint override supports S3-compatible stores such as MinIO. - **Audit logs.** [Audit logs](https://docs.getbifrost.ai/enterprise/audit-logs) record administrative activity, such as role changes and key creation, with optional HMAC signing, configurable retention, and export as JSON, JSON Lines, or Syslog. Audit logs show who changed the gateway's configuration, while request logs show what each caller sent and received; regulated teams usually need both. The [AI Observability page](https://www.getmaxim.ai/ai-observability) shows how these signals reach existing monitoring tools. ## Step 6: Plan Updates for an Offline LLM Stack An offline LLM stack is updated in scheduled batches: new images, chart versions, model weights, and datasheets are staged, verified, transferred, and rolled out one component at a time. Because each change crosses the gap manually, the update process needs the same design attention as the initial deployment. 1. **Refresh reference data.** Bifrost re-reads the local pricing and catalog files on each sync tick (24 hours by default, one hour minimum), so replacing the files on disk updates the data without a restart. 2. **Upgrade the gateway.** Transfer the new pinned image, update the Helm release, and let the cluster roll nodes one at a time. 3. **Add or swap models.** Start the new inference server, add it as a provider key, shift traffic with key weights, then retire the old server. 4. **Configure fallbacks.** [Retries and fallbacks](https://docs.getbifrost.ai/features/retries-and-fallbacks) move a request to the next provider when the primary fails, so a server down for maintenance does not surface errors. 5. **Review the audit trail.** Confirm every configuration change in the window appears in the audit log under a named identity. Keep the previous bundle inside the network so a rollback needs no new transfer. For coding agents in the same environment, see how teams run [air-gapped Claude Code in regulated industries](https://www.getmaxim.ai/articles/air-gapped-claude-code-in-regulated-industries-with-bifrost/), and for public sector programs, the [government industry page](https://www.getmaxim.ai/industry-pages/government) outlines deployment expectations. ## Frequently Asked Questions ### What does air-gapped mean in cyber security? In cyber security, air-gapped means a system or network has no physical connection to untrusted networks such as the internet, and data moves only manually under human control. The isolation removes remote attack paths, so the main risk shifts to the transfer process: removable media, staging hosts, and the people who carry artifacts across. ### How does an air gap work? An air gap works by removing every automated network path between a protected environment and outside systems. Data moves only through a controlled process, such as removable media checked at a staging station or a one-way data diode. For an LLM stack, weights, images, and reference files are downloaded on a connected host, verified, and carried in. ### Can you run an LLM in an air-gapped environment? Yes. Open-weight models run on local inference servers such as vLLM, SGLang, or Ollama, loading weights from internal storage with no internet access. A production deployment also needs a gateway for authentication, routing, guardrails, and logging. Bifrost runs offline once its datasheets load from local files and its MCP catalog sync is disabled. ### What is the difference between an on-premise LLM and an air-gapped LLM? An on-premise LLM runs on hardware the organization controls, but its network may still allow outbound internet access for updates or telemetry. An air-gapped LLM runs in a network with no automated outside connection at all. Every air-gapped deployment is on-premise or in an isolated private environment, but many on-premise deployments are not air-gapped. ### Does Bifrost send any data outside an air-gapped network? No. Bifrost runs inside the customer's infrastructure and sends no prompts, responses, files, or telemetry to Maxim AI. By default it contacts getbifrost.ai for two things only, the pricing datasheets and the MCP server catalog, and both can be replaced with local files or disabled in config.json for a fully disconnected deployment. ### Do you need Bifrost Enterprise for an air-gapped deployment? The open-source gateway can run offline using the same local datasheet settings. [Bifrost Enterprise](https://www.getmaxim.ai/enterprise) adds the controls regulated air-gapped environments usually require: clustering for high availability, SSO with OIDC and SCIM, RBAC, guardrails, signed audit logs, log exports, and secret management. Enterprise images come from a private registry and are mirrored into the internal registry. ## Start Building Your Air-Gapped LLM Stack with Bifrost An air-gapped environment changes how an LLM stack is assembled: every artifact crosses a verified transfer, and every control, from identity to guardrails to audit logs, has to run on the inside. Bifrost gives that stack one self-hosted control plane for routing, access, guardrails, and observability across vLLM, SGLang, and Ollama. For deployment patterns, see the [Bifrost resources on on-prem and VPC deployment](https://www.getmaxim.ai/resources/enterprise-deployment) and our guide to the [best self-hosted AI gateway](https://www.getmaxim.ai/articles/best-self-hosted-ai-gateway-in-2026/). To plan a private LLM deployment for your network, [book a demo](https://www.getmaxim.ai/book-a-demo) with the Bifrost team. ## Read next [![LLM Routing: How Routing Decisions Get Made](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w720/2026/10/llm-routing-how-routing-decisions-get-made-bifrost-1920x1080-1.png) LLM routing is the per-request choice of which provider, model, and API key serve an LLM call. This guide walks through the signals a router evaluates, the precedence order Bifrost applies, what happens when a route fails, and how to debug a routing decision from its logs.](https://www.getmaxim.ai/articles/llm-routing-decisions/) [![GPT Realtime API in Production: Routing, Governing and Billing Voice Sessions](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w720/2026/10/gpt-realtime-api-in-production-routing-governing-and-billing-bifrost-1920x1080.png) GPT Realtime is OpenAI's speech-to-speech model family on the Realtime API, with GPT-Live as its duration-billed sibling. This guide covers connections, client secrets and billing, then shows how Bifrost routes, governs and prices every voice session.](https://www.getmaxim.ai/articles/gpt-realtime-api-production/) [![API Key Management for LLM Apps: How to Manage Client Access](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w720/2026/10/how-to-enforce-ai-budgets-per-team-project-and-key-bifrost-1920x1080--1-.png) API key management for LLM applications means issuing every app, service, and environment its own revocable credential instead of sharing raw provider keys. This guide shows how to do it with Bifrost virtual keys, covering scoping, rate limits, rotation, revocation, and log attribution.](https://www.getmaxim.ai/articles/api-key-management-for-llm-applications/) ## More in AI Gateway [![GPT Realtime API in Production: Routing, Governing and Billing Voice Sessions](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w720/2026/10/gpt-realtime-api-in-production-routing-governing-and-billing-bifrost-1920x1080.png) GPT Realtime is OpenAI's speech-to-speech model family on the Realtime API, with GPT-Live as its duration-billed sibling. This guide covers connections, client secrets and billing, then shows how Bifrost routes, governs and prices every voice session.](https://www.getmaxim.ai/articles/gpt-realtime-api-production/) [![LLM Routing: How Routing Decisions Get Made](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w720/2026/10/llm-routing-how-routing-decisions-get-made-bifrost-1920x1080-1.png) LLM routing is the per-request choice of which provider, model, and API key serve an LLM call. This guide walks through the signals a router evaluates, the precedence order Bifrost applies, what happens when a route fails, and how to debug a routing decision from its logs.](https://www.getmaxim.ai/articles/llm-routing-decisions/) ```json { "@context": "https://schema.org", "@type": "Article", "publisher": { "@type": "Organization", "name": "Maxim Articles", "url": "https://www.getmaxim.ai/articles/", "logo": { "@type": "ImageObject", "url": "https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w256h256/2025/08/thumbnail.png", "width": 60, "height": 60 } }, "author": { "@type": "Person", "name": "Kamya Shah", "image": { "@type": "ImageObject", "url": "https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/2025/09/WhatsApp-Image-2025-08-29-at-17.40.40-1.jpeg", "width": 1200, "height": 1600 }, "url": "https://www.getmaxim.ai/articles/author/kamya/", "sameAs": [] }, "headline": "How to Deploy an LLM Stack in an Air-Gapped Environment", "url": "https://www.getmaxim.ai/articles/llm-stack-air-gapped-environment/", "datePublished": "2026-10-29T18:52:00.000Z", "dateModified": "2026-10-30T18:55:24.000Z", "image": { "@type": "ImageObject", "url": "https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w1200/2026/10/how-to-deploy-an-llm-stack-in-an-air-gapped-environment-bifrost-1920x1080.png", "width": 1200, "height": 675 }, "keywords": "AI Gateway", "description": "An air-gapped environment has no automated connection to the internet, so every model, image, and update arrives by controlled transfer. This guide walks through each layer of an offline LLM stack, from model serving with vLLM and SGLang to gateway policy, guardrails, identity, and audit logs.\n", "mainEntityOfPage": "https://www.getmaxim.ai/articles/llm-stack-air-gapped-environment/" } ```