--- description: "API key management for LLM apps: give each service and environment its own scoped, revocable virtual key while provider keys stay inside the gateway." title: "API Key Management for LLM Apps: How to Manage Client Access" image: https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w1200/2026/10/how-to-enforce-ai-budgets-per-team-project-and-key-bifrost-1920x1080--1-.png --- Try Bifrost Enterprise free for 14 days. [Request access](https://www.getmaxim.ai/#enterprise-trial) **TL;DR** - API key management for LLM applications works best when provider API keys live only in the gateway and every service and environment gets its own client credential. - Bifrost virtual keys are those client credentials, each with its own provider and model scope, rate limits, active status, optional expiry, and metadata. - Clients authenticate with a virtual key in the `x-bf-vk`, `Authorization`, `x-api-key`, `x-goog-api-key`, or `api-key` header, so existing SDKs only need a new base URL. - A virtual key can be rotated with a grace window, deactivated instantly, or set to expire without touching the provider keys other clients depend on. - Every request is logged against the virtual key that made it, with per-key metadata such as service, environment, and owner copied onto each entry. A common first step with LLMs is pasting one OpenAI or Anthropic key into every service that needs it, and that shared key becomes the weakest point of API key management. [Bifrost](https://www.getmaxim.ai/bifrost), the [open-source AI gateway on GitHub](https://github.com/maximhq/bifrost) built by Maxim AI, holds provider keys centrally and issues each client its own virtual key. This guide covers how clients authenticate, how to scope, rate-limit, rotate, and revoke each credential, and how to attribute every request, with a worked example of three services across dev, staging, and production. ## What Is API Key Management for LLM Applications? API key management for LLM applications is the practice of issuing, scoping, rotating, revoking, and auditing the credentials that applications use to call model providers. Done well, every caller has its own credential, provider secrets stay in one place, and any credential can be cut off without affecting other callers. Without it, one provider key spreads from a chatbot into batch jobs, agents, and staging. That sprawl produces four failures: - **No attribution.** One key serves many callers, so nobody can say which service caused a cost spike. - **No targeted revocation.** Revoking a leaked shared key breaks every service using it, so revocation gets delayed. - **No per-caller limits.** One runaway batch job consumes the provider quota the customer-facing chatbot depends on. - **Wide blast radius.** A leaked key opens the whole provider account and every model it can reach. | Concern | Shared provider keys | Per-client virtual keys through a gateway | | --- | --- | --- | | Who holds the provider secret | Every service, pipeline, and developer | Only the gateway | | Attribution | One key, many callers | One credential per caller, logged per request | | Revoking one caller | Breaks every caller on that key | Disables only that caller | | Model and provider scope | Whatever the provider key allows | Explicit allowlist per client | | Rotation impact | Coordinated redeploy of every caller | One caller updates, with an optional grace window | An AI gateway enforces this split because every request already passes through it ([AI gateway vs API gateway for LLM traffic](https://www.getmaxim.ai/articles/ai-gateway-vs-api-gateway-5-options-compared-for-llm-traffic-in-2026/) compares the options). ## Why Shared LLM API Keys Become a Security Problem A shared LLM API key is a high-value secret with wide distribution. The more places it lives, the more likely it leaks, and a leak exposes the whole provider account rather than one application's slice of it. GitGuardian's [State of Secrets Sprawl 2026](https://gitguardian.com/state-of-secrets-sprawl-report-2026) report counted 28,649,024 new secrets in public GitHub commits during 2025, with secrets for AI-related services up 81% year over year. The same report found that 64% of valid secrets detected in 2022 were still active four years later. Revocation is rare because shared keys make it expensive: rotating a key used by eight services means eight coordinated deploys. The [OWASP Secrets Management Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html) warns that shared secrets make the source of a leak hard to identify, and the [OWASP API Security Top 10](https://owasp.org/API-Security/editions/2023/en/0xa2-broken-authentication/) says API keys belong to API client authentication, not user authentication. Per-client virtual keys follow both rules, which is why platform teams treat [virtual keys as the unit of LLM access governance](https://www.getmaxim.ai/articles/virtual-keys-explained-how-platform-teams-govern-llm-access-across-100-engineers/). ## Separate Provider Keys From Client Credentials With Virtual Keys The core design is two layers of keys. Provider API keys for OpenAI, Anthropic, Bedrock, and others are configured once in the gateway. Client credentials, which Bifrost calls virtual keys, are issued per application, service, and environment. Clients only ever see their virtual key; the gateway maps it to provider keys at request time. [Virtual keys](https://docs.getbifrost.ai/features/governance/virtual-keys) are the primary governance entity in the [Bifrost gateway](https://www.getmaxim.ai/bifrost). Each carries provider and model filtering, budgets, rate limits, restrictions to specific provider keys, an active or inactive status, and an optional owner such as a team or customer. New values use an `sk-bf-` prefix, which separates them from provider keys in code review and secret scanners. ![](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/2026/10/image-28.png) ***Figure 1: A leaked client credential exposes one scoped virtual key, never the provider account behind it.*** Provider keys take a `name`, `value`, `models` list, and `weight`, and the [provider configuration reference](https://docs.getbifrost.ai/deployment-guides/config-json/providers) uses `env.` references so secrets never sit in `config.json`. On Bifrost Enterprise, [vault-backed secret management](https://docs.getbifrost.ai/enterprise/secret-management) adds `vault.` references to AWS Secrets Manager, GCP Secret Manager, HashiCorp Vault, or Azure Key Vault for both provider keys and virtual key values. ## API Key Authentication: How Clients Present a Virtual Key Clients authenticate to Bifrost by sending their virtual key in a request header. Bifrost accepts the dedicated `x-bf-vk` header plus the header each major SDK already sends, so existing client code authenticates without modification. Setting `enforce_auth_on_inference` makes a valid virtual key mandatory on every inference request. | Header | Example value | Typical client | | --- | --- | --- | | `x-bf-vk` | `sk-bf-...` | Any HTTP client; the only header for older keys without the `sk-bf-` prefix | | `Authorization` | `Bearer sk-bf-...` | OpenAI SDK and OpenAI-compatible clients | | `x-api-key` | `sk-bf-...` | Anthropic SDK | | `x-goog-api-key` | `sk-bf-...` | Google Gemini SDK | | `api-key` | `sk-bf-...` | Azure OpenAI clients | Governance is optional by default, so set `client.enforce_auth_on_inference` to `true` and keyless requests get `401 virtual_key_required`. If [gateway authentication](https://docs.getbifrost.ai/quickstart/gateway/setting-up-auth) also covers inference, `Authorization` is consumed by that layer and clients use `x-bf-vk`. Enterprise SSO tokens are the path for people, not services; see [LLM access control with RBAC, SSO, and virtual keys](https://www.getmaxim.ai/articles/llm-access-control-rbac-sso-and-virtual-keys-for-ai-traffic/). ![](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/2026/10/image-29.png) ***Figure 2: Every rejection names the check that failed, so a 403 or 429 tells the client team exactly what to fix.*** As Figure 2 shows, each failed check returns a typed error (`access_blocked`, `provider_blocked`, `model_blocked`, `token_limited`, `request_limited`, or `budget_exceeded`) in a top-level `type` field that clients can branch on. ## Worked Example: Three Services Across Dev, Staging, and Production This example covers a support chatbot, a nightly batch summarizer, and an internal coding agent, each in dev, staging, and production. Nine callers become nine virtual keys, while the provider keys never leave the gateway. | Virtual key | Allowed models | Provider keys (`key_ids`) | Rate limit | | --- | --- | --- | --- | | `chatbot-prod` | OpenAI `gpt-4o-mini` | `openai-prod-1`, `openai-prod-2` | 600 requests/min | | `summarizer-prod` | OpenAI `gpt-4o-mini`, `gpt-4o` | `openai-prod-1`, `openai-prod-2` | 2M tokens/hour | | `agent-prod` | Anthropic `claude-sonnet-4-5` | `anthropic-prod` | 120 requests/min | | `*-staging`, `*-dev` (6 keys) | Same as prod | Non-production keys only | Lower limits; dev keys expire | The production slice of `config.json` for the chatbot looks like this. Each virtual key's `key_ids` references provider key `id` values, and every secret is an `env.` reference: ```json { "client": { "enforce_auth_on_inference": true, "vk_rotation_cooldown": "24h" }, "providers": { "openai": { "keys": [ { "id": "openai-nonprod", "name": "openai-nonprod", "value": "env.OPENAI_KEY_NONPROD", "models": ["*"], "weight": 1.0 }, { "id": "openai-prod-1", "name": "openai-prod-1", "value": "env.OPENAI_KEY_PROD_1", "models": ["*"], "weight": 0.5 }, { "id": "openai-prod-2", "name": "openai-prod-2", "value": "env.OPENAI_KEY_PROD_2", "models": ["*"], "weight": 0.5 } ] } }, "governance": { "virtual_keys": [ { "id": "vk-chatbot-prod", "name": "chatbot-prod", "value": "env.VK_CHATBOT_PROD", "is_active": true, "metadata": { "service": "support-chatbot", "env": "prod", "owner": "cx-platform@example.com" }, "provider_configs": [ { "provider": "openai", "allowed_models": ["gpt-4o-mini"], "key_ids": ["openai-prod-1", "openai-prod-2"] } ], "rate_limit_id": "rl-chatbot-prod" } ], "rate_limits": [ { "id": "rl-chatbot-prod", "request_max_limit": 600, "request_reset_duration": "1m" } ] } } ``` The other keys follow the same pattern, and keys can also be created at runtime with `POST /api/governance/virtual-keys` or the dashboard, as shown in [how to set up virtual keys for LLM access control](https://www.getmaxim.ai/articles/how-to-set-up-virtual-keys-for-llm-access-control/). ![](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/2026/10/image-31.png) ***Figure 3: Pinning provider keys per environment means a dev credential can never spend against the production provider account.*** Figure 3 shows the boundary `key_ids` creates: dev and staging keys list only non-production provider keys, so a misplaced dev key cannot draw on production quota. ## Per-Client Scoping and Rate Limits Per-client scoping means each virtual key reaches only the providers, models, and provider keys it was granted. Virtual keys are deny-by-default for providers: a provider absent from `provider_configs` is blocked, and an empty `provider_configs` blocks everything. Rate limits then apply to each key independently of every other client. The scoping fields on each provider config are: - `**allowed_models**`: the model allowlist; `["*"]` allows all, `[]` allows none, and `regex:` entries are RE2 patterns. - **`blacklisted_models`**: a denylist that always wins over the allowlist. - **`key_ids`**: the provider keys this client may use; an empty or omitted list denies all. - **`allow_all_providers`** (on the key itself): grants every configured provider, while listed providers keep their own restrictions. `GET /v1/models` with a virtual key lists only the providers it may reach. Mapping allowlists to roles and teams is a separate concern, shown in [restricting GPT-5 access to one team](https://www.getmaxim.ai/articles/how-to-restrict-gpt-5-access-to-one-team-with-virtual-keys/). Rate limits use `token_max_limit`/`token_reset_duration` and `request_max_limit`/`request_reset_duration`, and [budgets and limits](https://docs.getbifrost.ai/features/governance/budget-and-limits) are checked at every level that has them. Per-client limits keep the batch summarizer from consuming the chatbot's capacity; dollar caps are covered in [LLM API rate limiting with virtual keys and budgets](https://www.getmaxim.ai/articles/llm-api-rate-limiting-with-virtual-keys-and-budgets/). ## API Key Rotation, Deactivation, and Revocation API key rotation replaces a credential's value on a schedule or after a suspected leak, while deactivation and expiry stop a credential from working without replacing it. Bifrost treats these as separate controls on each virtual key, and none of them requires changing a provider key or redeploying any other client. ![](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/2026/10/image-32.png) ***Figure 4: Rotation, deactivation, and expiry are separate controls, and only deletion is irreversible.*** Each control in Figure 4 maps to a field or endpoint on the [virtual key governance API](https://docs.getbifrost.ai/features/governance/virtual-keys): - **Rotate.** `POST /api/governance/virtual-keys/{vk_id}/rotate` issues a new value for one key, and `POST /api/governance/virtual-keys/rotate` rotates several. With `client.vk_rotation_cooldown` set (for example `"24h"`), the old value keeps working until the window ends, so the client redeploys without downtime. At `0`, the old value stops immediately, which suits incident response. - **Deactivate.** Setting `is_active` to `false` blocks the key at once with `403 access_blocked`, keeping its configuration and history; reactivating restores access. - **Expire.** `expires_at` takes an RFC3339 timestamp; afterward, inference and MCP tool calls fail closed while the key stays visible for auditing. - **Delete.** `delete_after_expire`, or `client.delete_expired_virtual_keys` workspace-wide, lets a daily job remove expired keys. Deletion cannot be undone. On Bifrost Enterprise, [access profiles](https://docs.getbifrost.ai/enterprise/access-profiles) can rotate user-owned keys on a schedule with `auto_rotation_interval`. Keys owned by teams, business units, or customers are not rotated automatically, so service keys still need a rotation runbook. Provider keys rotate without client involvement: add the new key, shift traffic with [weighted key load balancing](https://docs.getbifrost.ai/features/keys-management), then disable the old one. With Enterprise secret management, updating the vault value and calling `POST /api/vault/flush-cache` is enough. ## Request Attribution in Logs and Audit Trails Attribution means every LLM request traces back to the client credential that made it. Bifrost logs each request with provider, model, tokens, cost, latency, and the provider key that served it, and ties the entry to the virtual key used. Per-key metadata turns that into service, environment, and owner fields with no client-side changes. The [built-in request logging](https://docs.getbifrost.ai/features/observability/default) runs asynchronously, so it adds no latency to the request path. Three attribution mechanisms stack on top of it: - **Virtual key metadata.** Up to 50 key/value entries per key are copied onto every log entry made with it. Values are snapshotted at request time and override caller-sent headers with the same name, so a client cannot misattribute its own traffic. - **Logging headers.** Any header prefixed `x-bf-lh-` is captured into log metadata, which suits request-level context such as a tenant ID. - **Provider key fields.** `selected_key_id`, `selected_key_name`, and `attempt_trail` show which provider key served each request. Logs filter by metadata (for example `GET /api/logs?metadata_env=prod`), which also drives spend breakdowns, as described in [how to monitor LLM API costs in production](https://www.getmaxim.ai/articles/how-to-monitor-llm-api-costs-in-production/). Bifrost Enterprise [audit logs](https://docs.getbifrost.ai/enterprise/audit-logs) separately record administrative activity (initiator, action, target, outcome) with optional HMAC signing, answering who created, rotated, or deactivated a key. See [AI audit trail controls for LLM traffic](https://www.getmaxim.ai/articles/ai-audit-trail-controls-and-audit-logs-for-llm-traffic/). ## Drop-In SDK Replacement: Switching Clients to Virtual Keys Moving an application to per-client credentials changes two values: the SDK base URL, which points to Bifrost, and the API key, which becomes the application's virtual key. The SDK sends the key in the header it already uses, and the rest of the code stays the same. The [drop-in replacement](https://docs.getbifrost.ai/features/drop-in-replacement) pattern for the OpenAI and Anthropic Python SDKs: ```python import os import openai, anthropic chat_client = openai.OpenAI( base_url="", api_key=os.environ["BIFROST_VK"], # chatbot-prod virtual key ) agent_client = anthropic.Anthropic( base_url="", api_key=os.environ["BIFROST_VK"], # agent-prod virtual key ) ``` One container image then runs in every environment with a different credential. The same approach covers the [OpenAI SDK integration](https://docs.getbifrost.ai/integrations/openai-sdk/overview), Google GenAI, LangChain, and coding agents such as [Claude Code routed through Bifrost](https://docs.getbifrost.ai/cli-agents/claude-code). To migrate without outages, switch each service to its virtual key with `enforce_auth_on_inference` off, confirm in the logs that its traffic arrives under that key, then turn enforcement on and revoke the provider keys services used to hold. ## API Key Security Best Practices for an LLM Gateway API key security for LLM traffic comes down to a short list: one credential per caller, provider secrets held only by the gateway, least-privilege scope, enforced authentication, routine rotation, and tamper-resistant records. This checklist applies those controls to a Bifrost deployment. - **Issue one virtual key per service per environment** and enforce authentication on inference. - **Keep provider secrets out of files** with `env.` or `vault.` references, and leave **Allow Direct API Keys** off (its default) so callers cannot bypass virtual keys. - **Scope narrowly.** The [security hardening guide](https://docs.getbifrost.ai/enterprise/moving-from-oss/security-hardening) recommends one provider config, one model alias, and one budget envelope per key for CI/CD and internal services. - **Pin environments** with `key_ids`, and set `expires_at` on dev, contractor, and experiment keys. - **Scan prompts for leaked keys** with the Enterprise [secrets detection guardrail](https://docs.getbifrost.ai/enterprise/guardrails/secrets-detection). People need different controls: SSO, role-based model allowlists, and team budget hierarchies are covered in the [Bifrost governance overview](https://www.getmaxim.ai/bifrost/resources/governance) and in [LLM budget management with hierarchical spend controls](https://www.getmaxim.ai/articles/llm-budget-management-virtual-keys-and-hierarchical-spend-controls/). ## Frequently Asked Questions ### What is API key management? API key management is the process of creating, distributing, scoping, rotating, revoking, and auditing the API keys clients use to call a service. For LLM applications, it means keeping provider keys in one place, such as an AI gateway, and giving every application and environment its own credential so access can be attributed and revoked per caller. ### How do I manage my API keys for multiple LLM applications? Put an AI gateway between applications and providers. Configure provider keys once in the gateway, then issue each application and environment its own [scoped virtual key for LLM access](https://www.getmaxim.ai/articles/virtual-keys-explained-how-platform-teams-govern-llm-access-across-100-engineers/) with only the providers, models, and limits it needs. Applications point their SDK base URL at the gateway, so provider keys never appear in application code or CI secrets. ### What can someone do with my API key? Anyone holding a raw LLM provider key can call any model it allows, spend against the account, and exhaust the rate limits production services depend on. A leaked [Bifrost AI gateway](https://www.getmaxim.ai/bifrost) virtual key is contained: it reaches only its allowed models and provider keys, is capped by its own limits, and can be deactivated instantly. ### What is an API key for LLM? An API key for an LLM is a secret token that authenticates calls to a model provider's API, such as OpenAI or Anthropic, and links usage to a billing account. Through Bifrost, applications use a virtual key instead, which starts with `sk-bf-` and maps to provider keys stored inside the gateway, so the provider secret never reaches the application. ### How often should API keys be rotated? Rotate client credentials on the schedule your security policy sets, commonly every 30 to 90 days, and immediately after any suspected exposure. Bifrost supports per-key and bulk rotation, a grace window through `vk_rotation_cooldown`, and scheduled rotation of user-owned keys through Enterprise access profiles. ### How to store API keys securely? Store provider API keys only in the gateway, referenced from environment variables or a secret manager rather than written into configuration files or source code. Store each application's virtual key in its own secret store or deployment environment, and never commit either key type to a repository. ## Start Managing LLM Client Access With Bifrost Per-client credentials are the foundation of API key management for LLM applications: provider keys stay in one place, and every service and environment gets a credential that is scoped, limited, rotated, revoked, and attributed on its own. Bifrost delivers that model with virtual keys, drop-in SDK compatibility, and [11 microseconds of overhead at 5,000 RPS](https://www.getmaxim.ai/bifrost/resources/benchmarks). To see how it fits your services and environments, [book a demo with the Bifrost team](https://getmaxim.ai/bifrost/book-a-demo). ## Read next [![Claude Agent SDK Through a Gateway: Routing, Budgets, MCP](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w720/2026/10/claude-agent-sdk-through-a-gateway-routing-budgets-mcp-bifrost-1920x1080.png) The Claude Agent SDK packages Claude Code's agent loop, tools, and context management as a Python and TypeScript library. This guide shows how to route Claude Agent SDK traffic through Bifrost for provider routing and failover, per-tenant budgets, and governed MCP tool access.](https://www.getmaxim.ai/articles/claude-agent-sdk-through-a-gateway-routing-budgets-mcp/) [![How to Enforce AI Budgets Per Team, Project and Key](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w720/2026/10/how-to-enforce-ai-budgets-per-team-project-and-key-bifrost-1920x1080.png) An AI budget is a hard spending ceiling enforced in the LLM request path, not a monthly report. This guide covers budget hierarchies, project mapping, reset windows, rate limits, exhausted-budget behavior, and a worked $10,000 example in Bifrost.](https://www.getmaxim.ai/articles/enforce-ai-budgets/) [![OpenAI Responses API vs Chat Completions vs Anthropic Messages](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w720/2026/10/openai-responses-api-vs-chat-completions-vs-anthropic-messag-bifrost-1920x1080.png) The OpenAI Responses API is OpenAI's stateful, item-based interface for agentic apps, while Chat Completions and Anthropic Messages remain stateless message APIs. This guide compares the three on state, tools, reasoning, and retention, and shows how Bifrost serves all three formats.](https://www.getmaxim.ai/articles/openai-responses-api-vs-chat-completions-vs-anthropic-messages/) ## More in AI Gateway [![Claude Agent SDK Through a Gateway: Routing, Budgets, MCP](https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w720/2026/10/claude-agent-sdk-through-a-gateway-routing-budgets-mcp-bifrost-1920x1080.png) The Claude Agent SDK packages Claude Code's agent loop, tools, and context management as a Python and TypeScript library. This guide shows how to route Claude Agent SDK traffic through Bifrost for provider routing and failover, per-tenant budgets, and governed MCP tool access.](https://www.getmaxim.ai/articles/claude-agent-sdk-through-a-gateway-routing-budgets-mcp/) ```json { "@context": "https://schema.org", "@type": "Article", "publisher": { "@type": "Organization", "name": "Maxim Articles", "url": "https://www.getmaxim.ai/articles/", "logo": { "@type": "ImageObject", "url": "https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w256h256/2025/08/thumbnail.png", "width": 60, "height": 60 } }, "author": { "@type": "Person", "name": "Kamya Shah", "image": { "@type": "ImageObject", "url": "https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/2025/09/WhatsApp-Image-2025-08-29-at-17.40.40-1.jpeg", "width": 1200, "height": 1600 }, "url": "https://www.getmaxim.ai/articles/author/kamya/", "sameAs": [] }, "headline": "API Key Management for LLM Apps: How to Manage Client Access", "url": "https://www.getmaxim.ai/articles/api-key-management-for-llm-applications/", "datePublished": "2026-10-22T22:08:00.000Z", "dateModified": "2026-10-23T22:10:45.000Z", "image": { "@type": "ImageObject", "url": "https://storage.ghost.io/c/84/03/8403f2f6-141c-411a-8f55-a32d4291533e/content/images/size/w1200/2026/10/how-to-enforce-ai-budgets-per-team-project-and-key-bifrost-1920x1080--1-.png", "width": 1200, "height": 675 }, "keywords": "AI Gateway", "description": "API key management for LLM applications means issuing every app, service, and environment its own revocable credential instead of sharing raw provider keys. This guide shows how to do it with Bifrost virtual keys, covering scoping, rate limits, rotation, revocation, and log attribution.", "mainEntityOfPage": "https://www.getmaxim.ai/articles/api-key-management-for-llm-applications/" } ```