Policy-Based Governance at the Gateway: One Control Plane for Every AI Call

Policy-based AI governance is the practice of declaring access, spend, routing, and safety rules once and enforcing them at runtime on every model request, instead of writing them down and trusting each application to comply. Most enterprises now run several providers across production services, internal tools, IDE assistants, and agents, and the rules meant to control that traffic live in wiki pages rather than in the request path. Bifrost, the open-source AI gateway built in Go by Maxim AI, is the best overall choice for enterprise teams that need one control plane for every AI call across providers, models, agents, and tools. This post covers how policy is modeled, evaluated, and enforced at the gateway, and what that architecture gives platform and security teams that application-level controls cannot.

What Policy-Based AI Governance Means at the Gateway

Policy-based AI governance at the gateway means access, spend, throughput, routing, and content rules are declared centrally and evaluated inside the request path. Every model call passes through the same enforcement point, so a rule written once applies to every application, provider, agent, and user without a change to application code.

A complete gateway policy answers six questions about a request before it reaches a provider:

• Identity: which consumer, team, or customer is making this call?
• Access: which providers, models, and provider API keys may they use?
• Spend: which budgets does this call draw down, and what happens when one is exhausted?
• Throughput: how many requests and tokens are permitted in the current window?
• Content: which safety rules apply to the prompt and to the response?
• Evidence: what record of the decision is retained, and for how long?

Because the Bifrost AI gateway sits in the data path between applications and providers, all six are resolved before the request leaves the network. Governance becomes part of the request pipeline rather than a reporting layer assembled after the fact.

Why Policy at the Application Layer Does Not Hold

Application-level governance fails because policy is copied rather than centralized. Each service holds its own provider keys, its own approved-model list, and its own retry logic. Nothing evaluates those choices at runtime, and nothing produces a single record of who called which model with what data.

The failure modes are consistent across multi-provider environments:

• Credential sprawl: raw provider keys sit in environment variables across dozens of services and cannot be scoped or revoked individually.
• Untracked spend: cost lands on a provider invoice, attributed to an account rather than to a team, product, or customer.
• Policy drift: an approved-model list in a document does not stop a new service from calling an unapproved model.
• Blind spots: coding agents, IDE assistants, and desktop chat apps never pass through the platform team's code path.
• Unusable evidence: logs are per-application, in different formats, with different retention windows.

The visibility gap is measurable. A Cloud Security Alliance survey of 418 IT and security professionals, published in April 2026, found that 82% of organizations discovered AI agents running in their environments that had never been provisioned, and 65% reported an AI agent related incident in the previous 12 months, even though 68% believed their visibility into agents was strong. Governance that depends on each team opting in produces exactly this result, which is why the governance model has to live at the infrastructure layer.

The Bifrost Policy Model: Virtual Keys, Hierarchy, and Access Profiles

In the open-source Bifrost gateway, policy attaches to a virtual key. A virtual key is the credential an application or user presents, and it carries the allowed providers and models, the budget, the rate limits, and the MCP tool access for that consumer. Real provider keys stay inside the gateway and are never handed to an application.

Virtual keys are the primary governance entity, and the model has a few properties worth knowing before designing policy:

• Access is deny-by-default: a virtual key with no provider configuration blocks all providers, so permissions are granted explicitly rather than assumed.
• Attachment is exclusive: a virtual key belongs to one team, or one customer, or neither, which keeps cost and access hierarchies unambiguous.
• Keys can be pinned to specific provider API keys, so a key issued for a development environment cannot spend production quota.
• Status is a switch: marking a virtual key inactive stops its access immediately, without a redeploy.
• Authentication uses familiar headers, including the OpenAI, Anthropic, and Azure styles, so existing clients present a virtual key the way they present a provider key.

Issuing keys by hand does not scale past a few teams. Access profiles solve that: a profile is a reusable policy template (provider list, model allow-list, budgets, rate limits, MCP tool access) that Bifrost copies per user and materializes as an auto-issued virtual key. Mark a profile as a role default and users in that role are provisioned automatically, each with independent counters. Profile-managed keys are write-protected, so a user cannot edit around their own policy, and every profile change is recorded with a snapshot history.

Enforcing Spend and Rate Policy on Every AI Call

Bifrost enforces spend policy hierarchically. Budgets exist independently at the customer, team, virtual key, and provider-config levels, and every applicable budget is checked before a request proceeds. Any single budget without remaining balance blocks the call, so overspend is prevented at request time rather than discovered at invoice time.

The hierarchical budget structure supports the patterns platform teams actually need:

• Customer level: cap total spend for an external tenant across all of their teams and keys.
• Team level: give each internal team a monthly ceiling that its keys draw from.
• Virtual key level: bound a single application, environment, or agent, with token and request rate limits alongside the budget.
• Provider config level: cap spend and throughput per provider within one key, so an expensive frontier model cannot absorb the whole allocation.

Reset windows run from one minute to one year, and calendar-aligned budgets reset at UTC day, week, month, or year boundaries instead of on a rolling window, which matches how finance teams close periods. Cost is computed from live model pricing and token usage, with cached and batch requests priced accordingly, so attribution reflects what was actually consumed.

Expression-Based Policy: CEL Rules for Routing and Guardrails

Static allow-lists cannot express rules like "send batch embedding traffic to a cheaper provider" or "inspect any prompt from the support application for credentials." Bifrost handles these with expression-based policy, using CEL, the open Common Expression Language maintained by Google, to evaluate conditions against the live request.

Routing rules run before governance provider selection and can override it. Rules are organized by scope with first-match-wins evaluation: virtual key scope first, then team, then customer, then global, with priority ordering inside each scope. A matched rule can chain, making its resolved provider and model the new context for another pass. If no rule matches, the incoming provider and model are used unchanged.

Guardrails use the same expression model with two objects:

• Rules define when and what to evaluate, written in CEL, and can apply to inputs, outputs, or both.
• Profiles define how content is evaluated, covering native Secrets Detection (Gitleaks-backed) and Custom Regex (including a PII template), plus Microsoft Presidio, Azure AI Language PII, AWS Bedrock Guardrails, Azure Content Safety, Google Model Armor, CrowdStrike AIDR, GraySwan, and Patronus AI.

Profiles are reusable across rules, and a single rule can call several profiles for layered checks. Validation is dual-stage, so a prompt is inspected before it reaches a model and a response is inspected before it returns to the caller.

Compliance Policy: RBAC, Audit Logs, and Deployment Control

Evidence is a policy dimension of its own. Frameworks such as the NIST AI Risk Management Framework and the phased obligations under the EU AI Act expect continuous, demonstrable control rather than periodic attestation, which means the enforcement point and the record of enforcement need to be the same system.

Bifrost separates operator permissions from operator visibility:

• Role-based access control governs which operations a user can perform, using the Admin, Developer, and Viewer system roles or custom roles, with assignment driven by OIDC groups and claims.
• Data access control governs which rows a user can see, scoping virtual keys, prompts, and routing rules to the teams a role is entitled to view.
• Audit logs record administrative activity with HMAC-signed entries, configurable retention, filtering by action, outcome, and date, export as JSON, JSON Lines, or Syslog, and continuous archival to S3 or GCS for long-term retention.

For regulated industries, the deployment target matters as much as the policy. Bifrost Enterprise runs in-VPC, on-prem, and in air-gapped environments, so policy enforcement and the audit trail stay inside the same boundary as the data they describe.

One Control Plane for MCP Tools and Endpoint AI

Model calls are only part of the traffic. Agents reach tools through MCP servers, and those tool calls read files, query internal APIs, and take actions. Policy that stops at the model boundary leaves the tool boundary open, which is why tool scope belongs in the same control plane.

MCP tool groups are named bundles of tools drawn from one or more MCP servers, attachable across six dimensions: virtual keys, teams, customers, users, providers, and API keys. At request time, Bifrost resolves the union of tools from every matching group against an in-process index, so an agent sees only the tools its policy allows. Used as an MCP gateway, Bifrost applies the same identity, budget, and audit model to tool calls that it applies to model calls.

The remaining gap is traffic that never points at the gateway. Bifrost, the AI gateway, is the control plane and policy engine, and Bifrost Edge extends that same governance to the endpoint by running on company machines and routing AI traffic from desktop chat apps, browser AI, coding agents, and the MCP servers those tools connect to through the organization's Bifrost. The virtual keys, budgets, guardrails, and audit logs already configured at the gateway are what apply on the laptop, with no per-app setup and nothing new to learn on the policy side. Bifrost Edge is currently in alpha, and teams register to be onboarded.

Common Questions About Policy-Based AI Governance

How is this different from API gateway policy?

An API gateway authenticates callers and shapes HTTP traffic, but it has no concept of tokens, model pricing, prompts, or tool calls. Policy-based AI governance adds model-aware dimensions: per-model access, token and cost budgets, prompt and response inspection, and MCP tool scope.

Does enforcing policy in the request path add latency?

Policy checks run in process against in-memory state rather than a database lookup per request. Bifrost adds 11 microseconds of overhead per request at 5,000 requests per second in sustained benchmarks, which keeps enforcement well inside the noise of a provider round trip.

Who defines policy, and who consumes it?

Platform and security teams define the governance rules once: profiles, budgets, guardrail rules, and routing rules. Developers consume them by presenting a virtual key against an OpenAI-compatible endpoint, which means adopting policy costs them a base URL change rather than a rewrite.

Getting Started with Policy-Based AI Governance

Policy-based AI governance holds only when the policy engine sits where the traffic is. Bifrost puts access, spend, routing, content safety, tool scope, and audit evidence in one control plane that evaluates every AI call, and extends that plane to MCP servers and endpoint AI so the rules cover the tools people actually use. To see how policy-based AI governance maps to your providers, teams, and compliance requirements, book a demo with the Bifrost team.