The Complete Guide to AI Governance: From Policy to Runtime Enforcement

AI governance is the set of policies, controls, and enforcement mechanisms that determine which models an organization can call, who can call them, how much they can spend, and what data is allowed to leave the building. Most governance programs stop at the policy document: a wiki page that says "use approved models only" with nothing enforcing that intent on the request path. Bifrost, the open-source AI gateway built in Go by Maxim AI, closes that gap by enforcing governance policy at runtime on every LLM request, so the rules a security team writes are the rules that execute in production. This guide covers AI governance end to end: defining policy, enforcing it at the gateway control plane, and extending that same enforcement to every employee machine with Bifrost Edge.

What Is AI Governance

AI governance is the discipline of defining and enforcing organizational policy over how AI systems are accessed, used, funded, and audited. It spans access control (who can use which models), cost control (budgets and spend limits), rate control (throttling), content control (guardrails), and accountability (audit logs). Effective governance is not a document; it is a control plane that applies these decisions to every request in real time.

The distinction that matters is policy versus enforcement. A policy states intent. Enforcement makes the intent unavoidable. The NIST AI Risk Management Framework frames governance as a continuous function across the AI lifecycle rather than a one-time review, and industry work such as the Cloud Security Alliance's Agentic AI Governance reference architecture has pushed toward enforcing those requirements continuously at machine speed. A gateway is where that enforcement naturally lives, because every AI request already passes through it.

Why Governance Matters for Enterprise AI Teams

Ungoverned AI usage creates four measurable risks that compound as an organization scales its AI adoption:

• Uncontrolled spend: Without per-team or per-project budgets, LLM costs grow without attribution, and a single misconfigured job can exhaust a monthly budget in hours.
• Data exposure: Prompts and completions can carry PII, secrets, and proprietary source code to third-party model providers with no inspection layer in between.
• No access boundaries: When every developer holds a raw provider API key, there is no way to restrict models, revoke access instantly, or attribute usage to a person or team.
• No audit trail: Compliance frameworks such as SOC 2, GDPR, HIPAA, and ISO 27001 require a record of who did what. Raw provider keys produce no such record.

Bifrost is built as the AI gateway for enterprises and large teams that need to answer these four risks with enforced controls rather than written guidelines. Governance moves from a policy PDF to a runtime property of the infrastructure.

How Governance Works at the Gateway Control Plane

The Bifrost AI gateway is the control plane where AI governance policy is defined and enforced. Every request routes through a single OpenAI-compatible endpoint, which gives one place to apply access control, budgets, rate limits, guardrails, and audit logging. Because enforcement happens on the request path, a policy change takes effect on the next request rather than requiring a code deploy in every downstream application.

Virtual keys: the primary governance entity

Virtual keys are the core governance primitive in Bifrost. Instead of distributing raw provider API keys, teams issue virtual keys that carry their own access permissions, budgets, and rate limits. Each virtual key can be scoped to specific models and providers, restricted to specific underlying provider keys, and switched active or inactive to grant or revoke access instantly. Because governance is centralized on the virtual key, revoking access is a single toggle rather than a key-rotation project.

Budgets and rate limits: cost and traffic control

Bifrost enforces cost control through a hierarchical budget structure. Budgets and rate limits can be set independently at the customer, team, virtual key, and provider-config levels, and are checked cumulatively so a request must satisfy every budget in its chain. Budgets are defined as a dollar limit with a reset duration (from one minute to one year) and can be calendar-aligned to reset on UTC day, week, month, or year boundaries. Rate limits throttle by both token count and request count per period, which prevents a runaway job from consuming a shared quota.

Access control: RBAC and data access control

For team-level administration, Bifrost applies role-based access control with three system roles (Admin, Developer, Viewer) plus custom roles scoped to your organization. Data access control adds row-level isolation on top: RBAC decides which operations a user can perform, and data access control scopes the result set to the rows that user's role is entitled to see, so a developer on one team cannot view another team's virtual keys or routing rules. RBAC integrates with OIDC user provisioning so roles map to identity-provider groups.

Guardrails: runtime content enforcement

Guardrails validate the content of prompts and responses in real time against policies you define. Bifrost evaluates inputs and outputs before they reach a model or return to a user, catching PII leakage, credential leakage, prompt injection, and policy violations. Guardrails are built from reusable rules (written in Common Expression Language) and profiles that configure providers, including native Secrets Detection (Gitleaks-backed), native Custom Regex with a built-in PII template, AWS Bedrock Guardrails, Azure Content Safety, Google Model Armor, CrowdStrike AIDR, GraySwan Cygnal, and Patronus AI.

Audit logs: the accountability record

Audit logs in Bifrost Enterprise record administrative activity so operators can review who changed what and when. Entries can be signed with an HMAC key for tamper-evidence, filtered by action and outcome, retained for a configurable number of days, and exported as JSON, JSON Lines, or Syslog for downstream review. This is the record that SOC 2, GDPR, HIPAA, and ISO 27001 audits require, produced automatically rather than reconstructed after the fact.

Extending Governance to the Endpoint with Bifrost Edge

The Bifrost AI gateway governs every request that routes through it. The problem is that not all AI usage does. Employees install Claude Desktop, use ChatGPT in the browser, run coding agents in the terminal, and wire MCP servers into their tools, all without pointing anything at the gateway. That ungoverned usage is shadow AI: sensitive data leaving through tools security teams cannot see, with no budget control, no guardrails, and no audit trail.

Bifrost, the AI gateway, is the control plane and policy engine; Bifrost Edge extends that same governance to every machine. Edge runs on each computer and routes all AI traffic (desktop chat apps, browser AI, coding agents, and the MCP servers those tools connect to) through the organization's Bifrost. The virtual keys, budgets, rate limits, guardrails, and audit logs already configured at the gateway are exactly what Edge enforces on the endpoint. There is nothing new to learn on the policy side. Bifrost Edge is currently in alpha.

What Edge governs on each machine

• AI apps: Administrators decide which AI applications are permitted, and app governance enforces that decision on each device. Allowed apps run normally and fully governed; disallowed apps are blocked before any data leaves the machine.
• MCP servers: Edge inventories the MCP servers configured inside each AI app and builds a live, fleet-wide inventory, then enforces per-server allow or deny decisions directly on the device. A denied server cannot be used even by an app that had it configured before the policy existed.
• Guardrails everywhere: Because Edge routes traffic through Bifrost, the guardrail profiles you already configured apply automatically to endpoint AI, so secrets and PII are caught before they leave the laptop.

Edge runs natively on macOS, Windows, and Linux, and deploys fleet-wide through existing device management platforms including Jamf, Microsoft Intune, and Kandji using a managed MDM configuration that carries only non-sensitive connection settings. Identity and keys come from the user's SSO sign-in, so no secrets live on the device.

Key Considerations for Implementing Governance

Teams rolling out AI governance should sequence the work so enforcement follows visibility rather than preceding it:

• Centralize access first: Replace raw provider keys with virtual keys so every request carries an identity, a budget, and a policy before adding finer controls.
• Set budgets at the level you report on: Attach budgets to the customer, team, or project boundary you already track spend against, so cost attribution matches your accounting.
• Enforce guardrails on both input and output: PII and secrets can appear in a prompt or in a model's response, so validate both directions.
• Cover the endpoint, not just the datacenter: A gateway only governs configured traffic. Pair the gateway with Bifrost Edge to bring shadow AI on employee machines under the same policy.
• Deploy in your own environment for regulated data: For regulated industries and strict data-residency requirements, Bifrost Enterprise supports in-VPC, on-prem, and air-gapped deployments so no AI traffic leaves your boundary.

How is governance different from AI observability?

AI governance controls and constrains AI usage (access, spend, content, and audit), while observability measures and reports on it (latency, errors, traces, and metrics). Governance decides whether a request is allowed; observability tells you what happened after it ran. Bifrost provides both on the same request path.

Does AI governance require changing application code?

No. Because Bifrost is a drop-in replacement for existing provider SDKs, adopting gateway-level governance means changing only the base URL. Endpoint governance through Bifrost Edge requires no per-app configuration at all, since Edge routes traffic transparently at the machine level.

Where are governance policies enforced?

Governance policies are enforced at runtime on the request path. At the gateway, every API call is checked against its virtual key, budgets, rate limits, and guardrails before it reaches a provider. With Bifrost Edge, the same checks apply to AI traffic originating from apps on employee machines. For teams evaluating gateways, the LLM Gateway Buyer's Guide breaks down governance capabilities across the category.

Getting Started with AI Governance in Bifrost

AI governance succeeds when policy becomes runtime enforcement rather than a document nobody enforces. Bifrost provides that enforcement at the gateway control plane with virtual keys, budgets, rate limits, guardrails, and audit logs, and Bifrost Edge extends the same controls to every machine to close the shadow AI gap. Together they give enterprise teams one policy model that applies from the datacenter to the endpoint. To see how Bifrost can bring your AI usage under governance, book a demo with the Bifrost team.