What Is AI Governance? Frameworks and Enforcement

AI governance is the set of policies, roles, and technical controls that determine which AI models and tools an organization may use, who may use them, on what data, at what cost, and with what record. Most enterprise programs produce the policies and the roles, then stop before the controls. Bifrost, the open-source AI gateway written in Go by Maxim AI, is built for enterprises that need those controls applied on the request path instead of described in a document. This post covers the frameworks, the ownership split, and what enforcement requires in practice.

What Is AI Governance?

AI governance is the combination of policy, ownership, and enforcement that controls how an organization builds and uses AI systems. It answers five questions about every request: which model, which identity, which data, which budget, and which record. Governance exists when all five answers are decided by a system rather than attested by a team.

Each of those five decisions has to be resolvable at request time, not at review time:

• Model and provider scope: which models and providers are approved, and for which use cases.
• Identity: which user, service, team, or customer is behind a request.
• Data boundary: what content may reach a model, and what must be redacted or blocked before it does.
• Cost limit: which budget and rate limit apply, and what happens when either is exceeded.
• Record: what is logged, how long it is retained, who can read it, and whether the record can be verified.

The discipline overlaps with model risk management, which validates the models used in decisions, and with application security, which covers the attack surface. Governance grants permission and produces evidence, which is why AI governance controls increasingly live in infrastructure rather than in a policy wiki.

AI Governance Framework Options: NIST, ISO, and the EU AI Act

An AI governance framework gives an organization a structured vocabulary for AI risk and a defensible process for managing it. Three matter most to engineering teams: the NIST AI Risk Management Framework, ISO/IEC 42001, and the EU AI Act. The first two are voluntary and process-oriented. The third is binding law with dated obligations.

The NIST AI Risk Management Framework organizes AI risk work into four functions: Govern, Map, Measure, and Manage. Govern establishes accountability, Map identifies context and risk, Measure tracks it, and Manage allocates resources to treat it. NIST later published the Generative AI Profile (NIST AI 600-1), which applies those functions to generative-specific risks including data leakage, harmful output, and dependence on third-party models.

ISO/IEC 42001 specifies an AI management system an organization can be certified against, which makes it the framework most often raised during enterprise procurement. It asks for documented policy, defined roles, decision records, and internal audit, in the pattern ISO/IEC 27001 established for information security.

The EU AI Act is the regulatory one, and its timeline is staged. It entered into force on 1 August 2024. Prohibitions on unacceptable-risk practices and the AI literacy obligation applied from 2 February 2025. Obligations for general-purpose AI models under Articles 51 to 56 applied from 2 August 2025, and Article 50 transparency obligations for AI-generated content apply from 2 August 2026. Under the Omnibus agreement, which is agreed and pending formal adoption, stand-alone high-risk obligations under Annex III move to 2 December 2027 and embedded high-risk systems under Annex I move to 2 August 2028.

One artifact appears in every row: a durable, queryable record of what happened. Signed audit logs with configurable retention, export as JSON, JSON Lines, or Syslog, and archival to S3 or GCS are the engineering answer. Frameworks describe the obligation; a running system produces the evidence.

AI Governance Ownership: Who Writes Policy and Who Enforces It

AI governance ownership is normally split across four groups. A governance or risk committee writes policy, a platform or AI infrastructure team implements enforcement, a security team owns data and secrets exposure, and application teams consume what is approved. The split works only when all four share one enforcement point.

• Governance or risk committee: sets acceptable use, risk tiers, retention periods, and approval paths, and owns the framework mapping.
• Platform or AI infrastructure team: owns the layer where policy becomes configuration, including model allow-lists, budgets, identity integration, and the audit trail.
• Security: owns data classification, credential and PII exposure, tool authorization, and incident response for AI-specific failures.
• Application teams: consume approved models through approved credentials and inherit controls rather than reimplementing them.

The common failure is structural. Policy is written in a document, no enforcement point is assigned, and compliance becomes self-attested per team. One team hard-codes a provider key in an environment variable, another wires a new model into a service, a third connects an agent to an MCP server nobody reviewed. No control blocks any of them, so the record is a set of claims rather than observations.

Bifrost resolves the split by making the platform team's configuration the place policy lands. Role-based access control defines what an operator may change, and the committee's decisions become objects that application teams inherit instead of interpreting.

Runtime AI Governance Enforcement on the Request Path

Runtime AI governance enforcement means policy is evaluated on the request itself, before it reaches a model provider. Identity, model allow-lists, budgets, rate limits, guardrails, tool authorization, and logging apply to every request regardless of which team, SDK, or agent framework produced it. Governance written as documentation cannot do this. An enforcement point on the path can.

A gateway is the natural location for that enforcement point because it is the one component all AI requests already pass through. It sits below the application and is language-agnostic: a Python service using the OpenAI SDK, a Go service calling Anthropic, and a coding agent in a terminal arrive at the same place with the same credential format. Policy applied there is not reimplemented per codebase, and it cannot be skipped by a team that forgot a middleware.

The Bifrost AI gateway implements that enforcement point through a set of governance objects:

• Virtual keys are the primary governance entity, carrying per-consumer access permissions, model and provider filtering, budgets, rate limits, and key restrictions.
• Hierarchical budgets and rate limits apply at the virtual key, team, and customer levels, so a team ceiling holds even when individual keys sit under their own limits.
• Routing rules direct traffic to specific models, providers, and keys with weighted strategies and fallback chains.
• RBAC and Data Access Control separate what an operator may do from which rows of configuration and operational data each role can see.
• Access Profiles define reusable provider, model, budget, rate-limit, and MCP policies that auto-allocate virtual keys at scale, so nobody is handed a raw key by hand.
• SSO/OIDC with SCIM user provisioning connects Okta, Microsoft Entra, Keycloak, Zitadel, Auth0, and Google Workspace so identity, roles, and team membership come from the directory of record.
• Guardrails validate prompts and responses in real time against reusable rules and profiles, covering prompt injection, PII, credential leakage, and policy violations, with redaction modes supported.
• MCP tool filtering scopes which tools an agent may call per virtual key, with deny-by-default behavior when no tools are listed.
• Audit logs record administrative activity, can be signed with an HMAC key, and export or archive on a schedule for long-term retention.

The standing objection to a central enforcement point is latency. Bifrost adds 11 microseconds of overhead per request in sustained benchmarks at 5,000 requests per second, far below the variance of any provider call. Teams evaluating runtime governance controls should measure this directly, because an enforcement layer that adds meaningful latency will be routed around by the first team under a deadline.

The AI Governance Policy Gap Beyond the Gateway

A gateway governs the traffic configured to route through it. AI governance policy does not reach desktop chat apps, AI in the browser, coding agents in the terminal, or the MCP servers those tools connect to, because nobody pointed them at the gateway. That traffic carries the same source code and customer data as governed traffic, with none of the controls and no audit trail.

Bifrost, the AI gateway, is the control plane and policy engine: virtual keys, budgets, rate limits, routing, guardrails, and audit logs are defined and enforced there. Bifrost Edge extends that same governance to every machine, routing endpoint AI traffic through the organization's Bifrost so the configured policies apply to the AI people actually use. The policy model does not change; its reach does. Edge also builds a fleet-wide inventory of the MCP servers configured inside each AI app, with per-server allow and deny decisions enforced on the device. Bifrost Edge is currently in alpha, and teams register to be onboarded.

AI Governance Best Practices: Common Questions

The most durable AI governance best practices reduce to one principle: every policy statement should map to a control that runs at request time. Written standards set direction, define risk tiers, and satisfy auditors. The control that stops an unapproved model, an over-budget team, or a leaked credential has to run on the request path.

Is AI governance the same as AI safety?

No. AI safety concerns model behavior, including harmful output, alignment, and misuse. Governance concerns organizational control over AI systems: which are approved, who may use them, on what data, and with what record. Guardrails are where the two meet operationally.

Who owns AI governance in an enterprise?

Ownership is shared across a governance or risk committee, a platform or AI infrastructure team, security, and application teams. Programs fail when the committee writes policy no team can enforce technically, leaving compliance to per-team self-attestation.

Can AI governance be enforced without changing application code?

Yes. A gateway used as a drop-in replacement for provider SDKs requires only a base URL change. Existing OpenAI, Anthropic, Bedrock, Google GenAI, LangChain, and PydanticAI code continues to work while identity, budgets, guardrails, and logging are applied centrally.

What is runtime AI governance enforcement?

Runtime enforcement evaluates policy on each live request rather than in periodic review. Model allow-lists, identity checks, budgets, rate limits, guardrails, and tool authorization are applied before a request reaches a provider, so a violation is blocked when it occurs rather than found in an audit weeks later.

Getting Started with AI Governance

An AI governance program becomes real where a policy statement turns into a control that runs on the request path. The frameworks (NIST AI RMF, ISO/IEC 42001, the EU AI Act) define what to control and what evidence to keep. A gateway is where those requirements become virtual keys, budgets, guardrails, tool scopes, and a verifiable record.

To see how AI governance policy is configured and enforced across models, teams, and endpoints, book a demo with the Bifrost team.