How Enterprises Govern AI Agents in Production
Most enterprise AI programs write policy before anyone can produce an accurate list of the models, providers, and MCP servers already running inside the company. That sequencing failure is why efforts to govern AI agents stall: the policy document is finished, the enforcement point does not exist, and each application team implements controls its own way. Bifrost, the open-source AI gateway written in Go by Maxim AI, is built for enterprises that need those controls applied on the request path rather than described in a spreadsheet. This post sets out the rollout sequence that holds up in production: what to do first, what breaks at each stage, and who owns what.
Stage 1: Inventory Before Policy
AI agent governance in production begins with an inventory, not a control document. Before any rule is written, an enterprise needs the real list: which models are called, which providers hold the credentials, which MCP servers are connected, and which teams produce the traffic. Policy written against an assumed architecture governs a system that does not exist.
The NIST AI Risk Management Framework sequences AI risk work as Govern, Map, Measure, and Manage; Map precedes Manage for this reason. The gap between the diagram and the running system is usually wide: an unapproved provider added for a prototype that was never decommissioned, or an agent calling forty MCP tools, six of which write to systems of record.
Bifrost produces this inventory as a side effect of handling traffic, and used as an MCP gateway it covers the tool layer where agent behavior lives:
- Request monitoring and dashboard logs: models, providers, latency, tokens, and cost per request.
- MCP tool logs: per-tool call records, tool-call volume histograms, and top tools by call count.
- Prometheus metrics: native scraping and Push Gateway support.
- OpenTelemetry export: OTLP traces into Grafana, New Relic, Honeycomb, or Datadog, so AI traffic lands in the existing APM.
What breaks if you skip it: controls get written for the estate the organization believes it has, so enforcement misses the traffic carrying real risk. The inventory also covers only what routes through the gateway; locally installed tools come at Stage 5.
Stage 2: Put Every Request on One Path
Governance applies only to traffic that passes the enforcement point, so the second stage of an AI agent rollout is consolidation. Every application, notebook, batch job, and coding agent moves onto a single request path where one policy engine sees all of it. Bifrost is a drop-in replacement, so the change is the base URL.
Existing code written against the OpenAI, Anthropic, Bedrock, Google GenAI, LiteLLM, LangChain, and PydanticAI SDKs keeps working unchanged. Coding agents and editors including Claude Code, Codex CLI, Gemini CLI, Cursor, Qwen Code, Roo Code, and Zed point at the gateway the same way, putting developer AI usage under the same policy.
Migration is normally negotiated on latency, and that negotiation is short: Bifrost adds 11 microseconds of overhead per request in sustained benchmarks at 5,000 requests per second.
What breaks if you skip it: partial coverage produces partial governance. A budget that sees 70% of spend is not a budget, and cost attribution becomes an estimate.
Stage 3: Bind Identity, Entitlement, and AI Agent Cost Control
Stage 3 replaces shared provider API keys with governed identities. Virtual keys are the primary governance entity in Bifrost: each carries its own access permissions, budgets, rate limits, routing rules, and MCP tool filters. Identity comes from the existing directory, and entitlement becomes policy rather than a shared secret.
Single sign-on through OIDC with SCIM provisioning connects Bifrost to Okta, Microsoft Entra, Keycloak, Zitadel, Auth0, and Google Workspace, so joiners, movers, and leavers propagate automatically. Role-based access control with custom roles governs who can change configuration; Data Access Control governs who can see request content.
Access Profiles are what make this stage survive the twentieth team. Instead of hand-building a virtual key per project, an administrator defines a reusable profile of provider, model, budget, rate-limit, and MCP policy that auto-allocates virtual keys at scale. Onboarding becomes an assignment, not a configuration exercise.
AI agent cost control uses the same structure. Hierarchical budgets and rate limits apply at the virtual key, team, and customer levels, so one agent cannot exhaust a department allocation. Model limits restrict which models a consumer may call, and routing rules direct requests using weighted strategies and fallback chains.
What breaks if you skip it: spend cannot be attributed to an owner, and revoking a departing employee's access means rotating a shared key that a dozen services depend on.
Stage 4: Constrain the Tool Surface With Agent Tool Permissions
What separates an agent from a chat application is the set of actions it can take, which makes agent tool permissions the control that matters most. Stage 4 defines which MCP tools each consumer can see, which can be executed, and which need approval first. Bifrost applies deny-by-default filtering across three stacked levels.
Those levels are client configuration (tools_to_execute), per-request headers, and per-virtual-key filtering. A tool must pass every applicable filter, and an empty or omitted list denies everything. This tool filtering model gives each virtual key its own MCP server view.
Two mechanisms keep this manageable at scale:
- MCP Tool Groups: reusable bundles attachable across six dimensions (virtual keys, teams, customers, users, LLM providers, and API keys). Tool Groups resolve at request time from an in-process index, merge and deduplicate across dimensions, and can be disabled centrally to revoke access without unpicking attachments.
- MCP authentication and sessions: auth types include None, Headers, OAuth 2.0 with token refresh and PKCE, per-user OAuth, and per-user headers. MCP Sessions let administrators inspect, re-authenticate, and revoke a user's credentials, and MCP with federated auth turns internal APIs into governed tools.
Execution is conservative by default. Bifrost does not auto-execute tool calls: an LLM tool call is a suggestion, and execution requires an explicit call to the tool execution endpoint. Agent Mode adds autonomous multi-step execution, with auto-approval opted into per tool. The OWASP Top 10 for LLM Applications 2025 lists excessive agency as its own risk category, and deliberate opt-in answers it.
Tool sprawl is also a cost problem. With three or more MCP servers attached, Code Mode has the model write Python in a sandbox to orchestrate tools instead of receiving a hundred or more tool definitions, reducing input token usage by up to 92.8% and estimated cost by up to 92.2% versus classic MCP.
What breaks if you skip it: the agent inherits every tool its MCP connections expose. The model is governed, the actions are not, and a prompt injection in retrieved content becomes a write to a production system.
Stage 5: Extend Coverage to Employee Machines and Shadow AI
The gateway governs what routes through it. Desktop AI apps, browser AI, individually installed coding agents, and the MCP servers those tools connect to generally route through nothing. Stage 5 closes that shadow AI gap by extending the same policy set to every company machine, so governance follows the user rather than waiting for a manual reconfiguration.
Bifrost, the AI gateway, is the control plane and policy engine where virtual keys, budgets, rate limits, guardrails, and audit logging are defined. Bifrost Edge extends that same governance to every machine, enforcing those policies on endpoint AI traffic with no per-app configuration: no base URLs to change, no SDKs to swap. Edge is currently in alpha.
The capability that matters for agent governance is MCP visibility on the device. Edge builds a fleet-wide inventory of the MCP servers configured inside each AI app, covering Claude Code, Claude Desktop, Gemini CLI, OpenCode, Codex, and Cursor, then enforces per-server allow or deny decisions on the device rather than publishing guidance. Rollout runs through existing device management platforms (Jamf, Microsoft Intune, Kandji, Omnissa Workspace ONE, and JumpCloud) on macOS, Windows, and Linux.
What breaks if you skip it: the governed estate looks clean while sensitive data leaves through tools that were never in scope. Shadow AI becomes a coverage problem, not a policy problem.
Stage 6: Keep a Record That Survives an Audit
Stage 6 converts enforcement into evidence. Auditors ask for two distinct artifacts: proof that a control existed and was configured a particular way on a given date, and proof of what happened on individual requests. Bifrost produces both, through different mechanisms, and conflating them is a common way to fail an evidence request.
Guardrails validate prompts and responses in real time, configured as reusable rules and profiles, not per-application code. Native providers include Secrets Detection (Gitleaks-backed) and Custom Regex with a built-in PII Detection template, alongside third-party providers such as Microsoft Presidio, AWS Bedrock Guardrails, Azure Content Safety, and Google Model Armor. Redaction modes let a request continue with sensitive content removed instead of failing outright.
Audit logs record administrative activity: who changed what, when, and on which resource. Entries can be signed with an HMAC key, retained for a configurable number of days, filtered by text, action, outcome, or date range, and exported as JSON, JSON Lines, or Syslog, with periodic archival to S3 or GCS. Request-level evidence comes from elsewhere: the logs and telemetry established at Stage 1, plus log exports to S3 and GCS. ISO/IEC 42001, the AI management system standard, is built around the same pairing of documented control and retained record.
What breaks if you skip it: the controls work and cannot be shown to have worked, so every audit becomes a manual reconstruction from screenshots.
The AI Agent Rollout Operating Model: Who Owns What
AI agent rollouts fail on ownership more often than on technology. The workable split gives policy definition to governance and security, enforcement configuration to the platform team, and bounded self-service to application teams, each working in a different surface of the same system.
| Role | Decisions they own | Bifrost surface they work in |
|---|---|---|
| Governance and risk | Applicable frameworks, required evidence, retention periods | Audit log retention and export, guardrail profiles as written policy |
| Platform / AI infrastructure | Deployment topology, model catalog, routing strategy, budget hierarchy | Provider config, routing rules, virtual keys, Access Profiles, budgets |
| Security | Identity source, tool approvals, data protection rules, endpoint coverage | SSO/OIDC and SCIM, RBAC and Data Access Control, MCP tool filtering, Edge approvals |
| Application teams | Which models and tools an agent needs, spend within their allocation | Their assigned virtual key, request headers, tools within the permitted set |
The common failure mode is policy written centrally with enforcement left to each team. It produces a document that is accurate, approved, and unimplemented, because twelve teams interpret it twelve ways. Centralize enforcement in the governance controls at the gateway, then give teams freedom inside their allocated boundary. Where data cannot leave a controlled network, Bifrost Enterprise preserves that model with in-VPC, on-premise, and air-gapped deployment.
How Enterprises Govern AI Agents: Common Questions
How long does it take to govern an existing AI estate?
Stages 1 and 2 are typically fastest, because inventory and consolidation are configuration work rather than application changes. Identity binding at Stage 3 depends on how many shared provider keys exist and how fast directory integration is approved. Tool constraints and endpoint coverage then run in parallel.
Do application teams need to change code?
No. Pointing an application at Bifrost changes the base URL and the credential, and existing OpenAI, Anthropic, Bedrock, LangChain, and PydanticAI SDK code keeps working. Budget, model-limit, and tool-permission changes then take effect at the gateway without a redeploy.
How do enterprises govern AI agents that run on employee laptops?
A gateway covers configured traffic, so laptops need the policy pushed to them. Bifrost Edge extends the virtual keys, budgets, guardrails, and audit logging defined at the gateway to each machine, and its MCP server governance inventories and enforces decisions on the MCP servers inside local AI apps. Edge is in alpha today.
Getting Started: Govern AI Agents on One Enforcement Point
Organizations that successfully govern AI agents do it in order: inventory what is running, consolidate onto one request path, bind identity and budgets to that path, constrain the tool surface, extend coverage to endpoints, and keep evidence of configuration and traffic. Each stage assumes the one before it, which is why AI agent governance that starts with a policy document and no enforcement point stays on paper. To see how this rollout maps to your estate, book a demo with the Bifrost team.
