AI Audit Trail: Controls and Audit Logs for LLM Traffic
TL;DR
- An AI audit trail for LLM traffic is a signed, retained record of who called which model, with what prompt and response, at what cost, and which policy fired, produced at the gateway so it covers every request.
- The runtime controls that belong at the LLM traffic layer are virtual keys, hierarchical budgets, request and token rate limits, model and provider allow-lists, PII and secrets redaction, and prompt and response guardrails.
- Bifrost writes HMAC-signed audit events to a database for search and export, and can additionally archive them to S3 or GCS as time-windowed JSONL objects with a manifest, so the archive is durable, verifiable, and independent of the live store.
- Retention, dashboard search, JSON/JSONL/Syslog export, and object-storage archival map directly onto SOC 2, ISO 27001, HIPAA, and EU AI Act Article 12 record-keeping obligations.
- Bifrost Edge extends the same runtime controls and audit records from the gateway out to every employee laptop, so desktop chat apps, browser AI, coding agents, and MCP servers are governed and logged under one policy set.
An AI audit trail for LLM traffic is the signed, complete record of every model call an organization makes, including who initiated it, which model and provider handled it, what the prompt and response were, how much it cost, and which policies applied. Bifrost, the open-source AI gateway built in Go by Maxim AI, produces this record inline with the request path, so the log is proof that the controls actually fired.
This post covers what controls belong at the LLM traffic layer, what a compliance-grade record looks like in Bifrost audit logs, and how the trail maps onto SOC 2, ISO 27001, HIPAA, and EU AI Act obligations. It is the runtime and audit half of AI governance; the policy half is covered in the companion enterprise LLM governance framework piece.
What Is an AI Audit Trail for LLM Traffic?
An AI audit trail for LLM traffic is a per-request, tamper-evident log emitted by the layer that mediates model calls, capturing identity, model, prompt, response, cost, latency, and policy outcome. Placing the trail at the gateway means every request is logged the same way, regardless of which SDK or provider handled it, and auditors can retrieve a specific prompt, response, and policy decision for any point in time.
Two properties separate a real audit trail from application logs. First, integrity: entries are signed or verifiable, so a change to a stored record can be detected. Second, completeness: the trail covers every call, not the ones a developer remembered to instrument. Bifrost delivers both because it sits inline with the request path and produces HMAC-signed audit events as a first-class output.
The Runtime Controls That Belong at the LLM Traffic Layer
The LLM traffic layer is the right place for controls that need to see every request and every response before either reaches a model or an application. Anything downstream, such as an application-level check inside one service, is invisible to every other service in the fleet. The table below lists the six control classes that belong at this layer, what each prevents, and which Bifrost feature enforces it.
| Control | What it prevents | Bifrost feature |
|---|---|---|
| Identity and access | Anonymous calls, unattributable spend, permission sprawl | Virtual keys with per-consumer permissions |
| Budgets | Runaway spend on any single user, team, customer, or provider | Hierarchical budgets across virtual key, team, customer, and provider config |
| Rate limits | Abusive traffic, provider throttling cascades, quota exhaustion | Request and token rate limits at virtual key and provider config levels |
| Model and provider allow-lists | Calls to unapproved or unreviewed models | Model and provider filtering per virtual key |
| PII and secrets redaction | Personal data or credentials leaving the machine in a prompt | Secrets Detection (Gitleaks-backed) and Custom Regex with the built-in PII Detection template |
| Prompt and response guardrails | Prompt injection, unsafe outputs, policy violations | Guardrails with AWS Bedrock, Azure Content Safety, Patronus AI, and other providers |
These are the AI controls a runtime governance layer must implement to produce a defensible audit trail. Placing them at the gateway makes the governance model uniform: every service reaches the same policies through the same enforcement point, and the log has a consistent shape across the fleet. Companion analysis of the deeper cluster is in what is LLM governance and AI governance with virtual keys for LLM and MCP traffic.
Virtual Keys, Budgets, and Rate Limits
Virtual keys are the primary governance entity in Bifrost. Every request carries a virtual key in an x-bf-vk, Authorization, x-api-key, x-goog-api-key, or api-key header, and the key resolves to a consumer with specific model and provider permissions, an independent budget, and independent rate limits. Provisioning access to a team or user is a matter of issuing a virtual key with the right scope, not distributing raw provider credentials.
Budgets are hierarchical. Each request is checked against the provider config budget, the virtual key budget, the team budget (if attached), and the customer budget (if attached); all applicable budgets must pass or the request is blocked. Costs are deducted from every level that applies, so a team's spend rolls up to a customer's spend without either being tracked twice.
Rate limits apply at both virtual key and provider config levels, in both request-count and token-count forms, with reset periods from one minute to one year. The audit trail records the virtual key, the budgets that were checked, and the rate-limit windows in effect, so an auditor's question of "who was allowed to spend, and on what" is answered with identities and numbers rather than policy description. For the vertical view of these same controls, gateway-level LLM controls for banks is the banking-specific walkthrough.
Guardrails: PII, Secrets, and Prompt and Response Policy
Guardrails are the content-safety layer that inspects prompts and responses before either side reaches the model or the caller. Bifrost's guardrail system is built around rules (CEL-defined conditions that decide when to evaluate) and profiles (reusable configurations for guardrail providers), so an organization defines policies once and applies them across many virtual keys. This is what a functional LLM firewall looks like: policy defined in one place, enforced on every request, with no per-app configuration.
Two Bifrost-native providers handle the most common data-protection cases. Secrets Detection uses embedded rules from Gitleaks v8.30.1 to catch leaked API keys, tokens, and private keys in either input or output, with a configurable action of detect_only, block, or redact. Custom Regex evaluates text against organization-defined patterns and ships with a built-in PII Detection template covering emails, phone numbers, SSNs, credit-card-shaped numbers, and IP addresses. Both can redact in three modes: runtime rewrites the live text, logs_only leaves the model call alone but redacts the stored record, and runtime_reversible uses reversible placeholders.
External providers plug in through the same rules-and-profiles model, including AWS Bedrock Guardrails, Azure Content Safety, Microsoft Presidio, Azure AI Language PII, Google Model Armor, CrowdStrike AIDR, GraySwan Cygnal, Patronus AI, Lakera Guard, and Repello Argus. When a rule fires, the audit event records which profile ran, which action was taken, and which entity type was redacted, so the decision can be reconstructed months later. This is the model behind LLM guardrails for fintech compliance and adjacent regulated-industry work.
The Anatomy of an Audit-Log Record
Every Bifrost audit event carries the fields an investigation needs: when the event happened, what happened, whether it succeeded, who initiated it, which resource was affected, the request path and method, the initiator IP, and the request duration. The underlying record is JSON, exportable as JSON, JSON Lines, or RFC 5424 Syslog for a SIEM pipeline. The example below is a representative JSONL line for an administrative action, in the shape Bifrost's dashboard, API, and export path all use.
{"time":"2026-08-14T09:12:47.318Z","action":"create","target":{"type":"virtual_key","id":"vk_9f0e3","name":"Engineering Team API"},"initiator":{"type":"user","id":"user_2a71","email":"platform-lead@acme.example"},"outcome":"success","path":"POST /api/governance/virtual-keys","ip":"10.4.2.19","duration_ms":42,"changes":{"budget":{"max_limit":100.00,"reset_duration":"1M"},"rate_limit":{"token_max_limit":10000,"token_reset_duration":"1h","request_max_limit":100,"request_reset_duration":"1m"},"allowed_models":["gpt-4o-mini","claude-3-5-sonnet-20241022"]},"signature":"hmac-sha256:6d9d0f4c…"}
Two elements make this record audit-grade rather than a developer log. The signature field is an HMAC-SHA256 hash computed with a key configured via hmac_key (minimum 32 bytes, resolved from an environment variable), so any change to a stored entry breaks verification. The outcome field explicitly records whether the operation succeeded, failed, or is pending, which is what an ISO 27001 or SOC 2 auditor needs to see when reviewing failed access attempts. Filtering by initiator, target, IP, path, action, outcome, or date range narrows the log to the slice under review, and users with the AuditLogs:Download permission can export the filtered set. This matches the pattern behind building an audit trail for every AI interaction and LLM monitoring metrics, audit logs, and controls.
Signed, Retained, and Archived
Retention in Bifrost has two independent loops. The primary loop is database retention, governed by retention_days: the database is the source of truth for dashboard search, HMAC verification, filtering, and export, and entries older than the configured window are pruned on a schedule. Setting retention_days: 365 keeps a year of live-queryable events; setting 0 disables age-based cleanup.
The second loop is object-storage archival. When object_storage is configured on the audit_logs block, a background job copies every event to S3, GCS, or an S3-compatible bucket (MinIO, R2) as time-windowed JSONL objects, laid out by UTC date under a {prefix}/audit-logs/{YYYY}/{MM}/{DD}/{start}-{end}/ path. Each window is committed by a manifest.json listing exactly the parts that belong to it, written last, so a partial upload is never mistaken for a completed archive. Parts roll at archive_max_object_bytes (default 128 MiB), and archival is at-least-once with retry, so a crashed node re-runs its window rather than skipping it.
Archive and database are decoupled by design. The archive is a mirror, not an offload, so the full event lives in both stores. Once retention_days deletes a row, its object in the bucket is untouched and is governed instead by the bucket's own lifecycle rules, which is where S3 Object Lock or WORM enters the picture for multi-year retention. Operational rule: keep retention_days comfortably larger than archive_interval + archive_grace_period, or the cleaner may delete a row before the archiver copies it. Log Exports is a separate feature that pairs the request logs database with object storage for large-payload offload, giving operators a complete story for both request traces and administrative events.
Compliance Mapping: SOC 2, ISO 27001, HIPAA, and the EU AI Act
The specific controls above map onto the record-keeping and access-control obligations regulators actually assess. The table below summarizes the mapping, with the corresponding Bifrost capability that produces the evidence.
| Framework or clause | What it requires | Bifrost evidence |
|---|---|---|
| SOC 2 Common Criteria (CC 6, CC 7) | Logical access controls, event logging, monitoring | Virtual keys, RBAC, signed audit events with initiator, outcome, and target |
| ISO/IEC 27001 Annex A.5 and A.8 | Information security policies, access control, event logging | Guardrail rules and profiles, hierarchical budgets, retention policy, exportable audit logs |
| HIPAA Security Rule audit controls (45 CFR §164.312(b)) | Record and examine activity in systems containing ePHI | HMAC-signed events, database and object-storage retention, filterable review, PII redaction at the traffic layer |
| EU AI Act Article 12 (record-keeping) | Automatic recording of events over the lifetime of a high-risk AI system, retained for the operator | Per-request events with model, provider, cost, latency, outcome, and policy fields, archived to durable object storage |
Bifrost is not a compliance product; the runtime controls and their audit records simply produce the artifacts auditors ask to see. Bifrost's RBAC and virtual-key configurations are the evidence for scoped access, the JSONL export is the evidence for activity logging, and the HMAC signature answers tamper-evidence directly. The enterprise governance posture is the same shape across all four regimes because the underlying evidence is the same.
For teams whose compliance program is anchored to NIST AI RMF, the ISO/IEC 42001 AI management system standard, or the EU AI Act, the fields above are what the framework's control statements need to reference.
Extending the Trail to Every Endpoint
A gateway audit trail is only as complete as the traffic that reaches the gateway. In practice, employees run Claude Desktop, browse to chatgpt.com, install Cursor, and wire MCP servers into their coding agents, and none of that traffic passes through a gateway unless something forces it to. That gap is shadow AI, and it is where an otherwise complete audit record loses coverage.
Bifrost, the AI gateway, is the control plane and policy engine; Bifrost Edge extends the same governance to every machine. Edge runs on macOS, Windows, and Linux, and once installed, it routes desktop chat apps, AI in the browser, coding agents, and the MCP servers those tools connect to through the organization's Bifrost.
The virtual keys, budgets, rate limits, guardrails, and audit logs configured at the gateway are exactly what Edge enforces at the endpoint; there is nothing new to define on the policy side. Rollout is fleet-wide through Jamf, Intune, Kandji, Workspace ONE, or JumpCloud with a managed configuration that points each machine at the right Bifrost. Edge is currently in alpha; early-access teams are onboarded on request. The compliance consequence is that the trail now covers the actual AI usage of the organization, not the subset developers remembered to route through the gateway.
Frequently Asked Questions
What is an example of an AI audit trail?
An example is a JSONL log line recording that user platform-lead@acme.example created a virtual key on 2026-08-14 at 09:12:47Z, from IP 10.4.2.19, with a $100 monthly budget and a 10,000-token-per-hour rate limit, accompanied by an HMAC-SHA256 signature. In Bifrost, this is exactly the shape of an event in the audit-logs store: time, action, initiator, target, outcome, path, IP, duration, and signature, exportable as JSON, JSON Lines, or RFC 5424 Syslog.
How can an AI audit trail be used in compliance audits?
Auditors use the trail to verify three things: that access was scoped (who could call what model), that policy was enforced (which guardrails ran and what they did), and that events are tamper-evident (that a stored record has not been altered). Bifrost's signed events, filterable dashboard, and JSONL exports supply all three, and the S3 or GCS archive provides the long-term retention that HIPAA and EU AI Act Article 12 expect beyond a live database window. See also AI agent audit logs for full visibility over tool usage.
What are AI controls at the LLM traffic layer?
AI controls at the LLM traffic layer are the runtime enforcements that decide whether a model call proceeds and in what form. In Bifrost, they include virtual-key access control, hierarchical budgets, request and token rate limits, model and provider allow-lists, PII and secrets redaction, and prompt and response guardrails. All are configured centrally and applied inline, so every request is checked against the same rules.
How are Bifrost audit logs kept tamper-evident?
Audit events are signed with an HMAC-SHA256 key configured via hmac_key in the audit_logs block, with a minimum key length of 32 bytes and support for environment-variable references. Any modification to a stored event breaks verification. The database is the source of truth for verification, and the S3 or GCS archive is an independent copy that can be retained under bucket lifecycle rules, WORM, or Object Lock.
What is the difference between audit logs and request logs in Bifrost?
Audit logs capture administrative and governance events (who created a virtual key, changed a budget, or approved an MCP server) and live under Governance > Audit Logs. Request logs capture individual model calls (prompt, response, model, latency, cost) and can be paired with an object-storage backend to offload large payloads while keeping searchable metadata in the database. Together they cover both operator-facing and traffic-facing halves of the record.
How does Bifrost help meet EU AI Act Article 12 record-keeping requirements?
Article 12 requires automatic recording of events over the lifetime of a high-risk AI system, retained for the operator. Bifrost writes a signed, per-request event with model, provider, cost, latency, outcome, and policy fields, retains it in the database for the configured window, and archives every event to S3 or GCS as manifest-committed JSONL. The archive is decoupled from the live database, so operators can hold the record for the multi-year windows Article 12 anticipates without inflating the live store.
Getting Started with an AI Audit Trail in Bifrost
An AI audit trail is only useful if the controls that produced it are the ones actually enforced on every request, and the record that describes them is one an auditor can verify. Bifrost combines the runtime controls, the signed events, and the durable archive in one system, so the operator description of the policy and the log record of its execution are the same artifact. To see how the governance and audit stack fits your compliance program, book a demo with the Bifrost team.
