Insights, integration guides, and updates from the Bifrost team.
JFrog Security Research recently disclosed two vulnerabilities in Bifrost's management API. They can only be exploited when both of these are true: the instance is exposed to the internet, and dashboard authentication has not been set up. Both are fixed in transports/v2.1.0
Bifrost now supports TypeSafe's jev judgment models behind a unified POST /v1/decisions endpoint and a 1:1 /typesafe drop-in for TypeSafe's official SDKs. This is available in v2.2.2 for both OSS and Enterprise. TypeSafe's jev family takes a piece of state - a support ticket, a model's answer, a review, any string or JSON blob - and returns calibrated judgments: a probability, a pick from your options, or a rubric score, each with confidence and a full probability distribution. That makes them
Today we are releasing Bifrost v2.0, the largest release we have shipped. A year ago we open sourced Bifrost with one belief: a gateway only earns the right to sit in front of every AI request if it is performant and resilient at scale. Bifrost now runs in production at Fortune 500 companies across financial services, healthcare, technology, pharma and defense, moving trillions of tokens a day. We also realized that most of an organization's AI traffic never reaches the gateway at all, because
Most teams that run AI in production eventually get the same question from finance. Spending went up last month, and someone wants to know which team, feature, or model caused it. Often there is no clear answer. The numbers exist, but they sit in two systems that were never built to share data. This is a design problem, not a sign that anyone was careless. Often a team builds its metrics pipeline and its budgeting system at different times, owned by different people. One tracks requests and tok
Introduction AI costs at enterprises can become expensive quickly. Recent examples show this clearly - Uber reportedly exhausted its annual AI budget only four months into 2026, while Pylon’s CEO said the company’s Anthropic bill was set to jump from $400K to $1.4M a year after crossing 150 seats. These are different companies at different scales, but the pattern is similar: once AI usage becomes normal across teams, cost control becomes an operational problem, not just a pricing problem. Most
The most damaging data leak in your organization this year probably won't come from an attacker. It'll come from an employee trying to hit a deadline. Right now, across nearly every company, people are pasting source code, customer records, contracts, and clinical notes into AI tools that security never approved, can't see, and can't audit, and most of it leaves no trace in the logs you actually watch. We called the last version of this problem shadow IT. This one is worse, and the gap between h
LLM features started out as text in, text out. They have not been that for a while. A support tool ingests a screenshot of a customer's account dashboard with billing details visible on screen. A document agent reads a vendor PDF where the malicious instructions live inside an embedded image, not in the text the model sees. A receipt-processing assistant accepts a photo of an invoice that includes a full credit card number. A vision-enabled medical tool gets handed a chart image with a patient's
Most AI security tooling is built around a binary: a prompt is either allowed through or rejected. That works for an obvious attack, but it is a blunt instrument for the situation enterprises actually hit most often, which is legitimate work that happens to carry sensitive data. A support agent pastes a customer's full record into a summarization tool because summarizing it is the job. A developer drops a stack trace with a live token into a debugging assistant. An analyst feeds a spreadsheet of
MCP adoption inside enterprise AI stacks moved from a single experimental server to fleets of tools wired into production agents in a remarkably short window. As that surface area expanded, three questions stopped being theoretical for platform teams: which tools is an agent allowed to call, who triggered any given call, and what arguments and results moved through it. Bifrost addresses those questions at the gateway layer with a first-class MCP audit log that captures every tool execution, the
LLM features in production fail in ways that classic guardrails are not looking for. A coding assistant returns JSON that looks valid and silently breaks the downstream pipeline that parses it. A customer support reply apologizes three times in five sentences and reads as defensive. A marketing draft names a competitor in a comparison the brand team never approved. An HR assistant writes advice in subtly gendered language. An internal tool answers a perfectly reasonable question with "As an AI a
See how Bifrost and CrowdStrike AIDR secure enterprise LLM traffic: AIDR handles detection, Bifrost enforces policy inline before any prompt reaches a model. Enterprise AI deployments run into the same set of security issues within a few months of going live. A support rep pastes a full customer record into a summarization tool, and the prompt leaves your network for a third-party API with its own retention policy. A retrieval agent ingests a vendor PDF with instructions hidden in white-on-whit
If you've spent any time around AI tooling in the last year, you've probably heard the phrase "Model Context Protocol" thrown around, usually followed by the analogy "it's USB-C for AI." It's a useful starting point, but it's also where most explanations stop. This post starts there too, and then keeps going, all the way down to what's actually moving on the wire when your agent calls a tool. The problem MCP exists to solve LLMs, on their own, are isolated. They know what they were trained on
Introduction Every team running LLMs in production runs into the same surface area sooner or later. Prompt injection. PII leaking into a model provider's training pipeline. Credentials pasted into a chat by accident. Toxic or off-policy responses going back to a user. Hallucinated content quoted as fact. Most teams solve these problems the first time they hit them, usually inside the application itself, with a one-off regex, a hand-rolled OpenAI moderation call, or a wrapper around a vendor SDK
Every MCP tool definition the model doesn't end up using is wasted context. That's the quiet bug in the way most teams run the Model Context Protocol today. Connect five MCP servers with thirty tools each, and the model receives 150 tool definitions before it ever sees the user's prompt, most of which it will never call. The token bill grows linearly with every server you add. Selection accuracy degrades. Latency creeps up. The framing this post wants to argue for is simple: tool discovery is n
Enterprise MCP governance comes down to three controls every production AI agent deployment needs: access, audit, and cost. Here is what each one looks like in practice. When a new engineer joins a team, they are not handed unrestricted access to every system the company runs. Their access is scoped to what they need, their actions are recorded, and the budget impact of what they do is tracked. The moment an AI agent is connected to a fleet of MCP servers without an equivalent layer in place, t
The Context Window Problem If you've built an agent that connects to more than a handful of MCP servers, you've already run into this. Every tool definition from every connected server gets loaded into the model's context window on every request. Five servers with 30 tools each means 150 tool definitions sitting in context before the model reads a single word of your prompt. This is more than a theoretical concern. Anthropic's engineering team documented a five-server setup where tool definiti
Most teams running multi-provider LLM stacks have moved past pure round-robin for traffic distribution across providers and API keys. They've adopted latency-based routing: measure response times, penalise slow backends, shift traffic toward the fast ones. The approach is reasonable, but it has a failure mode that's easy to miss until you're debugging a production incident. A 6-second response from your LLM provider can be either fast or slow, and a flat latency threshold has no way to tell whi
If you're running LLMs in production, you've probably built (or thought about building) a semantic cache. The pitch is simple: embed incoming prompts, find near-duplicates, serve the cached response, skip inference. It works. Except when it doesn't. A new paper from Apple introduces Krites, a caching policy that quietly fixes one of the most annoying problems in semantic caching, ie. the similarity grey zone, without adding a single millisecond to your serving latency. Let's break it down. Th
Scaling cost per query — Tools vs Tokens Average input tokens consumed per query as tool count increases Code Mode OFF Code Mode ON Estimated cost - Code Mode OFF vs ON Absolute dollar cost per round, same queries Code Mode OFF Code Mode ON
Gemini CLI is Google's command-line coding assistant that brings the power of Gemini models - including Gemini 2.0 Flash with its massive context window and multimodal capabilities - directly to your terminal. By routing Gemini CLI through Bifrost Gateway, you gain multi-model flexibility, comprehensive observability, MCP tool integration, and failover capabilities while preserving the familiar CLI experience. Architecture Overview Gemini CLI communicates with Google's Generative AI API for m
Zed is a high-performance, collaborative code editor built for the modern development workflow. With native AI assistant integration, Zed can leverage language models directly within the editor for code generation, refactoring, and explanation. Integrating Zed with Bifrost Gateway unlocks multi-provider model access, MCP tools, and observability - transforming Zed's AI assistant into a flexible, enterprise-ready coding companion. Architecture Overview Zed's language model integration supports
Qwen Code is Alibaba's advanced coding assistant, built on the Qwen large language model family with specialized reasoning capabilities for code generation, analysis, and refactoring. By routing Qwen Code through Bifrost Gateway, you unlock cross-provider flexibility, observability, and tool integration - allowing Qwen Code to work alongside or be substituted by other model providers while maintaining a consistent terminal experience. Architecture Overview Qwen Code communicates with APIs usi
Codex CLI is OpenAI's command-line tool for code generation and completion, bringing AI-assisted coding directly to the terminal. By routing Codex CLI through Bifrost Gateway, you gain access to multiple model providers, enhanced observability, and MCP tool integration - transforming a single-provider CLI into a flexible, multi-model development assistant. Architecture Overview Codex CLI communicates with OpenAI's API using their standard HTTP endpoints. Bifrost intercepts these requests by a
LibreChat is an open-source chat interface that provides a ChatGPT-like experience with support for multiple AI providers. Integrating it with Bifrost Gateway adds a layer of abstraction that enables unified model access, MCP tools, observability, and load balancing across different providers - all without modifying LibreChat's codebase. Architecture Overview LibreChat's custom endpoint feature allows integration with any OpenAI-compatible API. Bifrost leverages this by exposing an OpenAI-com
Claude Code brings Anthropic's Claude Sonnet 4.5 directly to the terminal with agentic coding capabilities. While powerful on its own, integrating it with Bifrost Gateway unlocks additional capabilities like multi-model access, MCP tools, and observability without modifying the Claude Code client. Bifrost acts as an API gateway that provides compatible endpoints for Anthropic, OpenAI, and Gemini APIs. By routing Claude Code through Bifrost, you intercept API calls at the transport layer, enabli
When building AI-powered applications, chances are you don’t want to rely on a single provider: * Claude for reasoning-heavy tasks * GPT-4o for multimodal inputs * Gemini for Google ecosystem integrations * Mistral for fast, cost-effective completions Each provider has different SDKs, auth methods, rate limits, and response formats. Maintaining them quickly becomes messy. Bifrost solves this by acting as a unified AI gateway: one API surface, multiple providers behind it. In this post, we
When building applications that depend on external LLM APIs, provider failures are inevitable. Network timeouts, rate limits, model unavailability, and service outages all cause request failures that break application functionality. This post explains how to implement automatic fallback mechanisms using Bifrost to maintain service availability when primary providers fail. Failure Scenarios in LLM Applications LLM API calls can fail for several reasons: * Rate limiting: HTTP 429 responses wh
Modern AI applications are rapidly moving beyond text-only interactions. Today's systems need to process images, generate speech, and transcribe audio, often within the same workflow. While individual providers offer these capabilities, building a robust multimodal system that spans multiple AI providers introduces significant complexity around API differences, error handling, and failover strategies. Bifrost, an open-source AI gateway, addresses these challenges by providing unified multimodal