Benchmarking Bifrost's upgraded Complexity Router

Bifrost’s Complexity Router sends each request to the model it needs, so your frontier model only handles the work that actually needs it. Since v2.0, you choose how incoming requests get classified into complexity tiers: a semantic classifier that matches requests against phrases you define, or a decision model such as Typesafe’s Jev (added in v2.3.0) that judges each request against tier definitions you can edit.

We benchmarked both approaches against sending every request to Claude Opus 5.5. On everyday questions, Bifrost with Jev spent 36% less than Opus, at nearly the same solve rate. On coding-agent tasks, it matched Opus at a comparable cost per solved task.

Two ways to classify incoming requests

Complexity Router puts each request into one of three tiers, Simple, Medium or Complex, and your routing rules map each tier to a model, for example Haiku, Sonnet and Opus.

The semantic classifier matches each incoming request to the closest of the example phrases you've assigned to each tier. Bifrost ships a default starting set, and you can add phrases from your users' requests until it classifies work the way your team would.

A decision model helps assign the complexity tier to an incoming request, using a short definition, a few signals and a few examples for each tier - all of which you can edit. Typesafe’s Jev is the default. You can switch to other decision models, including ones that run inside your own network, such as Laya or a model served through Ollama.

With session-aware routing on, a conversation can move up a tier when the work gets harder but never drops back down, so models don't keep switching mid-task, helping you save costs while also ensuring outcome quality doesn’t take a hit.

What was tested

We used two public benchmarks:

• RouterArena: 8,400 single questions with known right answers, to measure savings on everyday requests.
• Terminal-Bench 2.1: 89 hands-on engineering tasks completed by an agent (Claude Code), to see whether routing holds up when every turn of a long session is billed.

We compared four setups on the same tasks: route requests directly to Opus 5.5, route the same requests to Bifrost with Jev as a classifier, route the same requests to the Bifrost semantic classifier, and LiteLLM's heuristic v2 auto-router classifier. Every router chose between Haiku 4.5, Sonnet 5 and Opus 5.5, with default settings throughout.

What we found

Everyday questions: 36% less spend, nearly the same solve rate

The savings come from where Jev sent the work. Fewer than 2 in every 100 questions went to Opus, and majority of the rest went to Sonnet. LiteLLM and semantic (defaults) sent 38% to 46% to Opus and still solved fewer questions.

Classifying first adds a little time: Jev's answers came back about 0.7 seconds later than Opus's on average.

Read the full benchmark report and methodology

Coding agents: Opus-level results at a comparable cost

Jev used Opus on fewer than half the tasks and solved within one task of Opus, at about the same cost per solved task.

Read the full benchmark report and methodology

Cheaper models don't automatically make agent runs cheaper

Agents pay for every turn, and a smaller model can take more turns to get to the same result. On the tasks Jev sent to Sonnet, Sonnet solved as many as Opus but cost slightly more.

Push harder toward small models and quality drops while savings barely move. LiteLLM's heuristic v2 sent 58 of the 77 tasks to Haiku. On those tasks it spent 4% less than Opus did and solved 23 where Opus solved 56.

For agent traffic, it is best to watch cost per solved task, not how often the cheap model gets picked.

Try it

Both classifiers are in Bifrost v2.3.0. Get started today: Complexity Router docs

What's next

Our goal is to help enterprises spend less on models without compromising output quality, grounded in their own traffic rather than a classifier trained on someone else's data.

We are working on bringing capabilities that would use your data and your usage logs to surface signals/queries automatically, without the manual upkeep required to maintain the mappings manually. Stay tuned!