Fireworks AI Provider on Bifrost

Fireworks AI provider summary

Fireworks AI provides OpenAI-compatible inference with native Responses API support in Bifrost. The provider specializes in fast, low-latency serving of open models like Llama and Mixtral with full streaming support.

Supported operations

Fireworks supports 4 major operations across chat, text completions, embeddings, and responses API. Chat completions, text completions, and responses API all support full streaming.

Parameter handling

Fireworks natively supports the Responses API with field preservation for previous_response_id, max_tool_calls, and store. OpenAI parameters convert with special handling for prompt caching via prompt_cache_isolation_key.

Supported Fireworks parameters

Quick reference of OpenAI-compatible parameters accepted when routing through Bifrost to Fireworks.

[
  "stream",
  "temperature",
  "top_p",
  "top_k",
  "max_tokens",
  "stop",
  "presence_penalty",
  "frequency_penalty",
  "seed",
  "response_format",
  "tools",
  "tool_choice"
]

Supported Fireworks models

Use the provider prefix fireworks/ in Bifrost model routes for deterministic provider targeting.

API reference by operation

Gateway paths and Fireworks upstream endpoints.

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "fireworks/accounts/fireworks/models/llama-v3p1-405b-instruct",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

curl -X POST http://localhost:8080/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "fireworks/accounts/fireworks/models/llama-v3p1-405b-instruct",
    "messages": [{"role": "user", "content": "Hello"}],
    "store": true
    "max_tool_calls": 2
  }'

curl -X POST http://localhost:8080/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "fireworks/accounts/fireworks/models/llama-v3p1-70b-instruct",
    "prompt": "Hello, my name is",
    "max_tokens": 50
  }'

curl -X POST http://localhost:8080/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "fireworks/accounts/fireworks/models/text-embedding-v1",
    "input": "Hello world"
  }'

Implementation caveats