Hugging Face Provider on Bifrost

Model aliases and identification

Unlike single-string model IDs, Hugging Face routes use a composite key. See Model aliases in Bifrost docs.

huggingface/hf-inference/meta-llama/Meta-Llama-3-8B-Instruct
// inference_provider → backend (hf-inference, fal-ai, cerebras, …)
// model_id → Hugging Face Hub model ID

1) Chat Completions

OpenAI-compatible chat at /v1/chat/completions. Bifrost converts requests per backend via chat.go; dynamic model aliasing maps Hub IDs to provider-specific model names. On HTTP 404, the provider model cache is invalidated and the request is retried.

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "huggingface/cerebras/meta-llama/Llama-3.1-70B-Instruct",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

2) Responses API

All chat-supported inference providers also support /v1/responses via Bifrost's internal conversion to chat completions.

curl -X POST http://localhost:8080/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "huggingface/groq/meta-llama/Llama-3.1-70B-Instruct",
    "input": "Hello",
    "max_output_tokens": 1024
  }'

3) Embeddings

/v1/embeddings — supported on hf-inference, nebius, sambanova, scaleway. See Embedding requests in Bifrost docs.

• Most providers: JSON field input
• hf-inference: JSON field inputs (plural)
• Bifrost populates both fields in embedding.go for cross-provider compatibility

curl -X POST http://localhost:8080/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "huggingface/hf-inference/sentence-transformers/all-MiniLM-L6-v2",
    "input": "Hello world"
  }'

4) Speech (Text-to-Speech)

/v1/audio/speech — fal-ai and replicate backends. TTS uses a dedicated JSON body (no pipeline_tag in the request). See Speech in Bifrost docs.

// HuggingFaceSpeechRequest (simplified)
{
  "text": "Hello world",
  "provider": "fal-ai",
  "model": "…",
  "parameters": { }
}

5) Transcriptions (ASR)

/v1/audio/transcriptions — request format depends on the inference provider. See Transcription in Bifrost docs.

curl -X POST http://localhost:8080/v1/audio/transcriptions \
  -F file=@audio.mp3 \
  -F model=huggingface/hf-inference/openai/whisper-large-v3

6) Image Generation

/v1/images/generations — routes by model string. Only fal-ai supports streaming. See Image Generation in Bifrost docs.

fal-ai parameter mappings

curl -X POST http://localhost:8080/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "model": "huggingface/fal-ai/fal-ai/flux/dev",
    "prompt": "A futuristic cityscape at sunset",
    "size": "1024x1024",
    "n": 2,
    "response_format": "url"
  }'

7) Image Edit

Only fal-ai supports image edit; other providers return UnsupportedOperationError. Multipart images are converted to base64 data URLs. Image variation is not supported. See Image Edit in Bifrost docs.

Multi-image models use image_urls; single-image models use image_url. Override with extra_params.use_image_urls. Streaming: SSE with image_edit.partial_image / image_edit.completed.

8) List Models

GET /v1/models — parallel Hub queries per inference provider, filtered by pipeline_tag (chat, feature-extraction, text-to-speech, etc.). Returns IDs as huggingface/{provider}/{model_id}. See List Models in Bifrost docs.

curl http://localhost:8080/v1/models