Google Gemini Provider on Bifrost

1) Chat Completions

Primary path via POST /v1/chat/completions. Upstream: /v1beta/models/{model}:generateContent. Supports multimodal input, tools, thinking, and streaming.

Dropped: logit_biaslogprobstop_logprobsparallel_tool_callsservice_tier.

Tool choice: auto → AUTO, none → NONE, required → ANY. Assistant role maps to model; consecutive tool messages merge into one user message.

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini/gemini-2.0-flash-001",
    "messages": [{"role": "user", "content": "Hello"}],
    "top_k": 40
  }'

2) Responses API

Same upstream generateContent with Responses ↔ Gemini conversion. Gateway: POST /v1/responses.

• Tools: function, computer_use_preview, web_search, mcp
• Streaming emits content_part.added for text and reasoning

curl -X POST http://localhost:8080/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini/gemini-2.0-flash-001",
    "input": "Hello, how are you?",
    "instructions": "You are a helpful assistant."
  }'

3) Speech (TTS)

Text-to-speech via chat generation with responseModalities: ["AUDIO"]. Gateway: POST /v1/audio/speech. Supports streaming.

Gemini returns PCM (s16le, 24kHz, mono); Bifrost converts to WAV when response_format: "wav" (default). Multi-speaker configs supported via multiSpeakerVoiceConfig.

curl -X POST http://localhost:8080/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini/gemini-2.0-flash-001",
    "input": "Hello, welcome to Bifrost.",
    "voice": "Chant-Female"
  }'

4) Transcriptions (STT)

Implemented as chat completion with audio inline data. Gateway: POST /v1/audio/transcriptions. Supports streaming.

curl -X POST http://localhost:8080/v1/audio/transcriptions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini/gemini-2.0-flash-001",
    "file": "<binary-audio-data>",
    "prompt": "Transcribe this audio in the original language."
  }'

5) Embeddings

Single and batch text embeddings. Gateway: POST /v1/embeddings. Upstream: /v1beta/models/{model}:embedContent. Non-streaming.

• embeddings[].values → data[].embedding
• Usage from metadata.billableCharacterCount and token metadata

curl -X POST http://localhost:8080/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini/embedding-001",
    "input": "Hello world",
    "dimensions": 768
  }'

6) Batch API

Inline request arrays or file-based batch input. Gateway maps to OpenAI-style /v1/batches; upstream /v1beta/batchJobs.

• POST /v1beta/batchJobs — create
• GET /v1beta/batchJobs — list (pageToken)
• GET /v1beta/batchJobs/{batch_id} — retrieve
• POST /v1beta/batchJobs/{batch_id}:cancel — cancel

Status mapping includes in_progress, completed, failed, cancelled, expired. Results as inline responses or JSONL file output.

7) Files API

Upload files for batch jobs and multimodal requests. S3-style upload path on Google. Gateway: /v1/files.

• POST /upload/storage/v1beta/files — upload (multipart)
• GET /v1beta/files — list
• GET /v1beta/files/{file_id} — retrieve metadata
• DELETE /v1beta/files/{file_id} — delete
• GET /v1beta/files/{file_id}/content — download

Fields: name → id, displayName → filename, RFC3339 createTime → Unix timestamp.

curl -X POST http://localhost:8080/v1/files \
  -F "file=@document.pdf" \
  -F "filename=document.pdf"

8) Image Generation

Gemini models use :generateContent with responseModalities: ["IMAGE"]. Imagen models use :predict (auto-detected; API key via ?key=). Non-streaming.

curl -X POST http://localhost:8080/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini/imagen-4.0-generate-001",
    "prompt": "A sunset over the mountains",
    "size": "1024x1024",
    "n": 1,
    "output_format": "png"
  }'

9) Image Edit

multipart/form-data only. Gemini and Imagen paths; Imagen supports inpainting, outpainting, inpaint_removal, bgswap. Image variation is not supported.

curl -X POST http://localhost:8080/v1/images/edits \
  -F "model=gemini/gemini-2.0-flash-001" \
  -F "prompt=Add a rainbow in the sky" \
  -F "image[]=@photo.png;type=image/png"

10) List Models

Lists Gemini models with OpenAI-style metadata. Gateway: GET /v1/models. Upstream: GET /v1beta/models with pageSize / pageToken.

• name → id (with gemini/ prefix)
• displayName → name
• inputTokenLimit / outputTokenLimit → max token fields

curl http://localhost:8080/v1/models

11) Video Generation

Veo models via long-running predictLongRunning. JSON body on POST /v1/videos. Poll with GET /v1/videos/{id}; download via /content.

size maps to aspect ratio (e.g. 1280x720 → 16:9). Safety filters may return failed with content_filter.

curl -X POST http://localhost:8080/v1/videos \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini/veo-3.1-generate-preview",
    "prompt": "A calico cat playing piano on stage",
    "seconds": "8",
    "size": "1280x720"
  }'