Google Vertex AI Provider on Bifrost

1) Chat Completions

Primary chat path. Bifrost detects Gemini vs Anthropic from the model name and converts to the appropriate Vertex format. Upstream: /generate (Gemini) or Anthropic message format (Claude). See Chat Completions in Bifrost docs.

Gemini models

System prompts, tool usage, and streaming map to Gemini formats. See Gemini provider docs.

Anthropic models (Claude)

• Reasoning parameters convert to thinking structure
• System messages extracted to a separate system field
• API version set to vertex-2023-10-16; minimum reasoning budget 1024 tokens
• Model field removed from upstream request (Vertex uses different identification)

Region selection

Streaming: Gemini uses SSE; Anthropic uses Anthropic message streaming. Configure vertex_key_config with project_id, region, and auth_credentials (or leave empty for ADC).

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "vertex/gemini-2.0-flash",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

2) Responses API

Available for both Gemini and Anthropic (Claude) models on Vertex. Upstream routes to /messages for Claude. See Responses API in Bifrost docs.

• Anthropic: endpoint /v1/messages; anthropic_version set to vertex-2023-10-16
• Model and region fields removed from upstream request; raw body passthrough supported
• Gemini: conversion follows Gemini Responses API format

curl -X POST http://localhost:8080/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "vertex/claude-3-5-sonnet",
    "input": "What is AI?",
    "instructions": "You are a helpful assistant"
  }'

3) Embeddings

Supported for Gemini and other embedding-capable models. Upstream /embeddings — no streaming. Use extra_params for task-specific options. See Embeddings in Bifrost docs.

Advanced parameters (extra_params)

Response includes values, statistics.token_count, and statistics.truncated. Bifrost preserves float64 precision from Vertex.

curl -X POST http://localhost:8080/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "vertex/text-embedding-004",
    "input": ["text to embed"],
    "dimensions": 256,
    "task_type": "RETRIEVAL_DOCUMENT",
    "title": "Document title",
    "autoTruncate": true
  }'

4) Image Generation

Supported for Gemini and Imagen. Bifrost auto-detects model type and routes to /generateContent or /predict. Streaming is not supported. See Image Generation in Bifrost docs.

• Gemini: same conversion as Gemini Image Generation
• Imagen: Imagen-specific format via IsImagenModel()
• Fine-tuned: .../endpoints/{deployment}:generateContent
• Region field removed from request body before upstream call

curl -X POST http://localhost:8080/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "model": "vertex/imagen-4.0-generate-001",
    "prompt": "A sunset over the mountains",
    "size": "1024x1024",
    "n": 2
  }'

5) Image Edit

Uses multipart/form-data, not JSON. Supported for Gemini and Imagen only; other models return ConfigurationError. Image variation is not supported. See Image Edit in Bifrost docs.

Conversion matches Gemini Image Edit. Gemini strips unsupported fields; Imagen supports mask modes and dilation. Streaming not supported.

curl -X POST http://localhost:8080/v1/images/edits \
  -F "model=vertex/imagen-3.0-generate-002" \
  -F "prompt=Add sunglasses to the person" \
  -F "image[]=@photo.png"

6) List Models

GET /v1/models — uses project_id and region from key config. No request parameters required. See List Models in Bifrost docs.

Vertex's List Models API returns only custom fine-tuned models (digit-only deployment IDs). Bifrost performs three-pass discovery to include foundation models:

1. Custom models from the Vertex API response
2. Foundation models from your aliases configuration
3. Models in the key-level models allowlist not already in aliases

• Empty models and no aliases → no models returned
• models: ["*"] → all passes included
• Non-empty models → filtered to allowlist; duplicates prevented
• Pagination handled internally when next_page_token is present

curl http://localhost:8080/v1/models

Example response shape

{
  "models": [
    {
      "name": "projects/{project}/locations/{region}/models/gemini-2.0-flash",
      "display_name": "Gemini 2.0 Flash",
      "description": "Fast multimodal model",
      "version_id": "1",
      "version_aliases": ["latest", "stable"],
      "capabilities": [...],
      "deployed_models": [...]
    }
  ],
  "next_page_token": "..."
}

7) Video Generation

Veo models only via /predictLongRunning. Parameters match Gemini Video Generation. See Video Generation in Bifrost docs.

curl -X POST http://localhost:8080/v1/videos \
  -H "Content-Type: application/json" \
  -d '{
    "model": "vertex/veo-2.0-generate-001",
    "prompt": "A bird flying through clouds"
  }'