​
Streaming Chat Responses

Receive AI responses in real-time as they’re generated. Perfect for chat applications where you want to show responses as they’re being typed, improving user experience. 
stream, err := client.ChatCompletionStreamRequest(context.Background(), &schemas.BifrostRequest{
	Provider: schemas.OpenAI,
	Model:    "gpt-4o-mini",
	Input: schemas.RequestInput{
		ChatCompletionInput: &messages,
	},
})

if err != nil {
	log.Printf("Streaming request failed: %v", err)
	return
}

for chunk := range stream {
	// Handle errors in stream
	if chunk.BifrostError != nil {
		log.Printf("Stream error: %v", chunk.BifrostError)
		break
	}

	// Process response chunks
	if chunk.BifrostResponse != nil && len(chunk.BifrostResponse.Choices) > 0 {
		choice := chunk.BifrostResponse.Choices[0]

		// Check for streaming content
		if choice.BifrostStreamResponseChoice != nil &&
			choice.BifrostStreamResponseChoice.Delta.Content != nil {

			content := *choice.BifrostStreamResponseChoice.Delta.Content
			fmt.Print(content) // Print content as it arrives
		}
	}
}
Note: Streaming requests also follow the default timeout setting defined in provider configuration, which defaults to 30 seconds.

​
Text-to-Speech Streaming: Real-time Audio Generation

Stream audio generation in real-time as text is converted to speech. Ideal for long texts or when you need immediate audio playback. 
stream, err := client.SpeechStreamRequest(context.Background(), &schemas.BifrostRequest{
	Provider: schemas.OpenAI,
	Model:    "tts-1", // Using text-to-speech model
	Input: schemas.RequestInput{
		SpeechInput: &schemas.SpeechInput{
			Input: "Hello! This is a sample text that will be converted to speech using Bifrost's speech synthesis capabilities. The weather today is wonderful, and I hope you're having a great day!",
			VoiceConfig: schemas.SpeechVoiceInput{
				Voice: bifrost.Ptr("alloy"),
			},
			ResponseFormat: "mp3",
		},
	},
})

if err != nil {
	panic(err)
}

// Handle speech synthesis stream
var audioData []byte
var totalChunks int
filename := "output.mp3"

for chunk := range stream {
	if chunk.BifrostError != nil {
		panic(fmt.Sprintf("Stream error: %s", chunk.BifrostError.Error.Message))
	}

	if chunk.BifrostResponse != nil && chunk.BifrostResponse.Speech != nil {
		// Accumulate audio data from each chunk
		audioData = append(audioData, chunk.BifrostResponse.Speech.Audio...)
		totalChunks++
		fmt.Printf("Received chunk %d, size: %d bytes\n", totalChunks, len(chunk.BifrostResponse.Speech.Audio))
	}
}

if len(audioData) > 0 {
	// Save the accumulated audio to a file
	err := os.WriteFile(filename, audioData, 0644)
	if err != nil {
		panic(fmt.Sprintf("Failed to save audio file: %v", err))
	}

	fmt.Printf("Speech synthesis streaming complete! Audio saved to %s\n", filename)
	fmt.Printf("Total chunks received: %d, final file size: %d bytes\n", totalChunks, len(audioData))
}

​
Speech-to-Text Streaming: Real-time Audio Transcription

Stream audio transcription results as they’re processed. Get immediate text output for real-time applications or long audio files. 
// Read the audio file for transcription
audioFilename := "output.mp3"
audioData, err := os.ReadFile(audioFilename)
if err != nil {
	panic(fmt.Sprintf("Failed to read audio file %s: %v. Please make sure the file exists.", audioFilename, err))
}

fmt.Printf("Loaded audio file %s (%d bytes) for transcription...\n", audioFilename, len(audioData))

stream, err := client.TranscriptionStreamRequest(context.Background(), &schemas.BifrostRequest{
	Provider: schemas.OpenAI,
	Model:    "whisper-1", // Using Whisper model for transcription
	Input: schemas.RequestInput{
		TranscriptionInput: &schemas.TranscriptionInput{
			File:   audioData,
			Prompt: bifrost.Ptr("This is a sample audio transcription from Bifrost speech synthesis."), // Optional: provide context
		},
	},
})

if err != nil {
	panic(err)
}

for chunk := range stream {
	if chunk.BifrostError != nil {
		panic(fmt.Sprintf("Stream error: %s", chunk.BifrostError.Error.Message))
	}

	if chunk.BifrostResponse != nil && chunk.BifrostResponse.Transcribe != nil {
		// Print each chunk of text as it arrives
		fmt.Print(chunk.BifrostResponse.Transcribe.Text)
	}
}

​
Streaming Best Practices

​
Buffering for Audio

For audio streaming, consider buffering chunks before saving: 
const bufferSize = 1024 * 1024 // 1MB buffer

var audioBuffer bytes.Buffer
var lastSave time.Time

for chunk := range stream {
	if chunk.BifrostResponse != nil && chunk.BifrostResponse.Speech != nil {
		audioBuffer.Write(chunk.BifrostResponse.Speech.Audio)
		
		// Save every second or when buffer is full
		if time.Since(lastSave) > time.Second || audioBuffer.Len() > bufferSize {
			// Append to file
			file, err := os.OpenFile("streaming_audio.mp3", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
			if err == nil {
				file.Write(audioBuffer.Bytes())
				file.Close()
				audioBuffer.Reset()
				lastSave = time.Now()
			}
		}
	}
}

​
Context and Cancellation

Use context to control streaming duration: 
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

stream, err := client.ChatCompletionStreamRequest(ctx, &schemas.BifrostRequest{
	// ... your request
})

// Stream will automatically stop after 30 seconds

​
Voice Options

OpenAI TTS supports these voices:
  • alloy - Balanced, natural voice
  • echo - Deep, resonant voice
  • fable - Expressive, storytelling voice
  • onyx - Strong, confident voice
  • nova - Bright, energetic voice
  • shimmer - Gentle, soothing voice
// Different voice example
VoiceConfig: schemas.SpeechVoiceInput{
    Voice: bifrost.Ptr("nova"),
},
Note: Please check each model’s documentation to see if it supports the corresponding streaming features. Not all providers support all streaming capabilities.

​
Next Steps