OpenAI Responses API vs Chat Completions vs Anthropic Messages

TL;DR

• The OpenAI Responses API is OpenAI's recommended interface for new projects: it uses typed input and output items, supports server-side conversation state, and runs built-in tools in one request.
• Chat Completions remains supported and is the de facto OpenAI-compatible format, but it is stateless and has narrower support for reasoning models with tools.
• The Anthropic Messages API is stateless like Chat Completions, but structures content as typed blocks (text, tool_use, tool_result, thinking) with a top-level system prompt and a required max_tokens.
• Most production teams use more than one of these formats, because coding agents and SDKs each default to a different one.
• Bifrost accepts Chat Completions, Responses, and Anthropic Messages requests and routes them to 25+ providers, so the API choice no longer locks in the model provider.

The OpenAI Responses API, Chat Completions, and the Anthropic Messages API are the three request formats most LLM applications and agents are built on, and they differ in how they handle conversation state, tool calls, and reasoning. OpenAI now recommends the Responses API for new projects, Anthropic's ecosystem (including Claude Code) speaks Messages, and most of the industry still treats Chat Completions as the compatibility baseline. Bifrost, the open-source AI gateway that speaks all three formats, is built for enterprises running mission-critical AI workloads that need any of these APIs to reach any provider with one governance policy. This guide compares the three APIs field by field and explains when each one fits.

What Is the OpenAI Responses API?

The OpenAI Responses API is OpenAI's unified interface for model generation that represents inputs and outputs as typed items, can store conversation state on OpenAI's side, and can call built-in tools such as web search, file search, code interpreter, and remote MCP servers within a single request.

OpenAI's migration guide to the Responses API states that Chat Completions remains supported but that Responses is recommended for all new projects. The same guide lists the main design changes:

• Items instead of messages: input and output are arrays of typed items, where a message is one item type alongside reasoning, function_call, and function_call_output
• Top-level instructions: system guidance moves to an instructions field, and input can be a plain string
• Stateful context: with store: true, a request can reference a prior turn through previous_response_id, or use the Conversations API for a persistent conversation object
• Agentic loop by default: the model can call several built-in tools and custom functions within one API request
• Encrypted reasoning: stateless deployments can pass back encrypted reasoning items instead of storing state

OpenAI also reports internal results for the switch: a 3% SWE-bench improvement for reasoning models with the same prompt, and 40% to 80% better cache utilization than Chat Completions in internal tests. Teams already on OpenAI can use any LLM provider with the OpenAI SDK without leaving this request shape. The Responses API replaced the Assistants API, which OpenAI's deprecations page lists as removed on August 26, 2026, with Responses and Conversations as the replacement.

What Is the Chat Completions API?

The Chat Completions API is OpenAI's stateless conversation interface, where each request sends a messages array of role-tagged messages (system, developer, user, assistant, tool) and receives a list of choices. It is the most widely copied LLM request format and the basis of most OpenAI-compatible APIs.

Chat Completions remains the default for a large share of existing code, frameworks, and inference servers. Its strengths are simplicity and portability: the client owns the transcript, so any provider that implements the format can serve the request, which is why so many gateways and self-hosted engines expose an OpenAI-compatible API.

Its limits show up in agentic and reasoning workloads. OpenAI's migration guide notes that conversation state must be managed manually, and that starting with GPT-5.4, Chat Completions does not support tool calling with reasoning_effort values other than none. Teams building tool-heavy agents on newer OpenAI reasoning models therefore have a practical reason to move to Responses even without needing stored state.

What Is the Anthropic Messages API?

The Anthropic Messages API is Anthropic's stateless interface for Claude models, where each request sends a messages array of user and assistant turns, each holding typed content blocks, plus a top-level system prompt and a required max_tokens. Responses return content blocks such as text, tool_use, and thinking.

Anthropic's Messages API reference describes it as usable for single queries or stateless multi-turn conversations. Key characteristics:

• Content blocks: text, images, documents, tool_use, tool_result, and thinking blocks rather than plain string content
• Top-level system prompt: system instructions sit in a system field, not in the messages array
• Required max_tokens: every request sets an explicit output ceiling
• Client-held state: the full transcript is resent each turn, as with Chat Completions
• Server tools and beta headers: features such as web search or code execution are declared as tools, and newer capabilities are often gated by the anthropic-beta header

The Messages API matters beyond direct Claude integrations because Claude Code and the Claude Agent SDK send every model call in this format. A gateway in front of those tools has to speak Messages natively, which is how Bifrost serves Claude Code through its Anthropic-compatible endpoint. The guide to using Claude Code with OpenAI models via an AI gateway shows the translation running in the other direction.

Responses API vs Chat Completions vs Messages at a Glance

The three APIs differ most in state handling, content structure, and tool execution: Responses can keep state and run built-in tools server-side, while Chat Completions and Messages are stateless and leave the loop to the client. This table maps the equivalent concepts.

For multi-provider teams, the table explains why a single format rarely holds up in production: each provider's newest features appear first in its own native API. A survey of multi-provider AI gateways for OpenAI, Anthropic, and Bedrock shows how gateways bridge that gap.

Chat Completions vs Responses API: Key Differences

The key differences between Chat Completions and the Responses API are state, structure, and tool execution: Responses adds optional server-side state, replaces messages with typed items, defaults function schemas to strict mode, and runs built-in tools inside one request, while Chat Completions keeps the client in charge of everything.

Figure 1: Only the Responses API can hold conversation state on the provider side; the other two keep the client as the source of truth.

Three differences have the largest migration impact:

• Multi-turn handling: with previous_response_id, the client sends only the new input, but OpenAI notes that all previous input tokens in the chain are still billed as input tokens, so stored state reduces payload size rather than token cost
• Function definitions: Chat Completions nests the schema under a function key; Responses flattens it and attempts strict mode unless told otherwise
• Output handling: code that reads choices[0].message.content must instead read the output_text helper or iterate output items by type

Teams moving coding tools between the two formats see this directly, and managing OpenAI rate limits at scale becomes part of the same migration. Codex CLI, for example, speaks the Responses API, which is why the guide to running Codex CLI with multiple model providers starts from a Responses-compatible endpoint.

OpenAI Responses API vs Anthropic Messages API

The OpenAI Responses API and the Anthropic Messages API both use typed content and both support reasoning and tool calls, but Responses can hold state on the provider side while Messages is always stateless. Messages also requires max_tokens and keeps tool results inside the user turn as tool_result blocks.

The structural similarities make translation between them more direct than either is to Chat Completions. Both represent a tool call as a typed object with an ID, both return reasoning as a distinct content type, and both separate system guidance from the conversation. The differences that matter in practice:

For agent builders, the more important question is often which client format a tool already speaks, because coding agents rarely let the user choose. The overview of using Bedrock, Vertex, Gemini, and Anthropic models through Bifrost shows how one client format can reach several provider APIs.

State, Storage, and Data Retention

State handling is also a data-retention decision. Responses are stored by default on OpenAI, Chat Completions are stored by default for new accounts, and the Anthropic Messages API keeps no conversation object, so each choice changes what leaves your environment and what persists afterward.

OpenAI's guide documents the controls: set store: false on either OpenAI API to disable storage, and for organizations with Zero Data Retention requirements, OpenAI enforces store: false automatically and supports encrypted reasoning items that are decrypted in memory and discarded after use. Teams with strict retention policies should treat stored state as an explicit opt-in rather than a default.

A gateway adds a second retention layer that the platform team controls. Bifrost request logging captures Chat Completions, Responses, and Messages traffic, and it can disable content logging while keeping usage metadata (tokens, cost, latency) for cost reporting. On Bifrost Enterprise, log exports offload payloads to S3 or GCS under the organization's own retention rules, and in-VPC deployments keep the gateway inside a private network.

Tool Calling Across the Three APIs

All three APIs support function calling with JSON Schema definitions, but they differ in where tool results go, how strictly schemas are enforced, and whether the provider can execute tools itself. Responses and Messages also support provider-executed tools and MCP, while Chat Completions relies mostly on client-executed functions.

The practical rule for agent builders is that the client-executed loop (model proposes a call, client runs it, client returns the result) works the same way conceptually in all three; only the field names change. Provider-executed tools differ more, because they depend on what each provider hosts. For tools the organization controls, an MCP gateway gives every format the same tool catalog: Bifrost connects to MCP servers and can inject their tools into requests, with per-key MCP tool filtering deciding which tools each consumer sees. The Bifrost MCP gateway resource page covers the full setup.

Which API Should You Use?

Use the Responses API for new OpenAI-first applications, especially agents on reasoning models; keep Chat Completions for stable code and broad compatibility; use the Anthropic Messages API for Claude-first applications. When several providers are involved, standardize on one format behind a gateway instead of choosing per provider.

Figure 2: The API choice follows the primary provider and the age of the codebase, and a gateway removes the need to choose per provider.

The cost of the wrong choice is mostly migration effort, not capability. Teams that route between OpenAI, Anthropic, and Gemini usually pick the format their primary SDK already uses and let the gateway handle the rest.

Running All Three APIs Through One Gateway

Bifrost runs all three APIs through one gateway by exposing an OpenAI-compatible endpoint for Chat Completions and Responses and an Anthropic-compatible endpoint for Messages, then adapting each request to the target provider. A client changes only its base URL, and the model name selects the provider.

Figure 3: Each team keeps the SDK and request shape it already uses, while Bifrost adapts the protocol and applies one governance policy.

How each format maps onto Bifrost:

• Chat Completions and Responses: the OpenAI SDK integration serves both from the /openai endpoint, and a prefixed model name such as anthropic/... or vertex/... sends the same request to another provider
• Anthropic Messages: the Anthropic SDK integration serves Messages from the /anthropic endpoint, including requests that target non-Anthropic models by prefix
• Provider coverage: the supported providers matrix lists Responses API support, including streaming, for providers such as Anthropic, Bedrock, Gemini, and Azure across 25+ providers and 10,000+ models
• Coding agents: Codex CLI connects through a custom Responses provider and Claude Code through the Messages endpoint

Governance applies identically whichever format arrives. Virtual keys carry allowed providers, models, budgets, and rate limits; automatic fallbacks fail over across providers when one returns errors, the pattern behind staying online when OpenAI goes down. Features that depend on a specific provider's servers, such as OpenAI's stored responses or Anthropic's server tools, still depend on that provider being the upstream.

The broader case for a protocol-translating layer is covered in what an LLM gateway is and how it works, and Bifrost adds 11 microseconds of overhead per request at 5,000 RPS in published benchmarks.

Frequently Asked Questions

What is the difference between the Responses API and Chat Completions?

The Responses API represents input and output as typed items, can store conversation state with previous_response_id, runs built-in tools inside one request, and defaults function schemas to strict mode. Chat Completions uses a stateless messages array, leaves state to the client, and is the format most OpenAI-compatible providers implement. OpenAI recommends Responses for new projects.

Is OpenAI deprecating Chat Completions?

No. OpenAI's migration guide states that Chat Completions remains supported, while recommending the Responses API for all new projects. The API OpenAI did retire is the Assistants API, removed on August 26, 2026, with the Responses API and Conversations API as its replacement. New features tend to arrive in Responses first.

Is the Anthropic Messages API compatible with the OpenAI API?

Not directly. The Messages API uses content blocks, a top-level system field, tool_use and tool_result blocks, and a required max_tokens, so OpenAI SDK requests do not work against it unchanged. A gateway such as Bifrost adapts between the formats, so an OpenAI SDK client can call Claude models and an Anthropic SDK client can call OpenAI models, as covered in what an LLM gateway does.

Does the Responses API cost more than Chat Completions?

The API choice mostly changes how many tokens are billed rather than their rate. OpenAI reports 40% to 80% better cache utilization with Responses in internal tests, which can lower cost. Chaining with previous_response_id still bills all prior input tokens as input, so stored state reduces payload size, not token charges.

Can I use the Responses API with Claude models?

Yes, through a gateway that translates the format. Bifrost supports the Responses API for Anthropic, Bedrock, and other providers, so a Responses request with a prefixed model name reaches Claude. Features specific to OpenAI's servers, such as stored responses and OpenAI-hosted tools, depend on OpenAI being the upstream provider.

Which API do coding agents use?

Coding agents use whichever format their vendor chose. Claude Code and the Claude Agent SDK use the Anthropic Messages API through ANTHROPIC_BASE_URL, Codex CLI uses the OpenAI Responses API, and many editors and open-source agents use Chat Completions. The Bifrost CLI agents overview lists the endpoint each one uses. A gateway that serves all three lets one organization govern every agent from a single policy.

Use Any of the Three APIs with Bifrost

The OpenAI Responses API, Chat Completions, and the Anthropic Messages API will coexist in most organizations for years, because SDKs and coding agents each default to a different one. Routing all three through the Bifrost AI gateway turns the API choice into a client-side detail, with one set of keys, budgets, fallbacks, and logs behind it. To see Bifrost serving your existing SDKs across providers, book a demo with the Bifrost team.