Protocol Reference

The ai-proxy and ai-proxy-multi plugins share the same request protocol detection and conversion pipeline. The pipeline identifies the client format before routing to the configured provider or selected instance.

For plugin-specific configuration, see ai-proxy and ai-proxy-multi.

Request Protocol Detection​

The plugins identify the client protocol before matching it to a protocol supported by the selected provider or instance. The following detection rules apply to both plugins.

Request Requirements​

Requests that include Content-Type must use application/json. If the header is omitted, the plugins treat the body as JSON. They reject unsupported content types and invalid bodies before selecting a provider or instance.

The request body cannot exceed max_req_body_size, which defaults to 67,108,864 bytes. A request that exceeds this limit receives an HTTP 413 response. In API7 Gateway, this setting is available from 3.9.14 in the 3.9.x line and from 3.10.1 in the 3.10.x line. It is available in APISIX 3.17.0 and later.

Detection Order​

The plugins check the following rules in order:

The path-specific rules run before the body-only rules. This prevents Bedrock Converse and Anthropic Messages requests containing messages from being identified as Chat Completions. Responses and Embeddings requests both use input, so a request containing input but not messages is identified as Embeddings unless its path ends in /v1/responses.

Any other non-empty JSON object is treated as passthrough. This mode keeps the original request path and can reuse the original body when no request transformation changes it. Provider authentication and override.endpoint still apply. Empty or invalid request bodies are rejected.

After Detection​

For a named protocol, the plugin uses the detected protocol without conversion when the selected provider supports it. Otherwise, the plugin looks for a registered converter to a protocol the provider supports. The request is rejected if neither native support nor a compatible converter is available.

The detected protocol and request body determine whether request_type is recorded as ai_stream or ai_chat for logging plugins. Response parsing uses the upstream response's content type to distinguish streaming from non-streaming responses.

Anthropic-to-OpenAI Conversion​

An Anthropic Messages client can send requests through ai-proxy or ai-proxy-multi to a backend that supports OpenAI Chat Completions. The plugins convert the client request to OpenAI format and convert the backend response to Anthropic format. This conversion supports a subset of the Anthropic Messages API: it preserves some fields, transforms others, and discards unsupported fields.

When Conversion Applies​

The plugins identify an Anthropic Messages request when the request path ends in /v1/messages and the body is a JSON object. If the selected provider supports Anthropic Messages, the request uses that protocol without conversion. If the provider supports OpenAI Chat Completions instead, the shared converter translates the request and response.

The reverse client/backend pairing is not supported: an OpenAI Chat Completions client cannot use this converter to call an Anthropic Messages backend.

For ai-proxy-multi, the selected instance determines whether conversion is required. The conversion configuration example uses ai-proxy; for multi-instance configuration, see ai-proxy-multi.

Request Conversion​

The converted request body is built from an allowlist. The following tables summarize the Anthropic inputs the converter reads. Unrecognized fields are discarded before the request reaches the backend.

Request Fields​

Message Content​

Dropped Fields​

The backend does not receive the following fields, and the response carries no signal that they were removed:

Request Headers​

If a request has an x-api-key header but no Authorization header, the converter sends the key as a bearer token in Authorization. It removes the original x-api-key header and headers whose names start with anthropic- or x-stainless-.

Response Conversion​

The converter reads completion fields only from choices[0]; any additional OpenAI choices are discarded. It maps top-level usage and error fields separately.

For streaming responses, the converter emits Anthropic message and content-block events for OpenAI text, reasoning, and tool-call deltas. The initial message_start usage values are zero; final token usage is emitted in message_delta. Available cache-token fields can be included when the backend supplies them in a supported usage chunk. Clients that report streaming usage should read the final event and should not assume that cache-token fields are present.

Release Compatibility​

API7 Gateway 3.9.x and 3.10.x receive fixes independently. Check the column for the release line you run.

These version differences affect structured output, message content, thinking effort, and tool history as follows:

Structured Output​

API7 Gateway 3.9.15 and earlier, API7 Gateway 3.10.0 through 3.10.2, and APISIX 3.17.0 recognize only the converter's legacy expected shape. That shape is output_config or output_format carrying type: json_schema together with a json_schema field, or carrying type: json or type: json_object.

Anthropic's current shape carries the schema in output_format.schema or in output_config.format. In the 3.9.x line, API7 Gateway 3.9.16 and later recognize this shape. In the 3.10.x line, 3.10.3 and later recognize it. These releases normalize the schema and send response_format with strict mode enabled. On earlier releases, the backend receives no response_format, and the client receives no error.

Message Content Shape​

On a release that predates the change, a user message carrying a single text block is sent as a plain string. An assistant message carrying several text blocks is sent as a content array. A backend that accepts only one of these shapes behaves differently across an upgrade.

Thinking Effort​

The budget_tokens mapping changes across releases. The earlier mapping applies to APISIX 3.17.0 and API7 Gateway releases before 3.9.16 or 3.10.3 in their respective lines. The later mapping applies from API7 Gateway 3.9.16 and 3.10.3 in their respective lines.

Tool History​

On an earlier release, a tool_use name in message history is not rewritten with the corresponding declared tool name. Ordinary text can also be sent before tool_result messages from the same user message, and media in that message is discarded. A strict backend may reject the name or ordering mismatch. API7 Gateway 3.9.16 and later and API7 Gateway 3.10.3 and later rewrite history names consistently, place tool messages first, and retain media.

Known Limitations​

The following limitations can affect converted requests and responses across the supported releases.

Streaming Can End Without a Terminating Event​

When a backend closes the stream without a properly delimited final frame, the plugins do not emit the closing message_delta and message_stop events. A client that relies on message_stop may wait indefinitely or treat the stream as incomplete. This affects every release listed above. Set a client-side timeout and treat an unexpected end of the stream as a failure.

Converted thinking Blocks Do Not Carry a Valid Signature​

For a non-streaming response, the plugins set the block's signature to an empty string. For a streaming response, they emit thinking deltas without a signature. Clients that require a valid signature cannot replay either converted form as a signed Anthropic thinking block. A backend that instead embeds reasoning in ordinary message content produces a response where the reasoning appears as visible text.

Error Responses Are Not Consistently Anthropic-Shaped​

HTTP 429, 5xx, and transport timeout responses can bypass response conversion. Clients should be prepared to receive an upstream or gateway error body that does not follow the Anthropic error schema.

Backend Capabilities Are Not Validated​

The plugins convert the request but do not check whether the backend model supports the result. A backend can return HTTP 200 while silently ignoring a capability, such as dropping an image, ignoring response_format, or returning no tool call.

Backend behavior varies by model and between dated snapshots of the same model name. Validate the specific models you plan to use rather than generalizing from a backend.

Validate Backend Compatibility​

Test the following converted inputs against each backend model because a backend can reject them even when the original Anthropic request is valid:

• A named tool_choice. The converter emits an object naming the function, or "required". Some backends accept only "auto" while the model is reasoning. If a backend rejects the converted form, send {"type": "auto"} from the client.
• thinking together with a small max_tokens. thinking becomes reasoning_effort, which can make a backend reserve a reasoning budget. When that budget exceeds the converted max_completion_tokens, the backend rejects the request. Raise max_tokens when you enable thinking.
• A document block. The converter can only offer it to the backend as an image. A model that cannot read it may answer with invented content instead of reporting an error.
• Mixed text, media, and tool_result content. Earlier releases can put ordinary text before the converted tool message and discard media from the same user message. Test this shape if the backend validates tool-message ordering, or upgrade to a release that places tool messages first and retains media.

Related Configuration​

• Configure ai-proxy to convert Anthropic requests.
• Configure native Anthropic Messages pass-through.
• Configure ai-proxy-multi.
• Convert Anthropic Messages with API7 Gateway.