Responses API

The Responses API is OpenAI's response-generation endpoint for applications that use its input/output item format instead of the chat-completions message format. AISIX AI Gateway exposes this route for Responses API clients while keeping caller authentication, model aliases, upstream credentials, and gateway policy in the gateway.

Use this route when an application or tool already speaks the Responses API. When the model's upstream serves the Responses API itself, AISIX forwards the request there. Otherwise AISIX translates the request through the provider adapter and returns a Responses API result to the caller.

Prerequisites

Before starting, prepare the following:

• A running AISIX gateway that can serve proxy requests.
• A caller API key that can access the model alias.
• A model alias backed by an upstream that serves the Responses API natively or supports the translated request shape.

Export the gateway connection and request values:

# AISIX_PROXY has no trailing slash or endpoint path such as /v1.
# The local quickstarts use http://127.0.0.1:3000.
export AISIX_PROXY="YOUR_AISIX_GATEWAY_URL"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="gpt-4o-prod"

Send a Responses Request

Send the request through the gateway proxy with the AISIX model alias in the request body:

curl -sS -X POST "${AISIX_PROXY}/v1/responses" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"${AISIX_MODEL}"'",
    "input": "Say hello from AISIX."
  }'

AISIX resolves the model alias and chooses the provider path for the selected model. When the provider key selects a native Responses surface, AISIX forwards the request without body translation. Otherwise, it uses the cross-provider bridge.

The response body is returned in the upstream Responses API format:

{
  "id": "resp_***",
  "object": "response",
  "model": "gpt-4o-prod",
  "output": [
    {
      "type": "message",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "Hello from AISIX."
        }
      ]
    }
  ],
  "usage": {
    "input_tokens": 12,
    "output_tokens": 5,
    "total_tokens": 17
  }
}

The example shows only the fields this page discusses. On the bridged path, a reasoning-capable upstream leads with a reasoning item ahead of this message item, and the Response object carries the full top-level field set of the Responses API. See What the Bridge Returns.

Provider Behavior

AISIX handles Responses requests in two ways:

Upstream	Behavior
Serves the Responses API	AISIX rewrites the request model to the upstream model ID and forwards the request to the upstream Responses API. Upstream-specific Responses features pass through when the upstream supports them.
Does not serve it	AISIX translates supported Responses fields into the gateway's chat format, dispatches through the provider adapter, and returns a Responses API result.

Which one applies is decided per provider key. Without a surface declaration, AISIX forwards natively for the openai provider and translates for every other provider. That default is wrong in both directions for some endpoints: an OpenAI-compatible endpoint reached through the openai provider with a custom api_base may have no /v1/responses route, and an endpoint on another provider may serve one. Declare the key's API surfaces to say which it is — see Declare the API Surfaces.

The bridge supports common text, tool call, tool result, sampling, and streaming fields, plus the message parts, tool parameters, and structured-output format described below. It carries reasoning.effort into the canonical chat request as reasoning_effort; provider adapters that support an effort control then translate it to their wire field. For example, Anthropic receives output_config.effort. You can also rewrite the value per direct model with Reasoning Effort Mapping. A Responses feature that the chat format cannot express is dropped rather than flattened onto the upstream body, so the request still reaches the provider instead of being rejected by it; Fields Ignored on Translated Requests lists what falls in that set.

Message Parts

Besides the text parts, the bridge carries the non-text parts of a message into their chat content-block counterparts:

Responses part	Chat content block
input_image	image_url, with the image addressed by a URL or a data: URL, and detail carried when the caller sent one
input_file	file, carrying whichever of file_data, filename, and file_id the caller sent
input_audio	input_audio, carrying data and format

A message slot that produces any non-text block is sent as an array content. A text-only slot keeps the bare-string shape it has always had.

Tool Parameters

• parallel_tool_calls is forwarded as the caller sent it, whenever at least one tool survives translation.
• Every tool_choice form is normalized to the provider-neutral chat shape. auto, none, required, and {"type": "function", "name": ...} are unchanged. {"type": "any"} becomes required. {"type": "custom", "name": ...} and {"type": "tool", "name": ...} become the named function. {"type": "allowed_tools", "mode": ...} keeps only its mode, and the tool subset it names is dropped, because a chat upstream cannot be restricted to a subset of the tools it was given.
• A custom (freeform) tool becomes a function tool that takes one required string parameter, with the tool's grammar folded into that parameter's description, which is the only place a chat upstream reads it.
• A namespace tool, such as the multi_agent_v1 tool Codex sends, is flattened: each of its function sub-tools is offered to the model as a function tool named <namespace>__<tool>, described by the namespace's description followed by the sub-tool's own. A sub-tool whose flattened name collides with a top-level function tool is not offered. A replayed function_call that carries a namespace, and a tool_choice that names a sub-tool with its namespace, are sent upstream under the flattened name.
• A tool result whose output is a JSON object, number, or boolean reaches the upstream as that value serialized to a string, because a chat tool message carries a string. null and an absent output stay the empty string.

When no tool survives translation, such as a context-compaction request carrying "tools": [] or a request carrying only hosted tools, tool_choice and parallel_tool_calls are both dropped, including the forms that force a call. Neither key is sent to an upstream that rejects it without a tools list, and the request proceeds as an ordinary tool-free call.

Structured Output

text.format becomes the chat response_format. A json_schema format becomes {"type": "json_schema", "json_schema": {...}}, carrying whichever of name, schema, strict, and description the caller sent; json_object maps straight across; a text format, or no text member at all, sends nothing.

This is honored by OpenAI-compatible chat upstreams, and by Anthropic, Gemini, and Bedrock upstreams, each of which has its own structured-output field. Those mappings differ in what the model supports and in what the provider accepts inside a schema: see Structured Output for Anthropic, Structured Output on Gemini, and Structured Output for Bedrock.

Replayed Conversation History

Clients such as Codex resend the whole conversation on every turn, including the model's earlier reasoning and tool calls. The bridge rebuilds the chat messages from those items as follows:

• Reasoning items. A reasoning input item reaches the upstream as reasoning_content on the assistant message it belongs to. The text comes from the item's content parts, or from its summary parts when content carries no text. encrypted_content is never read, and an item with no readable text replays nothing. Consecutive reasoning items are joined with newlines. Because this reasoning used to be dropped, multi-turn requests that replay it now send more input tokens upstream.
• Reasoning with no answer after it. Reasoning that no assistant content follows is sent as an assistant message of its own, carrying only reasoning_content. Upstreams reached through the anthropic, bedrock, and vertex adapters have no field for replayed reasoning: they ignore it on an assistant message that also carries text or tool calls, and skip an assistant message whose only payload is reasoning_content.
• Assistant text followed by tool calls. An assistant message item and the function_call or custom_tool_call items right after it are sent as one assistant message carrying both content and tool_calls, the shape the model produced them in, rather than as two consecutive assistant messages.

reasoning_content is sent to every OpenAI-compatible upstream. An OpenAI-compatible upstream whose request schema rejects unknown message fields rejects a request that replays reasoning; there is no setting that turns the replay off.

What the Bridge Returns

The Response object. Every Response the bridge emits, the non-streaming body and the response inside every streamed lifecycle event (response.created, response.in_progress, response.completed, response.incomplete, and response.failed), carries the full top-level field set the Responses API defines. That includes tools, tool_choice, parallel_tool_calls, text, reasoning, instructions, temperature, top_p, max_output_tokens, truncation, store, metadata, completed_at, error, incomplete_details, and usage. A field echoes the request's value when the request set it, and otherwise carries the API's default or null. In the tools echo, a function tool without description, parameters, or strict gets null for the missing ones; other tool types are echoed as sent. The native Responses path returns whatever the upstream returns.

Some output items on the bridged path do not come from an upstream that speaks Responses. AISIX builds them from the chat reply.

Reasoning. When a chat upstream reports its chain of thought as reasoning_content, AISIX returns it as a reasoning output item placed ahead of the message item. On a non-streaming reply, the item carries the text as summary[{"type": "summary_text"}]. On a stream, it arrives as response.output_item.added, then response.reasoning_summary_part.added, then one or more response.reasoning_summary_text.delta events, then the matching done events. The message item opens at the next output_index.

Custom tool calls. A model that calls a bridged custom tool returns a custom_tool_call item carrying the freeform text as input, never a function_call item: the single string parameter the request side wrapped it in is unwrapped again. On a stream, the item carries exactly one response.custom_tool_call_input.delta with that input, followed by response.custom_tool_call_input.done.

Namespace tool calls. A model that calls a flattened namespace sub-tool returns a function_call item whose name is the sub-tool's own name and which carries the namespace it belongs to, on both the non-streaming and streaming paths. A bare sub-tool name that identifies exactly one sub-tool is mapped back the same way.

Stream Failures on the Bridged Path

When a bridged stream fails after it has started, AISIX sends the flat error event, then a response.failed event. The response.failed event carries a full Response object with status: "failed", output: [], usage: null, and error: {"code": ..., "message": ...}. If a terminal event has already reached the client, response.failed is not sent. When the failure is the first thing the client would receive, AISIX opens the stream first: response.created and response.in_progress, both carrying the Response object with status: "in_progress" and an empty output, then error and response.failed, numbered 0 to 3 by sequence_number. This covers an upstream that fails before its first chunk, an empty upstream stream, and an output guardrail that blocks a held-back response or whose buffer the response outgrows. In the held-back case, numbering starts at 0 because the withheld events were never sent. Stream helpers that require response.created first, such as responses.stream() in the OpenAI Python SDK, therefore report the real failure. A stream that has already sent response.created never gets a second one.

The error.code in response.failed depends on why the stream stopped:

Cause	error.code in response.failed
An output guardrail blocks the response, or a held-back response outgrows the guardrail's buffer	invalid_prompt
The upstream sends an error inside the stream whose code is context_length_exceeded, insufficient_quota, rate_limit_exceeded, server_is_overloaded, or slow_down	That code
Any other upstream failure	The same code as the error event, such as upstream_error

Clients read their retry decision from this code. Codex, for example, shows the message and does not retry a turn that failed with invalid_prompt, since resending it would meet the same guardrail.

Two stream endings are handled specially:

• The connection drops after the finish. If the upstream has already sent a finish_reason and the connection drops before the usage chunk or [DONE], the response completes normally. When the usage chunk never arrived, the usage is estimated.
• The upstream returns an empty stream. A stream that ends without any content, reasoning, tool call, or finish_reason, such as a bare [DONE] or a usage-only chunk, fails with an error event and a response.failed event, both with code upstream_error and a message saying the upstream returned an empty stream. That code is retryable. A stream that produced content and then ended without a finish_reason still completes.

Fields Ignored on Translated Requests

• reasoning members other than effort, such as summary
• text.verbosity
• store
• previous_response_id
• metadata, service_tier, and other OpenAI-specific controls
• hosted tools such as web_search, file_search, code_interpreter, mcp, computer_use, and image_generation

A custom tool is no longer in that set. It is converted as described in Tool Parameters.

Two further limits hold because the chat format has no counterpart for them:

• An input_image addressed only by a file_id is not forwarded. The chat image part addresses an image by a URL or a data: URL and nothing else.
• A non-text part inside a tool result is not forwarded. The chat tool role is text-only, and an OpenAI-compatible upstream rejects an image in it.

Policy and Usage Behavior

Input guardrails can inspect request text before AISIX calls the provider. Output guardrails can inspect non-streaming responses before content reaches the caller.

When a client replays conversation history, a guardrail can limit its input window to the latest turn. AISIX uses Responses item types to identify assistant turns and current-turn tool results. See Choose the Input Message Window.

Reasoning that the model generates on the bridged path is outside output-guardrail scanning and masking, the same as on the verbatim path. See Reasoning and Thinking Content.

If an output guardrail blocks the response, AISIX returns a content-policy error to the caller and records the blocked request for observability. On a bridged stream, the block ends the stream as described in Stream Failures on the Bridged Path.

For streaming requests, AISIX preserves the Responses SSE shape. On native Responses targets, AISIX can pass through upstream SSE. On bridged targets, AISIX encodes provider stream chunks into Responses events. If output guardrails are enabled, AISIX buffers the stream for policy inspection before returning it or blocking it. A buffered frame that AISIX cannot parse is dropped rather than released unscanned, and a response left with nothing to return is refused with 422. See Frames AISIX Cannot Scan.

For successful responses, the gateway records usage when the upstream response includes token usage. Streaming usage is emitted when AISIX receives terminal usage information from the stream.

When the upstream reports no token usage, AISIX estimates the counts locally, and the caller reads those same numbers: in usage on a non-streaming reply, and in the usage of the response.completed event on a stream. A count the upstream did report stands. There is no marker in the response saying a count was estimated; the usage_estimated field on the usage record is the signal. See Usage Reporting.

On bridged providers whose token accounting differs from OpenAI's, AISIX converts the counts into the Responses API shape: input_tokens is the full input, input_tokens_details.cached_tokens is the cache-read subset of it, input_tokens_details.cache_creation_tokens is the cache-write subset when the upstream reported one, and total_tokens is input_tokens + output_tokens. See Token Usage for a worked example with an Anthropic upstream — it is written in Chat Completions field names, which map to these one for one (prompt_tokens to input_tokens, completion_tokens to output_tokens, prompt_tokens_details to input_tokens_details, completion_tokens_details to output_tokens_details). One shape difference: output_tokens_details.reasoning_tokens is always present on a Responses reply, including when it is 0, whereas Chat Completions omits the block entirely.

Next Steps

You have now seen when to use Responses API through AISIX and how provider handling differs between direct forwarding and bridging. Next, continue with Text Completions for the legacy completions route, or Streaming when your Responses API client depends on SSE behavior.