Skip to main content
API7.ai Logo
Version: Dev

Passthrough Routes

A passthrough route forwards matching requests to one upstream target without translating the request body. It is useful for provider-native endpoints that AISIX does not model as first-class routes and for forward-proxy traffic delivered with its original Host header.

Each route defines how traffic matches, where it goes, how AISIX authenticates the caller, and whether AISIX injects a provider credential or forwards the caller's credential. AISIX can still apply caller access controls, request limits, guardrails, and telemetry around the relay.

Explicit route required

Every passthrough path or host must be claimed by an explicit route. An unclaimed /passthrough/* path follows the ordinary empty-body 404 path. Create and verify the route before moving client traffic.

Passthrough routes do not rewrite model identifiers. If a provider-native request names a model in its body, path, query, or headers, send the identifier expected by that provider.

Prerequisites​

Prepare the following:

  • One AISIX deployment:
    • For AISIX Cloud, an environment with an attached gateway and a write-scoped admin token.
    • For the open-source AISIX gateway, a gateway configured to load a declarative resources file.
  • An upstream provider credential for an inject route. The examples use OpenAI. A forward_client route instead relays the caller's upstream credential.
  • curl and jq.

Understand the Passthrough Flow​

AISIX preserves the request body while handling gateway authentication, target construction, intentional header filtering, guardrails, and telemetry:

Each matched request makes one upstream attempt. AISIX does not retry a passthrough request after a transport failure or upstream 5xx response. A route failure also does not place any configured model into cooldown because the route does not resolve a model.

URL rewriting runs before route matching. Both stages below use the rewritten path; optional hosts on a rewrite rule can keep other hosts' paths unchanged. Route matching then runs in two stages:

  1. A request whose inbound Host matches a route's hosts allowlist is dispatched before the gateway's typed routes. This lets a forward proxy relay an upstream path such as /v1/messages without the gateway treating it as its own endpoint.
  2. Path-prefix matching runs after the typed routes, so a path-only route cannot shadow the gateway's /v1, /mcp, or /a2a endpoints.

When several routes match, host matches beat path-only matches, and a longer matching prefix beats a shorter one.

Configure a Provider-Native Route​

The following examples expose OpenAI's native model-list endpoint at /passthrough/openai/v1/models. AISIX injects the configured OpenAI credential and requires a caller key that grants openai-tunnel.

AISIX Cloud​

Export the AISIX Cloud connection details and the provider credential:

# AISIX_CP includes /api and has no trailing slash
# The local On-Premises quickstart uses http://localhost:8080/api
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"
export OPENAI_API_KEY="YOUR_OPENAI_API_KEY"

Create an OpenAI provider key that is available to the environment:

PROVIDER_KEY_ID=$(curl --fail-with-body -sS -X POST \
  "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "OpenAI passthrough",
    "provider": "openai",
    "api_key": "'"${OPENAI_API_KEY}"'",
    "api_base": "https://api.openai.com/v1",
    "allowed_environments": ["'"${ENV_ID}"'"]
  }' | jq -er '.provider_key.id')

Create the route with the provider key ID:

ROUTE_RESPONSE=$(curl --fail-with-body -sS -X POST \
  "$AISIX_CP/environments/$ENV_ID/passthrough_routes" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "openai-tunnel",
    "path_prefix": "/passthrough/openai",
    "target_url": "https://api.openai.com/v1",
    "credential_mode": "inject",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }')

export ROUTE_ID=$(printf '%s' "$ROUTE_RESPONSE" | jq -er '.passthrough_route.id')
printf '%s' "$ROUTE_RESPONSE" | jq '.warnings // []'

Keep ROUTE_ID for route updates or route-scoped guardrail attachments.

Create a dedicated caller key and grant the route in the same request. The plaintext is returned once:

CALLER_RESPONSE=$(curl --fail-with-body -sS -X POST \
  "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "OpenAI passthrough caller",
    "allowed_models": [],
    "allowed_routes": ["openai-tunnel"]
  }')

export AISIX_API_KEY=$(printf '%s' "$CALLER_RESPONSE" | jq -er '.plaintext')
printf '%s' "$CALLER_RESPONSE" | jq '.warnings // []'

The control plane projects the provider key, route, and caller grant to attached gateways. Review any returned compatibility warnings before rollout. Warnings are advisory, so verify traffic through each gateway. In the dashboard, the same workflow is available under Provider keys, an environment's Passthrough Routes, and the caller key's Passthrough route access section.

If you grant an existing caller instead, include every route grant it should keep. The AISIX Cloud Admin API replaces the complete allowed_routes list when that field is patched.

Open-Source AISIX Gateway​

Prepare an openai-prod provider key in the complete resources file, then choose a dedicated caller credential:

export PASSTHROUGH_CALLER_KEY="YOUR_CALLER_API_KEY"

Add the route and caller entries to the matching collections. Keep the provider key and other resources unchanged:

resources.yaml (route and caller key)
passthrough_routes:
  - name: openai-tunnel
    path_prefix: /passthrough/openai
    target_url: https://api.openai.com/v1
    provider_key: openai-prod

api_keys:
  - display_name: passthrough-caller
    key_env: PASSTHROUGH_CALLER_KEY
    allowed_models: []
    allowed_routes: [openai-tunnel]

The provider_key name is resolved to the provider key's derived ID when AISIX loads the file. An unknown name fails validation. Exact entries in allowed_routes are also checked against the routes defined in the file; wildcard patterns are allowed.

Validate the assembled complete file before loading it:

aisix validate --resources resources.yaml

Because this example introduces PASSTHROUGH_CALLER_KEY, start or recreate the gateway with that variable in its process environment. After it loads, use the same value for the verification request:

export AISIX_API_KEY="$PASSTHROUGH_CALLER_KEY"

Verify the Route​

Export the gateway origin without a trailing slash:

# AISIX_PROXY has no trailing slash or endpoint path
# The local quickstarts use http://127.0.0.1:3000
export AISIX_PROXY="YOUR_AISIX_GATEWAY_URL"

Request OpenAI's native model list through AISIX:

curl --fail-with-body -sS \
  "$AISIX_PROXY/passthrough/openai/v1/models" \
  -H "Authorization: Bearer $AISIX_API_KEY"

The route strips its path_prefix, avoids duplicating the /v1 segment already present in target_url, injects the OpenAI provider credential, and relays the upstream response.

Compare Cloud and Resources-File References​

Most route fields use the same names in both management paths. References to provider and caller credentials differ:

PurposeAISIX Cloud Admin APIRecommended resources-file fieldExplicit ID accepted in a resources file
Injected provider credentialprovider_key_id (UUID)provider_key (display_name)provider_key_id
Anonymous caller principalanonymous_key_id (UUID)anonymous_key (display_name)anonymous_key_id

Name references in a resources file receive stronger load-time checking and produce candidate names when a reference is unknown. Prefer them over explicit IDs. AISIX Cloud validates that the provider key is visible to the environment and that the anonymous caller key belongs to it.

The route name is fixed after creation in AISIX Cloud. In a resources file, changing the name changes the route identity, so update every caller's allowed_routes entry at the same time.

Configuration Reference​

FieldBehavior
nameRoute identity referenced by callers' allowed_routes patterns and recorded on usage events.
path_prefixGateway path prefix, matched on segment boundaries. A target_url route strips it before joining the remaining path to the target. A preserve_host route keeps the complete path. A path-only route cannot claim /v1, /mcp, /a2a, /admin, /livez, /readyz, or /metrics; a route that also matches hosts may use those upstream-owned paths.
hostsCase-insensitive inbound Host allowlist; a port is ignored. A leading *. wildcard matches one additional label and must retain at least two literal labels.
target_urlExplicit upstream base URL. Configure exactly one target shape: target_url, or preserve_host: true.
preserve_hostDerive https://<matched-host> as the target. It is accepted only with hosts, which bounds the derived destination.
auth_modegateway_key (default), header_key, or anonymous.
auth_header_nameLowercase side-channel header containing the gateway credential in header_key mode. AISIX strips it before forwarding, unless forward_client_headers names it in full; a glob such as x-* does not reach it. authorization, proxy-authorization, cookie, set-cookie, and x-api-key are rejected.
source_cidrsClient source allowlist. It is required and nonempty for anonymous; for other modes it is optional hardening.
credential_modeinject (default) or forward_client.
forward_client_headersInbound client headers relayed upstream even when this route would otherwise strip them, as exact names or single-* globs, matched case-insensitively. Empty by default. A patch replaces the stored list; send null to clear it. In the dashboard it is Forward client headers under Advanced, one entry per line. See Upstream Request Headers.
identity_headerOptional lowercase device-injected identity header. AISIX records a bounded value as client_identity and strips the header before forwarding, unless forward_client_headers names it in full; a glob such as x-* does not reach it. Configure it only behind a trusted device that removes and replaces any client-supplied value. authorization, proxy-authorization, cookie, set-cookie, and x-api-key are rejected.
timeout_msBounds upstream response headers and non-SSE body reads. For SSE, it bounds the wait for the first raw upstream body chunk and each gap between raw upstream body chunks. It does not bound the total lifetime of a healthy relay or require a complete SSE frame within that interval. When omitted, the route uses upstream.stream_timeout_ms, then upstream.timeout_ms.
enabledA disabled route matches nothing. Default: true.

At least one match dimension, path_prefix or hosts, is required. When both are present, the request must satisfy both.

Gateway Authentication Modes​

  • gateway_key reads the standard gateway credential from Authorization: Bearer or x-api-key.
  • header_key reads the gateway credential from auth_header_name, leaving Authorization available for the caller's upstream credential. This is the standard forward-proxy pairing with forward_client.
  • anonymous accepts no gateway credential. The request runs as the configured caller-key principal and must originate within source_cidrs.

Every resolved principal still needs the route name granted by allowed_routes; * grants every route. A valid key without a matching grant receives 403.

In AISIX Cloud, budgets that already apply to the resolved caller are checked before dispatch. Passthrough usage currently carries no model ID, is recorded with zero cost, and does not add spend to those budgets. The open-source AISIX gateway has no local budget resource.

Upstream Credential Modes​

  • inject strips inbound credential headers and injects the configured provider key. AISIX uses x-api-key plus anthropic-version for Anthropic and Authorization: Bearer for other providers. The provider key's configured header-strip and TLS settings also apply.
  • forward_client forwards the caller's upstream credential when the gateway credential arrived through header_key, or when the route is anonymous. With gateway_key, AISIX removes Authorization and x-api-key because either may contain the gateway credential.

A route relays the caller's other headers by default and strips a small set: hop-by-hop and transport headers, host, content-length, the x-aisix-* namespace, proxy-authorization, and the caller's W3C trace headers. AISIX consumes valid W3C context for its own OTLP trace and sends a gateway request ID upstream.

forward_client_headers overrides that strip, and is the one way to put a header back that the route would otherwise remove. Naming authorization under auth_mode: gateway_key therefore relays the caller's own credential — the very header the gateway consumed to authenticate them — in place of the injected one, which is what lets an internal service that authorizes on the end user keep doing so. host, content-length, hop-by-hop headers, and x-aisix-* are stripped whatever the patterns say, and a credential or trace-context header must be named exactly rather than matched by a wildcard. The route's own auth_header_name and identity_header are read the same way: AISIX consumes both, so a glob does not reach either and naming one in full is what forwards it. For every other name the route strips, a glob is enough — including a provider key strip_headers entry, though three of that list's four defaults (authorization, cookie, x-api-key) are credential slots that still need their own entry, leaving set-cookie the only default a glob restores.

There is no credential fallback: an inject route without a resolvable provider key fails closed, and a forward_client route cannot carry a provider key reference.

Envelope Detection and Usage​

AISIX identifies the protocol of each exchange for extraction only; identification does not change the relayed body. The request body's top-level carrier field selects the envelope. If several recognized carrier fields appear, detection uses this order:

  1. messages for chat traffic. A request that carries the anthropic-version header, or whose path ends in /messages, is Anthropic Messages. Any other messages body is OpenAI-compatible chat.
  2. input as a string or an array for the OpenAI Responses shape.
  3. prompt for legacy completions or fill-in-the-middle traffic.
  4. Opaque handling for every other body, including JSON-RPC, REST, non-JSON, and empty bodies.

The envelope also selects the guardrail content rules. See Guardrail Scan Coverage for the security boundary around malformed, opaque, and supplemental content.

Usage Recorded per Request Shape​

AISIX reads the usage of an identified exchange with the parser of the typed endpoint that serves the same protocol, so a passthrough route records the same token counts that endpoint records for the same upstream reply. Each token field keeps the meaning its protocol gives it; AISIX never infers a field's meaning from its name. A reply whose usage the protocol's parser cannot read records no usage, as on the typed endpoint.

The request path takes precedence over the body for two surfaces whose bodies carry another envelope's key: a path ending in /embeddings is metered as embeddings, and a path ending in /images/generations or /images/edits as image generation. Among bodies with no envelope, AISIX also meters a rerank body and an Alibaba Cloud DashScope native body. It reads only their model field and token counts, and guardrails inspect decoded string values in those JSON bodies.

ProtocolIdentified byUsage read as
OpenAI-compatible chatThe messages envelope./v1/chat/completions reads it, from a buffered response or each stream chunk.
Anthropic MessagesThe messages envelope with anthropic-version or a /messages path./v1/messages reads it, including the stream's message_start and message_delta events. Cache read and cache creation tokens are recorded beside the input tokens.
ResponsesThe input envelope./v1/responses reads it. A stream reports usage only on its terminal event.
CompletionsThe prompt envelope./v1/completions reads it.
EmbeddingsA path ending in /embeddings, or a buffered reply that is an object: "list" of object: "embedding" items./v1/embeddings reads it: input tokens from prompt_tokens, otherwise total_tokens. A stream records no usage.
Image generation and editingA path ending in /images/generations or /images/edits.Image generation reads it. A stream records no usage.
RerankA top-level query and a documents array. Covers Cohere, Jina, and Alibaba Cloud Model Studio's compatible rerank API.The rerank endpoint reads it: input tokens from usage.prompt_tokens, usage.input_tokens, or usage.total_tokens, or from Cohere's meta.billed_units.input_tokens. A stream records no usage.
DashScope nativeA top-level model string and an input object. Covers native rerank, text and multimodal embedding, and native generation.See DashScope Native Token Counts.
Any other bodyNothing above matched.Nothing, except the agent usage frames described below.

For a buffered reply to an identified request, the envelope the reply names itself takes precedence over the request: type: "message" is read as Anthropic Messages, output as Responses, and choices as chat or completions. In a stream, a message_start or message_delta event is read as Anthropic Messages. An Anthropic reply is therefore metered correctly even when the request carried neither signal.

An opaque stream reports usage only through a flat token report on a frame the server labels event: token_usage or event: usage, the shape IDE agent backends send. AISIX reads prompt_tokens there as the whole input, and records cache_read_input_tokens and cache_creation_input_tokens as cached and cache-write tokens inside it, because such a backend reports its total as prompt_tokens + completion_tokens. A usage object on any other opaque frame is not read.

The access log's total_tokens for a passthrough request includes cache creation and cache read tokens, as every typed endpoint's line does. The usage event's total_tokens remains the total the upstream reported, so an Anthropic Messages exchange records none, the same as /v1/messages.

Every passthrough usage event has the operation passthrough, whatever the body shape. See Tell Request Kinds Apart. For a recognized shape, requested_model is the request body's model value. That value is the upstream model ID the caller sent, not an AISIX model alias, because the route does not resolve a model. Every other body records no model. model_id is always empty, so passthrough usage records zero cost: pricing is keyed on the AISIX model.

Metrics for Identified Protocols​

An exchange identified as chat, Anthropic Messages, Responses, completions, embeddings, image generation or editing, or rerank counts as model inference in the gateway metrics, under endpoint="/passthrough_route" and inbound_protocol="passthrough":

  • aisix_llm_requests_total and aisix_llm_request_duration_seconds count it once, in addition to the proxy request families.
  • The aisix_llm_*_tokens_total counters, aisix_llm_tokens_by_client_total, and aisix_tokens_consumed_total record its tokens. aisix_llm_spend_micro_usd_total records nothing, because a route prices nothing.
  • A streamed reply records time to first token in aisix_llm_time_to_first_token_seconds and aisix_request_ttft_seconds. Chat, Anthropic Messages, and Responses exchanges also record aisix_request_e2e_latency_seconds.

On these series, the request families included, model is the request body's model when it names a configured model, or the wildcard model whose pattern covers it. Any other value reports unresolved, and a body without model reports unknown. upstream_model is unknown. A DashScope native body, an agent usage frame, and an opaque relay stay in the proxy request families only, with model="passthrough".

Changes from 1.6.0 and Earlier​

A gateway at 1.6.0 or earlier meters passthrough traffic differently. Check queries and reports that read passthrough usage:

  • Every passthrough request counts in the proxy request families only, with model="passthrough", and records no token or latency series. On a newer gateway, select passthrough traffic with endpoint="/passthrough_route" rather than model="passthrough".
  • Usage is read by field name rather than by protocol. Embeddings and image bodies are read under the Responses and completions field names, and an opaque stream's usage object on any frame is read.
  • An agent usage frame's cache_read_input_tokens and cache_creation_input_tokens are recorded as Anthropic-style cache tokens beside prompt_tokens, so a total that adds those counters to the input tokens counts the cache twice.
  • The access log's total_tokens excludes cache creation and cache read tokens.

DashScope Native Token Counts​

For a DashScope native body, AISIX reads the response's top-level usage object as follows:

  • Output tokens come from output_tokens, or completion_tokens.
  • When total_tokens is present, input tokens are total_tokens minus the output tokens.
  • Without total_tokens, input tokens are input_tokens, or prompt_tokens, plus the top-level image_tokens. A nested input_tokens_details.image_tokens value is already part of input_tokens and is not added.
  • Cached-input and reasoning token counts are read from the same object with the standard field names, such as prompt_tokens_details.cached_tokens and output_tokens_details.reasoning_tokens.

This counts image tokens once, whether a service reports them inside input_tokens or next to it:

ServiceUpstream usageRecorded inputRecorded output
Native multimodal rerank, qwen3-vl-rerankinput_tokens: 146, image_tokens: 64, total_tokens: 2102100
Multimodal embedding, multimodal-embedding-v1input_tokens: 4, image_tokens: 1281320
Native multimodal generationinput_tokens: 79, image_tokens: 66, output_tokens: 14, total_tokens: 937914

Example: Meter Alibaba Cloud Native Rerank​

The following AISIX Cloud example relays DashScope's native API for the mainland China region through /passthrough/dashscope. Create an alibaba-cn provider key for the DashScope credential. For the international region, use the alibaba provider and https://dashscope-intl.aliyuncs.com/api/v1. The provider key's api_base does not affect this route, because the route's target_url sets the upstream:

export DASHSCOPE_API_KEY="YOUR_DASHSCOPE_API_KEY"

DASHSCOPE_KEY_ID=$(curl --fail-with-body -sS -X POST \
  "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "DashScope native",
    "provider": "alibaba-cn",
    "api_key": "'"${DASHSCOPE_API_KEY}"'",
    "allowed_environments": ["'"${ENV_ID}"'"]
  }' | jq -er '.provider_key.id')

curl --fail-with-body -sS -X POST \
  "$AISIX_CP/environments/$ENV_ID/passthrough_routes" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "dashscope-native",
    "path_prefix": "/passthrough/dashscope",
    "target_url": "https://dashscope.aliyuncs.com/api/v1",
    "credential_mode": "inject",
    "provider_key_id": "'"${DASHSCOPE_KEY_ID}"'"
  }' | jq '.warnings // []'

In a resources file, declare the same route with provider_key set to the display_name of an alibaba-cn provider key. Qwen shows a provider key entry:

resources.yaml (route)
passthrough_routes:
  - name: dashscope-native
    path_prefix: /passthrough/dashscope
    target_url: https://dashscope.aliyuncs.com/api/v1
    provider_key: dashscope-cn

Grant dashscope-native in the caller key's allowed_routes, as in Configure a Provider-Native Route. Then send a native rerank request with the upstream model ID:

curl --fail-with-body -sS \
  "$AISIX_PROXY/passthrough/dashscope/services/rerank/text-rerank/text-rerank" \
  -H "Authorization: Bearer $AISIX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gte-rerank-v2",
    "input": {
      "query": "What does a passthrough route record?",
      "documents": [
        "A passthrough route relays provider-native requests.",
        "Usage events carry token counts and the route name."
      ]
    },
    "parameters": {"top_n": 2}
  }'

AISIX strips the prefix, relays the body to https://dashscope.aliyuncs.com/api/v1/services/rerank/text-rerank/text-rerank with the injected credential, and returns DashScope's response unchanged. The body's top-level model and input object make it a DashScope native body, so the usage event records gte-rerank-v2 as requested_model and the response's usage.total_tokens as input tokens.

Rate Limits​

Caller API-key, team, and member request limits apply before dispatch. On an inject route, a top-level JSON model that resolves to a configured AISIX model of the same provider also reserves that model's request limits. forward_client routes do not perform this model lookup.

Request-count dimensions (rps, rpm, rph, and rpd) are enforced. AISIX also rejects a request when an applicable tpm or tpd counter is already exhausted, but passthrough token usage does not increment those counters. Use recorded usage for telemetry, not passthrough token-quota enforcement.

Concurrency is checked before upstream dispatch. Every matching SSE concurrency reservation remains held for the lifetime of the stream. Redis-backed limits renew the reservation from acquisition until the stream ends and use concurrency_ttl_secs to recover one that is no longer renewed. If Redis reports that a live reservation no longer exists, AISIX ends the relay. The caller has already received 200, while the terminal usage event records status 429, error_class="rate_limit_lease_lost", and error_message="distributed concurrency lease was lost". See Redis-backed concurrency leases for failure and rollout behavior.

Passthrough routes have no rate-limit field or policy scope. To apply different request limits to different routes, grant them to separate caller keys and configure limits for those caller identities.

Guardrails and Streaming​

In AISIX Cloud, a guardrail can be attached to one passthrough route by selecting the Passthrough routes scope. The attachment uses the route UUID. Environment, caller-key, and team guardrails can also apply.

The open-source resources file declares attachments in its guardrail_attachments collection, so a file-defined guardrail reaches passthrough traffic only if an attachment scopes it there — scope_type: env, or scope_type: passthrough_route naming the route.

Input guardrails run before upstream dispatch. A block returns 422 without contacting the upstream. Buffered responses are checked before delivery. Once an SSE response has started, a block ends the stream with an SSE content_filter error frame; it cannot change the HTTP status to 422.

SSE responses relay incrementally unless a hold-back guardrail buffers frames for inspection. AISIX does not rewrite provider-native bodies to apply redaction. For the built-in pii guardrail, a mask-action-only match is forwarded without masking and recorded as mask_unsupported, or would_mask_unsupported in monitor mode; use a block action when the matched content must not reach the upstream. Guardrail kinds such as Presidio and Lakera instead block a maskable result when passthrough has no write-back channel. See Surfaces That Cannot Apply a Mask.

Guardrail Scan Coverage​

Guardrails use protocol-aware extraction. On successfully decoded requests, they also inspect ordinary JSON strings forwarded alongside the protocol's canonical fields, including nested values. Response coverage remains protocol-specific and is limited to the fields listed below.

Tool-call IDs are opaque handles, not caller content, so the request scan leaves them out of a recognized envelope:

  • Chat and Anthropic Messages: each tool call's id, call_id, tool_use_id, and tool_call_id, and a tool message's tool_call_id.
  • Responses: call_id and approval_request_id on every item; the id of any *_call or *_call_output item and of an item_reference, mcp_approval_request, or reasoning item; and a reasoning item's encrypted_content. A passthrough Responses body is therefore scanned the same way as a /v1/responses request.

Function names, arguments, and tool results are still scanned, and the response-side fields below never include IDs. A guardrail rule that matches only text inside one of these values does not fire. The Any other body row is unaffected and still reads every decoded string.

If AISIX recognizes an envelope but cannot decode it, AISIX does not fall back to scanning the raw body. A top-level model is routing metadata, not caller content. Typed opaque content is not reinterpreted as text: this includes image, audio, document, and file payloads, along with redacted_thinking data. The Chat row therefore covers both OpenAI-compatible chat and Anthropic Messages traffic:

EnvelopeRequestResponse
Chat or Anthropic MessagesMessage text and text-bearing content blocks, the Anthropic top-level system, tool_result content, chat tool_calls, and replayed tool_use input, thinking, and reasoning_content.Chat message text and refusals, plus function and custom tool names and arguments. For Anthropic, buffered replies scan text blocks and tool_use names and input; streams scan tool_use starts plus text and partial_json deltas.
ResponsesEvery input item the /v1/responses endpoint reads, including a replayed tool call's name, arguments, and input.Output text, refusals, and tool calls: message text and refusal parts; function_call and mcp_call names and arguments; and custom_tool_call names and input in a buffered reply. Stream carriers include the corresponding delta and done events, content-part and output-item snapshots, and terminal response snapshots.
Completionsprompt and suffix.Completion text.
Any other bodyEvery decoded JSON string value when the body is JSON; otherwise the textual body.A buffered JSON body contributes its decoded string values; other buffered text is scanned directly. An opaque SSE stream can scan only a bare JSON string. Other content-bearing payload shapes cannot be evaluated because AISIX cannot reliably associate their text with the preceding stream content.

Within a recognized Responses envelope, known media, reasoning, and unknown output items remain opaque. Malformed recognized buffered output cannot be evaluated and follows the configured failure policy. Unknown Responses stream events remain opaque.

A buffered reply that names two envelopes cannot be evaluated either: a Responses output beside chat choices, or chat choices beside an Anthropic type: "message" body. Each extraction rule reads only its own carrier, so a fail-closed output guardrail refuses such a reply with unscannable_body, and a fail-open one releases it. A gateway at 1.6.0 or earlier scans only the chat carrier of a reply that pairs choices with an Anthropic body. See What Happens to Content AISIX Cannot Read.

Reasoning the model generates, such as reasoning_content, Anthropic thinking, and Responses reasoning summaries, is not scanned on the response side. It still counts toward a held stream's max_buffer_bytes.

Audit Capture​

For successfully relayed traffic, an observability exporter with content_mode: full receives the request body as a string, subject to the exporter's content cap. Buffered responses record extracted text when the response matches a supported extraction shape and otherwise record the body as text. Streamed responses record accumulated extracted text; opaque data payloads are retained as text. Captured content is exporter-only and is never sent through the AISIX Cloud telemetry path.

Usage events carry the route name, caller, recorded token counts, and client_identity. External exporters can expose these values. The current AISIX Cloud Request Logs UI shows caller and token metadata, but it does not display passthrough_route_name or client_identity.

Errors​

Status or signalMeaning
401Missing or invalid gateway credential for the route's authentication mode.
403The resolved caller key does not grant the route, or the client source is outside source_cidrs.
404An unclaimed /passthrough/* path reaches the ordinary empty-body not-found path.
422A guardrail blocked the request before dispatch or blocked a buffered response before delivery.
SSE content_filter frameA guardrail blocked content after a stream had started.
429A gateway request limit or budget check rejected the request, or the upstream returned a relayed 429.
Terminal usage status 429 after SSE startsA Redis-backed concurrency lease disappeared after the caller received 200. The usage event records error_class="rate_limit_lease_lost"; see Streams That Fail After the Response Head.
Other upstream statusAISIX relays the upstream status and body after filtering response headers.

The unmatched 404 has an empty body. Other AISIX-generated failures use the gateway error envelope; upstream error statuses and bodies are relayed after response-header filtering.

Next Steps​