# Passthrough Routes

A passthrough route forwards matching requests to one upstream target without translating the request body. It is useful for provider-native endpoints that AISIX does not model as first-class routes and for forward-proxy traffic delivered with its original `Host` header.

Each route defines how traffic matches, where it goes, how AISIX authenticates the caller, and whether AISIX injects a provider credential or forwards the caller's credential. AISIX can still apply caller access controls, request limits, guardrails, and telemetry around the relay.

Explicit route required

Every passthrough path or host must be claimed by an explicit route. An unclaimed `/passthrough/*` path follows the ordinary empty-body `404` path. Create and verify the route before moving client traffic.

Passthrough routes do not rewrite model identifiers. If a provider-native request names a model in its body, path, query, or headers, send the identifier expected by that provider.

## Prerequisites[​](#prerequisites "Direct link to Prerequisites")

Prepare the following:

* One AISIX deployment:

  <!-- -->

  * For AISIX Enterprise or AISIX Cloud, an environment with an attached gateway and a write-scoped admin token.
  * For the open-source AISIX gateway, a gateway configured to load a declarative resources file.

* An upstream provider credential for an `inject` route. The examples use OpenAI. A `forward_client` route instead relays the caller's upstream credential.

* `curl` and `jq`.

## Understand the Passthrough Flow[​](#understand-the-passthrough-flow "Direct link to Understand the Passthrough Flow")

AISIX preserves the request body while handling gateway authentication, target construction, intentional header filtering, guardrails, and telemetry:

Each matched request makes one upstream attempt. AISIX does not retry a passthrough request after a transport failure or upstream `5xx` response. A route failure also does not place any configured model into cooldown because the route does not resolve a model.

[URL rewriting](https://docs.api7.ai/ai-gateway/deployment/url-rewriting.md) runs before route matching. Both stages below use the rewritten path; optional `hosts` on a rewrite rule can keep other hosts' paths unchanged. Route matching then runs in two stages:

1. A request whose inbound host matches a route's `hosts` allowlist is dispatched before the gateway's typed routes. This lets a forward proxy relay an upstream path such as `/v1/messages` without the gateway treating it as its own endpoint.
2. Path-prefix matching runs after the typed routes, so a path-only route cannot shadow the gateway's `/v1`, `/mcp`, or `/a2a` endpoints.

When several routes match, host matches beat path-only matches, and a longer matching prefix beats a shorter one.

A host match takes over that host. Requests on it never reach the gateway's own API, including `/v1/*`, `/mcp`, `/a2a`, `/livez`, and `/readyz`. A route with only `hosts` claims every path on the host. A route with both `hosts` and `path_prefix` claims only the paths under its prefix, and any other path on that host returns `404` unless another route matches it. The request does not fall back to a typed route.

## Configure a Provider-Native Route[​](#create-a-passthrough-route "Direct link to Configure a Provider-Native Route")

The following examples expose OpenAI's native model-list endpoint at `/passthrough/openai/v1/models`. AISIX injects the configured OpenAI credential and requires a caller key that grants `openai-tunnel`.

### AISIX Enterprise or AISIX Cloud[​](#aisix-enterprise-or-aisix-cloud "Direct link to AISIX Enterprise or AISIX Cloud")

Export the control-plane connection details and the provider credential:

```
# AISIX_CP includes /api and has no trailing slash

# The local AISIX Enterprise quickstart uses http://localhost:8080/api

export AISIX_CP="YOUR_AISIX_CONTROL_PLANE_API_URL"

export AISIX_TOKEN="YOUR_ADMIN_TOKEN"

export ENV_ID="YOUR_ENVIRONMENT_ID"

export OPENAI_API_KEY="YOUR_OPENAI_API_KEY"
```

Create an OpenAI provider key that is available to the environment:

```
PROVIDER_KEY_ID=$(curl --fail-with-body -sS -X POST \

  "$AISIX_CP/provider_keys" \

  -H "Authorization: Bearer $AISIX_TOKEN" \

  -H "Content-Type: application/json" \

  -d '{

    "display_name": "OpenAI passthrough",

    "provider": "openai",

    "api_key": "'"${OPENAI_API_KEY}"'",

    "api_base": "https://api.openai.com/v1",

    "allowed_environments": ["'"${ENV_ID}"'"]

  }' | jq -er '.provider_key.id')
```

Create the route with the provider key ID:

```
ROUTE_RESPONSE=$(curl --fail-with-body -sS -X POST \

  "$AISIX_CP/environments/$ENV_ID/passthrough_routes" \

  -H "Authorization: Bearer $AISIX_TOKEN" \

  -H "Content-Type: application/json" \

  -d '{

    "name": "openai-tunnel",

    "path_prefix": "/passthrough/openai",

    "target_url": "https://api.openai.com/v1",

    "credential_mode": "inject",

    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"

  }')



export ROUTE_ID=$(printf '%s' "$ROUTE_RESPONSE" | jq -er '.passthrough_route.id')

printf '%s' "$ROUTE_RESPONSE" | jq '.warnings // []'
```

Keep `ROUTE_ID` for route updates or route-scoped guardrail attachments.

Create a dedicated caller key and grant the route in the same request. The plaintext is returned once:

```
CALLER_RESPONSE=$(curl --fail-with-body -sS -X POST \

  "$AISIX_CP/environments/$ENV_ID/api_keys" \

  -H "Authorization: Bearer $AISIX_TOKEN" \

  -H "Content-Type: application/json" \

  -d '{

    "display_name": "OpenAI passthrough caller",

    "allowed_models": [],

    "allowed_routes": ["openai-tunnel"]

  }')



export AISIX_API_KEY=$(printf '%s' "$CALLER_RESPONSE" | jq -er '.plaintext')

printf '%s' "$CALLER_RESPONSE" | jq '.warnings // []'
```

The control plane projects the provider key, route, and caller grant to attached gateways. Review any returned compatibility warnings before rollout. Warnings are advisory, so verify traffic through each gateway. In the dashboard, the same workflow is available under **Provider keys**, an environment's **Passthrough Routes**, and the caller key's **Passthrough route access** section.

If you grant an existing caller instead, include every route grant it should keep. The AISIX Cloud Admin API replaces the complete `allowed_routes` list when that field is patched.

### AISIX OSS[​](#aisix-oss "Direct link to AISIX OSS")

Prepare an `openai-prod` provider key in the [complete resources file](https://docs.api7.ai/ai-gateway/reference/resources-file.md#apply-resource-examples-safely), then choose a dedicated caller credential:

```
export PASSTHROUGH_CALLER_KEY="YOUR_CALLER_API_KEY"

export PASSTHROUGH_CALLER_KEY_SHA256="$(printf '%s' "$PASSTHROUGH_CALLER_KEY" | sha256sum | cut -d ' ' -f 1)"
```

Add the route and caller entries to the matching collections. Keep the provider key and other resources unchanged:

resources.yaml (route and caller key)

```
passthrough_routes:

  - id: 625b56f6-6b34-4fc2-a6ea-ccf2aac72925

    name: openai-tunnel

    path_prefix: /passthrough/openai

    target_url: https://api.openai.com/v1

    provider_key_id: 83eba07c-0ad3-4b81-b760-247f2d978e95



api_keys:

  - id: ea00ce9d-922c-49a4-b3cd-037b5b8dd99c

    display_name: passthrough-caller

    key_hash: ${PASSTHROUGH_CALLER_KEY_SHA256}

    allowed_models: []

    allowed_routes: [openai-tunnel]
```

`provider_key_id` takes the `id` of the provider key entry, and `allowed_routes` names the route by its `name`; wildcard patterns are allowed.

Validate the assembled complete file before loading it:

```
aisix validate --resources resources.yaml
```

Because this example introduces `PASSTHROUGH_CALLER_KEY_SHA256`, start or recreate the gateway with that variable in its process environment. After it loads, use the plaintext key for the verification request:

```
export AISIX_API_KEY="$PASSTHROUGH_CALLER_KEY"
```

## Verify the Route[​](#verify-the-route "Direct link to Verify the Route")

Export the gateway origin without a trailing slash:

```
# AISIX_PROXY has no trailing slash or endpoint path

# The local quickstarts use http://127.0.0.1:3000

export AISIX_PROXY="YOUR_AISIX_GATEWAY_URL"
```

Request OpenAI's native model list through AISIX:

```
curl --fail-with-body -sS \

  "$AISIX_PROXY/passthrough/openai/v1/models" \

  -H "Authorization: Bearer $AISIX_API_KEY"
```

The route strips its `path_prefix`, avoids duplicating the `/v1` segment already present in `target_url`, injects the OpenAI provider credential, and relays the upstream response.

## Compare Cloud and Resources-File References[​](#compare-cloud-and-resources-file-references "Direct link to Compare Cloud and Resources-File References")

Route fields use the same names in both management paths. The credential references are IDs in both: `provider_key_id` takes the provider key's `id`, and `anonymous_key_id` takes the anonymous caller API key's `id`. A resources file does not check that either ID names an entry in the file. The AISIX Cloud Admin API validates that the provider key is visible to the environment and that the anonymous caller key belongs to it.

The route `name` is fixed after creation in AISIX Enterprise and AISIX Cloud. In a resources file, the route's identity is its `id`, but `allowed_routes` names the route, so when you rename a route, update every caller's `allowed_routes` entry at the same time.

## Configuration Reference[​](#configuration-reference "Direct link to Configuration Reference")

| Field                    | Behavior                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `name`                   | Route identity referenced by callers' `allowed_routes` patterns and recorded on usage events.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| `path_prefix`            | Gateway path prefix, matched on segment boundaries. A `target_url` route strips it before joining the remaining path to the target. A `preserve_host` route keeps the complete path. A path-only route cannot claim `/v1`, `/mcp`, `/a2a`, `/admin`, `/livez`, `/readyz`, or `/metrics`; a route that also matches `hosts` may use those upstream-owned paths.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| `hosts`                  | Case-insensitive inbound host allowlist; a port is ignored. The inbound host is the request URI authority when present, as in an absolute-form request from another proxy, and otherwise the `Host` header. A leading `*.` wildcard matches one additional label and must retain at least two literal labels.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| `target_url`             | Explicit upstream base URL. Configure exactly one target shape: `target_url`, or `preserve_host: true`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| `preserve_host`          | Derive `https://<matched-host>` as the target. It is accepted only with `hosts`, which bounds the derived destination.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `auth_mode`              | `gateway_key` (default), `header_key`, or `anonymous`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `auth_header_name`       | Lowercase side-channel header containing the gateway credential in `header_key` mode. AISIX strips it before forwarding, unless `forward_client_headers` names it in full; a glob such as `x-*` does not reach it. `authorization`, `proxy-authorization`, `cookie`, `set-cookie`, and `x-api-key` are rejected.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| `source_cidrs`           | Client source allowlist. It is required and nonempty for `anonymous`; for other modes it is optional hardening.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| `credential_mode`        | `inject` (default) or `forward_client`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| `forward_client_headers` | Inbound client headers relayed upstream even when this route would otherwise strip them, as exact names or single-`*` globs, matched case-insensitively. Empty by default. A patch replaces the stored list; send `null` to clear it. In the dashboard it is **Forward client headers** under **Advanced**, one entry per line. See [Upstream Request Headers](https://docs.api7.ai/ai-gateway/models/upstream-request-headers.md#forward-client-headers).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| `identity_header`        | Optional lowercase device-injected identity header. AISIX records a bounded value as `client_identity` and strips the header before forwarding, unless `forward_client_headers` names it in full; a glob such as `x-*` does not reach it. Configure it only behind a trusted device that removes and replaces any client-supplied value. `authorization`, `proxy-authorization`, `cookie`, `set-cookie`, and `x-api-key` are rejected. The AISIX Cloud Admin API also rejects, with `400`, every other header that carries a credential, because the value would be stored on every usage record: `api-key`, `x-goog-api-key`, `x-amz-security-token`, `x-amz-date`, `x-amz-content-sha256`, `x-aisix-api-key`, and the route's own `auth_header_name`. A route saved earlier with one of these keeps loading and recording, but the Admin API refuses any update to it that does not also change `identity_header` or clear it with `null`. In a resources file, never name these headers either. |
| `timeout_ms`             | Bounds upstream response headers and non-SSE body reads. For SSE, it bounds the wait for the first raw upstream body chunk and each gap between raw upstream body chunks. It does not bound the total lifetime of a healthy relay or require a complete SSE frame within that interval. When omitted, the route uses `upstream.stream_timeout_ms`, then `upstream.timeout_ms`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| `enabled`                | A disabled route matches nothing. Default: `true`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |

At least one match dimension, `path_prefix` or `hosts`, is required. When both are present, the request must satisfy both.

## Gateway Authentication Modes[​](#gateway-authentication-modes "Direct link to Gateway Authentication Modes")

* **`gateway_key`** reads the standard gateway credential from `x-aisix-api-key`, `Authorization: Bearer`, or `x-api-key`, in that order of precedence, exactly as `/v1/*` does. The first of those headers present decides, and AISIX never falls back to the next one: a present but empty or malformed `Authorization`, such as `Basic …` or `Bearer` with no token, is rejected with `401` even when `x-api-key` holds a valid key. To keep `Authorization` for the upstream, send the gateway key in `x-aisix-api-key`. See [Send the Gateway Key in `x-aisix-api-key`](https://docs.api7.ai/ai-gateway/reference/proxy-api.md#send-the-gateway-key-in-x-aisix-api-key).
* **`header_key`** reads the gateway credential from `auth_header_name`, leaving `Authorization` available for the caller's upstream credential. This is the standard forward-proxy pairing with `forward_client`.
* **`anonymous`** accepts no gateway credential. The request runs as the configured caller-key principal and must originate within `source_cidrs`.

Every resolved principal still needs the route name granted by `allowed_routes`; `*` grants every route. Entries are route names or single-`*` globs such as `openai-*`. The AISIX Cloud Admin API rejects an entry with two or more `*` with `400`; in a resources file such an entry grants nothing. A valid key without a matching grant receives `403`.

In AISIX Enterprise and AISIX Cloud, budgets that already apply to the resolved caller are checked before dispatch. Passthrough usage currently carries no model ID, is recorded with zero cost, and does not add spend to those budgets. AISIX OSS has no local budget resource.

## Typed Policies and Host Matching[​](#typed-policies-and-host-matching "Direct link to Typed Policies and Host Matching")

Passthrough routes skip the model-based policies of the typed endpoints, and host matching decides which requests reach them:

* **Model policies do not apply.** A route does not authorize the request against a model, so the caller key's `allowed_models` and model-scoped guardrails are not evaluated, even when the body names a model. The only model lookup is for [request limits on `inject` routes](#rate-limits).
* **`allowed_routes` controls access, and it denies by default.** A caller key with no `allowed_routes`, or with an empty list, can use no route. A grant covers everything the upstream serves, with no model-level restriction.
* **Other guardrails still run.** Guardrails attached to the caller key, its team, the environment, or the route apply as usual.
* **Keep health probes off route hosts.** A matching inbound host prevents `/livez` and `/readyz` from reaching the gateway's health endpoints. A host-only route forwards the probe upstream; a route whose `path_prefix` does not include the probe path returns `404`.
* **The inbound host is client-supplied.** Any client that reaches the listener can select a host-matched route through the request URI authority or the `Host` header, so `hosts` does not limit who can use a route. Use `source_cidrs` for that: a request from outside the list receives `403` before authentication. The list is compared with the [resolved client IP](https://docs.api7.ai/ai-gateway/deployment/client-ip-resolution.md).
* **`auth_mode: anonymous` ignores caller credentials.** The route neither reads nor validates them. A request from inside `source_cidrs` runs as the configured caller key, whatever `Authorization` it sends.

## Upstream Credential Modes[​](#upstream-credential-modes "Direct link to Upstream Credential Modes")

* **`inject`** removes every credential slot from the request, whatever the provider key's `strip_headers` lists and whatever the `auth_mode`, then injects the configured provider key. The slots are `authorization`, `proxy-authorization`, `x-api-key`, `api-key`, `x-goog-api-key`, `cookie`, `x-amz-security-token`, `x-amz-date`, and `x-amz-content-sha256`. AISIX also removes the other headers `strip_headers` lists, and the provider key's TLS settings apply. Injection has two shapes: `x-api-key` for Anthropic, with `anthropic-version: 2023-06-01` added only when the caller sent no `anthropic-version` header, and `Authorization: Bearer` for every other provider, including Azure and Gemini provider keys. A caller's own `api-key` or `x-goog-api-key` is therefore not relayed unless `forward_client_headers` names it exactly.
* **`forward_client`** relays the caller's credentials without any `forward_client_headers` entry, and a glob does not change that. AISIX removes only `proxy-authorization`, the route's `identity_header`, and the slot it consumed to authenticate the caller: `authorization` and `x-api-key` under `gateway_key`, or `auth_header_name` under `header_key`. Under `gateway_key`, a caller that sends the gateway key in `x-aisix-api-key`, which is never forwarded, keeps its `Authorization` and `x-api-key` for the upstream. A consumed slot that `forward_client_headers` names exactly is forwarded as well.

In both modes, a route relays the caller's other headers by default and strips a small set: hop-by-hop and transport headers, `host`, `content-length`, the `x-aisix-*` namespace, `proxy-authorization`, and the caller's W3C trace headers. AISIX consumes valid W3C context for its own [OTLP trace](https://docs.api7.ai/ai-gateway/reference/otlp-traces.md#understand-otlp-trace-structure) and sends a gateway request ID upstream.

`forward_client_headers` overrides that strip, and is the one way to put a header back that the route would otherwise remove. A credential slot the route removes comes back only when a pattern names it exactly; `*` and `x-*` do not count. On an `inject` route, naming the slot the route injects into (`x-api-key` for an Anthropic provider key, `authorization` for any other) gives the upstream the caller's value instead of the injected one, and a caller that sends nothing in that slot still gets the injected credential. Naming any other slot sends the caller's value beside the injected credential, so the upstream receives both. Naming `authorization` under `auth_mode: gateway_key` with a non-Anthropic provider key therefore relays the caller's own credential — the very header the gateway consumed to authenticate them — in place of the injected one, which is what lets an internal service that authorizes on the end user keep doing so. `host`, `content-length`, hop-by-hop headers, and `x-aisix-*` are stripped whatever the patterns say, and a credential or trace-context header must be [named exactly](https://docs.api7.ai/ai-gateway/models/upstream-request-headers.md#headers-that-must-be-named-exactly) rather than matched by a wildcard. The route's own `auth_header_name` and `identity_header` are read the same way: AISIX consumes both, so a glob does not reach either and naming one in full is what forwards it. For every other name the route strips, a glob is enough — including a provider key `strip_headers` entry, though three of that list's four defaults (`authorization`, `cookie`, `x-api-key`) are credential slots that still need their own entry, leaving `set-cookie` the only default a glob restores.

There is no credential fallback: an `inject` route without a resolvable provider key fails closed, and a `forward_client` route cannot carry a provider key reference.

### Where an Inject Route Can Send the Key[​](#inject-route-targets "Direct link to Where an Inject Route Can Send the Key")

In AISIX Enterprise and AISIX Cloud, starting with 1.11.0, an `inject` route can send its provider key only to the key's own origin, unless the caller holds `update` permission on that provider key. The built-in owner and admin roles hold it on every key, as does a [custom role](https://docs.api7.ai/ai-gateway/cloud/custom-roles.md) with a rule that grants `update` on `provider_keys` and whose [object scope](https://docs.api7.ai/ai-gateway/cloud/custom-roles.md#object-scope), if any, picks that key. `update` on other provider keys does not count, and neither does `read` on this one: a role that can read every provider key but update only those tagged `vendor=openai` can point a route at another key's own origin, and nowhere else. Permissions on resources inside an environment do not count either, however many of them a role covers, and neither does a role that grants only `create` or `update` on `passthrough_routes`.

The origin is the scheme, host, and effective port of the provider key's `api_base`, or of the provider's default endpoint when the key sets none. A URL without a port uses `80` for `http` and `443` for `https`. For a caller without that permission on the key:

* **`target_url`** must have the same origin. With an `api_base` of `https://api.openai.com/v1`, the target `https://api.openai.com/v1` and `https://api.openai.com:443/v1` are accepted, while `http://api.openai.com/v1`, `https://api.openai.com:8443/v1`, and `https://proxy.example.com/v1` are rejected.
* **`preserve_host`** sends each request to `https://<matched-host>`, so every `hosts` entry is judged as `https://<entry>:443` and must equal the key's origin. A `*.` wildcard entry never qualifies, and a key whose `api_base` uses `http` or a port other than `443` cannot be the target of a `preserve_host` route.

A request that fails the check receives `400`. The control plane runs the check when a route is created, and when an update sets or changes any of `provider_key_id`, `credential_mode`, `target_url`, `hosts`, or `preserve_host`. Routes saved before the check existed keep working and are not re-validated, and an update that leaves those fields alone, such as toggling `enabled` or changing `timeout_ms`, saves as before.

The open-source AISIX gateway does not apply this check to routes in a resources file.

## Envelope Detection and Usage[​](#envelope-detection-and-token-usage "Direct link to Envelope Detection and Usage")

AISIX identifies the protocol of each exchange for extraction only; identification does not change the relayed body. The request body's top-level carrier field selects the envelope. If several recognized carrier fields appear, detection uses this order:

1. `messages` for chat traffic. A request that carries the `anthropic-version` header, or whose path ends in `/messages`, is Anthropic Messages. Any other `messages` body is OpenAI-compatible chat.
2. `input` as a string or an array for the OpenAI Responses shape.
3. `prompt` for legacy completions or fill-in-the-middle traffic.
4. Opaque handling for every other body, including JSON-RPC, REST, non-JSON, and empty bodies.

The envelope also selects the guardrail content rules. See [Passthrough Guardrail Scan Coverage](#passthrough-guardrail-scan-coverage) for the security boundary around malformed, opaque, and supplemental content.

### Usage Recorded per Request Shape[​](#usage-recorded-per-request-shape "Direct link to Usage Recorded per Request Shape")

AISIX reads the usage of an identified exchange with the parser of the typed endpoint that serves the same protocol, so a passthrough route records the same token counts that endpoint records for the same upstream reply. Each token field keeps the meaning its protocol gives it; AISIX never infers a field's meaning from its name. A reply whose usage the protocol's parser cannot read records no usage, as on the typed endpoint.

The request path takes precedence over the body for two surfaces whose bodies carry another envelope's key: a path ending in `/embeddings` is metered as embeddings, and a path ending in `/images/generations` or `/images/edits` as image generation. Among bodies with no envelope, AISIX also meters a rerank body and an Alibaba Cloud DashScope native body. It reads only their `model` field and token counts, and guardrails inspect decoded string values in those JSON bodies.

| Protocol                     | Identified by                                                                                                                 | Usage read as                                                                                                                                                                                                                                              |
| ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| OpenAI-compatible chat       | The `messages` envelope.                                                                                                      | [`/v1/chat/completions`](https://docs.api7.ai/ai-gateway/endpoints/openai-compatible-chat.md) reads it, from a buffered response or each stream chunk.                                                                                                     |
| Anthropic Messages           | The `messages` envelope with `anthropic-version` or a `/messages` path.                                                       | [`/v1/messages`](https://docs.api7.ai/ai-gateway/endpoints/anthropic-messages.md) reads it, including the stream's `message_start` and `message_delta` events. Cache read and cache creation tokens are recorded beside the input tokens.                  |
| Responses                    | The `input` envelope.                                                                                                         | [`/v1/responses`](https://docs.api7.ai/ai-gateway/endpoints/responses-api.md) reads it. A stream reports usage only on its terminal event.                                                                                                                 |
| Completions                  | The `prompt` envelope.                                                                                                        | [`/v1/completions`](https://docs.api7.ai/ai-gateway/endpoints/text-completions.md) reads it.                                                                                                                                                               |
| Embeddings                   | A path ending in `/embeddings`, or a buffered reply that is an `object: "list"` of `object: "embedding"` items.               | [`/v1/embeddings`](https://docs.api7.ai/ai-gateway/endpoints/embeddings.md) reads it: input tokens from `prompt_tokens`, otherwise `total_tokens`. A stream records no usage.                                                                              |
| Image generation and editing | A path ending in `/images/generations` or `/images/edits`.                                                                    | [Image generation](https://docs.api7.ai/ai-gateway/endpoints/image-generation.md) reads it. A stream records no usage.                                                                                                                                     |
| Rerank                       | A top-level `query` and a `documents` array. Covers Cohere, Jina, and Alibaba Cloud Model Studio's compatible rerank API.     | The [rerank endpoint](https://docs.api7.ai/ai-gateway/endpoints/rerank.md) reads it: input tokens from `usage.prompt_tokens`, `usage.input_tokens`, or `usage.total_tokens`, or from Cohere's `meta.billed_units.input_tokens`. A stream records no usage. |
| DashScope native             | A top-level `model` string and an `input` object. Covers native rerank, text and multimodal embedding, and native generation. | See [DashScope Native Token Counts](#dashscope-native-token-counts).                                                                                                                                                                                       |
| Any other body               | Nothing above matched.                                                                                                        | Nothing, except the agent usage frames described below.                                                                                                                                                                                                    |

For a buffered reply to an identified request, the envelope the reply names itself takes precedence over the request: `type: "message"` is read as Anthropic Messages, `output` as Responses, and `choices` as chat or completions. In a stream, a `message_start` or `message_delta` event is read as Anthropic Messages. An Anthropic reply is therefore metered correctly even when the request carried neither signal.

An opaque stream reports usage only through a flat token report on a frame the server labels `event: token_usage` or `event: usage`, the shape IDE agent backends send. AISIX reads `prompt_tokens` there as the whole input, and records `cache_read_input_tokens` and `cache_creation_input_tokens` as cached and cache-write tokens inside it, because such a backend reports its total as `prompt_tokens` + `completion_tokens`. A `usage` object on any other opaque frame is not read.

A stream whose protocol is identified is metered by that protocol's own parser. If the parser reads no usage on any frame of the whole stream, AISIX meters the stream from its `event: token_usage` or `event: usage` frames instead, read the same way. Those frames are never added on top of usage the parser did read.

The access log's `total_tokens` for a passthrough request includes cache creation and cache read tokens, as every typed endpoint's line does. The usage event's `total_tokens` remains the total the upstream reported, so an Anthropic Messages exchange records none, the same as `/v1/messages`.

Every passthrough usage event has the operation `passthrough`, whatever the body shape. See [Tell Request Kinds Apart](https://docs.api7.ai/ai-gateway/observability/metrics-and-usage-events.md#tell-request-kinds-apart). For a recognized shape, `requested_model` is the request body's `model` value. That value is the upstream model ID the caller sent, not an AISIX model alias, because the route does not resolve a model. Every other body records no model. `model_id` is always empty, so passthrough usage records zero cost: pricing is keyed on the AISIX model.

### Metrics for Identified Protocols[​](#metrics-for-identified-protocols "Direct link to Metrics for Identified Protocols")

An exchange identified as chat, Anthropic Messages, Responses, completions, embeddings, image generation or editing, or rerank counts as model inference in the [gateway metrics](https://docs.api7.ai/ai-gateway/reference/metrics.md#request-metrics), under `endpoint="/passthrough_route"` and `inbound_protocol="passthrough"`:

* `aisix_llm_requests_total` and `aisix_llm_request_duration_seconds` count it once, in addition to the proxy request families.
* The `aisix_llm_*_tokens_total` counters, `aisix_llm_tokens_by_client_total`, and `aisix_tokens_consumed_total` record its tokens.
* A streamed reply records time to first token in `aisix_llm_time_to_first_token_seconds` and `aisix_request_ttft_seconds`.

On these series, the request families included, `model` is the request body's `model` when it names a configured model, or the wildcard model whose pattern covers it. Any other value reports `unresolved`, and a body without `model` reports `unknown`. `upstream_model` is `unknown`. A DashScope native body, an agent usage frame, and an opaque relay stay in the proxy request families only, with `model="passthrough"`.

Every passthrough request, identified or not, records `aisix_request_e2e_latency_seconds` under `endpoint="/passthrough_route"`, with an upstream side when the call was relayed. A streamed reply records time to its first frame in both time-to-first-token series.

### DashScope Native Token Counts[​](#dashscope-native-token-counts "Direct link to DashScope Native Token Counts")

For a DashScope native body, AISIX reads the response's top-level `usage` object as follows:

* Output tokens come from `output_tokens`, or `completion_tokens`.
* When `total_tokens` is present, input tokens are `total_tokens` minus the output tokens.
* Without `total_tokens`, input tokens are `input_tokens`, or `prompt_tokens`, plus the top-level `image_tokens`. A nested `input_tokens_details.image_tokens` value is already part of `input_tokens` and is not added.
* Cached-input and reasoning token counts are read from the same object with the standard field names, such as `prompt_tokens_details.cached_tokens` and `output_tokens_details.reasoning_tokens`.

This counts image tokens once, whether a service reports them inside `input_tokens` or next to it:

| Service                                         | Upstream `usage`                                                                | Recorded input | Recorded output |
| ----------------------------------------------- | ------------------------------------------------------------------------------- | -------------- | --------------- |
| Native multimodal rerank, `qwen3-vl-rerank`     | `input_tokens: 146`, `image_tokens: 64`, `total_tokens: 210`                    | 210            | 0               |
| Multimodal embedding, `multimodal-embedding-v1` | `input_tokens: 4`, `image_tokens: 128`                                          | 132            | 0               |
| Native multimodal generation                    | `input_tokens: 79`, `image_tokens: 66`, `output_tokens: 14`, `total_tokens: 93` | 79             | 14              |

### Example: Meter Alibaba Cloud Native Rerank[​](#example-meter-alibaba-cloud-native-rerank "Direct link to Example: Meter Alibaba Cloud Native Rerank")

The following control-plane example relays DashScope's native API for the mainland China region through `/passthrough/dashscope`. Create an `alibaba-cn` provider key for the DashScope credential. For the international region, use the `alibaba` provider and `https://dashscope-intl.aliyuncs.com/api/v1`. The provider key's `api_base` does not affect this route, because the route's `target_url` sets the upstream:

```
export DASHSCOPE_API_KEY="YOUR_DASHSCOPE_API_KEY"



DASHSCOPE_KEY_ID=$(curl --fail-with-body -sS -X POST \

  "$AISIX_CP/provider_keys" \

  -H "Authorization: Bearer $AISIX_TOKEN" \

  -H "Content-Type: application/json" \

  -d '{

    "display_name": "DashScope native",

    "provider": "alibaba-cn",

    "api_key": "'"${DASHSCOPE_API_KEY}"'",

    "allowed_environments": ["'"${ENV_ID}"'"]

  }' | jq -er '.provider_key.id')



curl --fail-with-body -sS -X POST \

  "$AISIX_CP/environments/$ENV_ID/passthrough_routes" \

  -H "Authorization: Bearer $AISIX_TOKEN" \

  -H "Content-Type: application/json" \

  -d '{

    "name": "dashscope-native",

    "path_prefix": "/passthrough/dashscope",

    "target_url": "https://dashscope.aliyuncs.com/api/v1",

    "credential_mode": "inject",

    "provider_key_id": "'"${DASHSCOPE_KEY_ID}"'"

  }' | jq '.warnings // []'
```

In a resources file, declare the same route with `provider_key_id` set to the `id` of an `alibaba-cn` provider key. [Qwen](https://docs.api7.ai/ai-gateway/providers/qwen.md#configure-with-aisix-oss) shows a provider key entry:

resources.yaml (route)

```
passthrough_routes:

  - id: 452fc855-7073-4de5-a2c4-5f2192f66566

    name: dashscope-native

    path_prefix: /passthrough/dashscope

    target_url: https://dashscope.aliyuncs.com/api/v1

    provider_key_id: a619ae19-b3ed-4ff8-8c3e-ffff9fa80cc0
```

Grant `dashscope-native` in the caller key's `allowed_routes`, as in [Configure a Provider-Native Route](#create-a-passthrough-route). Then send a native rerank request with the upstream model ID:

```
curl --fail-with-body -sS \

  "$AISIX_PROXY/passthrough/dashscope/services/rerank/text-rerank/text-rerank" \

  -H "Authorization: Bearer $AISIX_API_KEY" \

  -H "Content-Type: application/json" \

  -d '{

    "model": "gte-rerank-v2",

    "input": {

      "query": "What does a passthrough route record?",

      "documents": [

        "A passthrough route relays provider-native requests.",

        "Usage events carry token counts and the route name."

      ]

    },

    "parameters": {"top_n": 2}

  }'
```

AISIX strips the prefix, relays the body to `https://dashscope.aliyuncs.com/api/v1/services/rerank/text-rerank/text-rerank` with the injected credential, and returns DashScope's response unchanged. The body's top-level `model` and `input` object make it a DashScope native body, so the usage event records `gte-rerank-v2` as `requested_model` and the response's `usage.total_tokens` as input tokens.

## Rate Limits[​](#rate-limits "Direct link to Rate Limits")

Caller API-key, team, and member request limits apply before dispatch. On an `inject` route, a top-level JSON `model` that resolves to a configured AISIX model of the same provider also reserves that model's request limits. `forward_client` routes do not perform this model lookup.

Request-count dimensions (`rps`, `rpm`, `rph`, and `rpd`) are enforced. AISIX also rejects a request when an applicable `tpm` or `tpd` counter is already exhausted, but passthrough token usage does not increment those counters. Use recorded usage for telemetry, not passthrough token-quota enforcement.

Concurrency is checked before upstream dispatch. Every matching SSE concurrency reservation remains held for the lifetime of the stream. Redis-backed limits renew the reservation from acquisition until the stream ends and use `concurrency_ttl_secs` to recover one that is no longer renewed. If Redis reports that a live reservation no longer exists, AISIX ends the relay. The caller has already received `200`, while the terminal usage event records status `429`, `error_class="rate_limit_lease_lost"`, and `error_message="distributed concurrency lease was lost"`. See [Redis-backed concurrency leases](https://docs.api7.ai/ai-gateway/traffic-controls/rate-limits.md#choose-rate-limit-counter-storage) for failure and rollout behavior.

Passthrough routes have no rate-limit field or policy scope. To apply different request limits to different routes, grant them to separate caller keys and configure limits for those caller identities.

## Guardrails and Streaming[​](#guardrails-and-streaming "Direct link to Guardrails and Streaming")

In AISIX Enterprise and AISIX Cloud, a guardrail can be attached to one passthrough route by selecting the **Passthrough routes** scope. The attachment uses the route UUID. Environment, caller-key, and team guardrails can also apply.

The open-source resources file declares attachments in its `guardrail_attachments` collection, so a file-defined guardrail reaches passthrough traffic only if an attachment scopes it there — `scope_type: env`, or `scope_type: passthrough_route` naming the route.

Input guardrails run before upstream dispatch. A block returns `422` without contacting the upstream. Successful buffered responses are checked before delivery. Once a successful SSE response has started, a block ends the stream with an SSE `content_filter` error frame; it cannot change the HTTP status to `422`.

SSE responses relay incrementally unless a hold-back guardrail buffers frames for inspection. AISIX does not rewrite provider-native bodies to apply redaction. For the built-in `pii` guardrail, a mask-action-only match is forwarded without masking and recorded as `mask_unsupported`, or `would_mask_unsupported` in monitor mode; use a block action when the matched content must not reach the upstream. Guardrail kinds such as Presidio and Lakera instead block a maskable result when passthrough has no write-back channel. See [Surfaces That Cannot Apply a Mask](https://docs.api7.ai/ai-gateway/traffic-controls/guardrails/behavior.md#surfaces-that-cannot-apply-a-mask).

### Passthrough Guardrail Scan Coverage[​](#passthrough-guardrail-scan-coverage "Direct link to Passthrough Guardrail Scan Coverage")

Guardrails use protocol-aware extraction. On successfully decoded requests, they also inspect ordinary JSON strings forwarded alongside the protocol's canonical fields, including nested values. Output scanning applies only to successful upstream responses. AISIX relays non-2xx buffered bodies and SSE streams without inspecting their content. Successful response coverage remains protocol-specific and is limited to the fields listed below.

A successful response with a non-identity `Content-Encoding` cannot be inspected. When an applicable output guardrail fails closed, AISIX refuses the response with `unscannable_body`. When every applicable output guardrail fails open, AISIX relays the original encoded body and records an `unscannable_body` bypass. This behavior applies to both buffered and SSE responses.

Tool-call IDs are opaque handles, not caller content, so the request scan leaves them out of a recognized envelope:

* Chat and Anthropic Messages: each tool call's `id`, `call_id`, `tool_use_id`, and `tool_call_id`, and a `tool` message's `tool_call_id`.
* Responses: `call_id` and `approval_request_id` on every item; the `id` of any `*_call` or `*_call_output` item and of an `item_reference`, `mcp_approval_request`, or `reasoning` item; and a `reasoning` item's `encrypted_content`. A passthrough Responses body is therefore scanned the same way as a `/v1/responses` request.

Function names, arguments, and tool results are still scanned, and the response-side fields below never include IDs. A guardrail rule that matches only text inside one of the excluded IDs does not fire. The **Any other body** row is unaffected and still reads every decoded string.

If AISIX recognizes an envelope but cannot decode it, AISIX does not fall back to scanning the raw body. A top-level `model` is routing metadata, not caller content. Typed opaque content is not reinterpreted as text: this includes image, audio, document, and file payloads, along with `redacted_thinking` data. The Chat row therefore covers both OpenAI-compatible chat and Anthropic Messages traffic:

| Envelope                   | Request                                                                                                                                                                                                                                                                                                                                                                                                                                     | Response                                                                                                                                                                                                                                                                                                                         |
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Chat or Anthropic Messages | Message text and text-bearing content blocks, the Anthropic top-level `system`, `tool_result` content, chat `tool_calls`, and replayed `tool_use` input, `thinking`, and `reasoning_content`. Replayed reasoning content parts contribute each entry's `text`: a Mistral `thinking` array, a Databricks `reasoning` part's `summary`, and an Anthropic-shaped `thinking` string. Their `signature` and `encrypted_content` are not scanned. | Chat message text and refusals, plus function and custom tool names and arguments. For Anthropic, buffered replies scan `text` blocks and `tool_use` names and input; streams scan `tool_use` starts plus `text` and `partial_json` deltas.                                                                                      |
| Responses                  | Every input item the `/v1/responses` endpoint reads, including a replayed tool call's name, arguments, and input.                                                                                                                                                                                                                                                                                                                           | Output text, refusals, and tool calls: message text and refusal parts; `function_call` and `mcp_call` names and arguments; and `custom_tool_call` names and input in a buffered reply. Stream carriers include the corresponding delta and done events, content-part and output-item snapshots, and terminal response snapshots. |
| Completions                | `prompt` and `suffix`.                                                                                                                                                                                                                                                                                                                                                                                                                      | Completion `text`.                                                                                                                                                                                                                                                                                                               |
| Any other body             | Every decoded JSON string value when the body is JSON; otherwise the textual body.                                                                                                                                                                                                                                                                                                                                                          | A buffered JSON body contributes its decoded string values; other buffered text is scanned directly. An opaque SSE stream can scan only a bare JSON string. Other content-bearing payload shapes cannot be evaluated because AISIX cannot reliably associate their text with the preceding stream content.                       |

Within a recognized Responses envelope, known media, reasoning, and unknown output items remain opaque. Malformed recognized buffered output cannot be evaluated and follows the configured failure policy. Unknown Responses stream events remain opaque.

A buffered reply that names two envelopes cannot be evaluated either: a Responses `output` beside chat `choices`, or chat `choices` beside an Anthropic `type: "message"` body. Each extraction rule reads only its own carrier, so a fail-closed output guardrail refuses such a reply with `unscannable_body`, and a fail-open one releases it. See [What Happens to Content AISIX Cannot Read](https://docs.api7.ai/ai-gateway/traffic-controls/guardrails/behavior.md#what-happens-to-content-aisix-cannot-read).

Reasoning the model generates, such as `reasoning_content`, Anthropic `thinking`, and Responses reasoning summaries, is not scanned on the response side. It still counts toward a held stream's `max_buffer_bytes`.

## Audit Capture[​](#audit-capture "Direct link to Audit Capture")

For successfully relayed traffic, an observability exporter with `content_mode: full` receives the request body as a string, subject to the exporter's content cap. Buffered responses record extracted text when the response matches a supported extraction shape and otherwise record the body as text. Streamed responses record accumulated extracted text; opaque data payloads are retained as text. Base64 media is replaced in each, as described in [Base64 Media in Captured Content](https://docs.api7.ai/ai-gateway/observability/exporters.md#base64-media-in-captured-content). Captured content is exporter-only and is never sent through the control-plane telemetry path.

Usage events carry the route name, caller, recorded token counts, and `client_identity`. External exporters can expose these values. The current control-plane Request Logs view shows caller and token metadata, but it does not display `passthrough_route_name` or `client_identity`.

## Errors[​](#errors "Direct link to Errors")

| Status or signal                             | Meaning                                                                                                                                                                                                                                                                                                            |
| -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `401`                                        | Missing or invalid gateway credential for the route's authentication mode.                                                                                                                                                                                                                                         |
| `403`                                        | The resolved caller key does not grant the route, or the client source is outside `source_cidrs`.                                                                                                                                                                                                                  |
| `404`                                        | An unclaimed `/passthrough/*` path, or a path no route claims on a host that matches a route's `hosts`, reaches the ordinary empty-body not-found path.                                                                                                                                                            |
| `422`                                        | A guardrail blocked the request before dispatch or blocked a buffered response before delivery.                                                                                                                                                                                                                    |
| SSE `content_filter` frame                   | A guardrail blocked content after a stream had started.                                                                                                                                                                                                                                                            |
| `429`                                        | A gateway request limit or budget check rejected the request, or the upstream returned a relayed `429`.                                                                                                                                                                                                            |
| Terminal usage status `429` after SSE starts | A Redis-backed concurrency lease disappeared after the caller received `200`. The usage event records `error_class="rate_limit_lease_lost"`; see [Streams That Fail After the Response Head](https://docs.api7.ai/ai-gateway/observability/metrics-and-usage-events.md#streams-that-fail-after-the-response-head). |
| Other upstream status                        | AISIX relays the upstream status and body after filtering response headers.                                                                                                                                                                                                                                        |

The unmatched `404` has an empty body. Other AISIX-generated failures use the gateway error envelope; upstream error statuses and bodies are relayed after response-header filtering.

## Next Steps[​](#next-steps "Direct link to Next Steps")

* Receive TLS-terminated IDE traffic with host-matched routes: [Forward Proxy for IDE AI Traffic](https://docs.api7.ai/ai-gateway/deployment/forward-proxy.md).
* Configure guardrail behavior: [Guardrail Behavior](https://docs.api7.ai/ai-gateway/traffic-controls/guardrails/behavior.md).
* Export request and response content: [Observability Exporters](https://docs.api7.ai/ai-gateway/observability/exporters.md).
