Skip to main content
API7.ai Logo
Version: Dev

Passthrough Routes

A passthrough route forwards matching requests to one upstream target without translating the request body. It is useful for provider-native endpoints that AISIX does not model as first-class routes and for forward-proxy traffic delivered with its original Host header.

Each route defines how traffic matches, where it goes, how AISIX authenticates the caller, and whether AISIX injects a provider credential or forwards the caller's credential. AISIX can still apply caller access controls, request limits, guardrails, and telemetry around the relay.

Explicit route required

Every passthrough path or host must be claimed by an explicit route. An unclaimed /passthrough/* path follows the ordinary empty-body 404 path. Create and verify the route before moving client traffic.

Passthrough routes do not rewrite model identifiers. If a provider-native request names a model in its body, path, query, or headers, send the identifier expected by that provider.

Prerequisites​

Prepare the following:

  • One AISIX deployment:
    • For AISIX Cloud, an environment with an attached gateway and a write-scoped admin token.
    • For the open-source AISIX gateway, a gateway configured to load a declarative resources file.
  • An upstream provider credential for an inject route. The examples use OpenAI. A forward_client route instead relays the caller's upstream credential.
  • curl and jq.

Understand the Passthrough Flow​

AISIX preserves the request body while handling gateway authentication, target construction, intentional header filtering, guardrails, and telemetry:

Each matched request makes one upstream attempt. AISIX does not retry a passthrough request after a transport failure or upstream 5xx response. A route failure also does not place any configured model into cooldown because the route does not resolve a model.

URL rewriting runs before route matching. Both stages below use the rewritten path; optional hosts on a rewrite rule can keep other hosts' paths unchanged. Route matching then runs in two stages:

  1. A request whose inbound Host matches a route's hosts allowlist is dispatched before the gateway's typed routes. This lets a forward proxy relay an upstream path such as /v1/messages without the gateway treating it as its own endpoint.
  2. Path-prefix matching runs after the typed routes, so a path-only route cannot shadow the gateway's /v1, /mcp, or /a2a endpoints.

When several routes match, host matches beat path-only matches, and a longer matching prefix beats a shorter one.

Configure a Provider-Native Route​

The following examples expose OpenAI's native model-list endpoint at /passthrough/openai/v1/models. AISIX injects the configured OpenAI credential and requires a caller key that grants openai-tunnel.

AISIX Cloud​

Export the AISIX Cloud connection details and the provider credential:

# AISIX_CP includes /api and has no trailing slash
# The local On-Premises quickstart uses http://localhost:8080/api
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"
export OPENAI_API_KEY="YOUR_OPENAI_API_KEY"

Create an OpenAI provider key that is available to the environment:

PROVIDER_KEY_ID=$(curl --fail-with-body -sS -X POST \
  "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "OpenAI passthrough",
    "provider": "openai",
    "api_key": "'"${OPENAI_API_KEY}"'",
    "api_base": "https://api.openai.com/v1",
    "allowed_environments": ["'"${ENV_ID}"'"]
  }' | jq -er '.provider_key.id')

Create the route with the provider key ID:

ROUTE_RESPONSE=$(curl --fail-with-body -sS -X POST \
  "$AISIX_CP/environments/$ENV_ID/passthrough_routes" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "openai-tunnel",
    "path_prefix": "/passthrough/openai",
    "target_url": "https://api.openai.com/v1",
    "credential_mode": "inject",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }')

export ROUTE_ID=$(printf '%s' "$ROUTE_RESPONSE" | jq -er '.passthrough_route.id')
printf '%s' "$ROUTE_RESPONSE" | jq '.warnings // []'

Keep ROUTE_ID for route updates or route-scoped guardrail attachments.

Create a dedicated caller key and grant the route in the same request. The plaintext is returned once:

CALLER_RESPONSE=$(curl --fail-with-body -sS -X POST \
  "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "OpenAI passthrough caller",
    "allowed_models": [],
    "allowed_routes": ["openai-tunnel"]
  }')

export AISIX_API_KEY=$(printf '%s' "$CALLER_RESPONSE" | jq -er '.plaintext')
printf '%s' "$CALLER_RESPONSE" | jq '.warnings // []'

The control plane projects the provider key, route, and caller grant to attached gateways. Review any returned compatibility warnings before rollout. Warnings are advisory, so verify traffic through each gateway. In the dashboard, the same workflow is available under Provider keys, an environment's Passthrough Routes, and the caller key's Passthrough route access section.

If you grant an existing caller instead, include every route grant it should keep. The AISIX Cloud Admin API replaces the complete allowed_routes list when that field is patched.

Open-Source AISIX Gateway​

Prepare an openai-prod provider key in the complete resources file, then choose a dedicated caller credential:

export PASSTHROUGH_CALLER_KEY="YOUR_CALLER_API_KEY"

Add the route and caller entries to the matching collections. Keep the provider key and other resources unchanged:

resources.yaml (route and caller key)
passthrough_routes:
  - name: openai-tunnel
    path_prefix: /passthrough/openai
    target_url: https://api.openai.com/v1
    provider_key: openai-prod

api_keys:
  - display_name: passthrough-caller
    key_env: PASSTHROUGH_CALLER_KEY
    allowed_models: []
    allowed_routes: [openai-tunnel]

The provider_key name is resolved to the provider key's derived ID when AISIX loads the file. An unknown name fails validation. Exact entries in allowed_routes are also checked against the routes defined in the file; wildcard patterns are allowed.

Validate the assembled complete file before loading it:

aisix validate --resources resources.yaml

Because this example introduces PASSTHROUGH_CALLER_KEY, start or recreate the gateway with that variable in its process environment. After it loads, use the same value for the verification request:

export AISIX_API_KEY="$PASSTHROUGH_CALLER_KEY"

Verify the Route​

Export the gateway origin without a trailing slash:

# AISIX_PROXY has no trailing slash or endpoint path
# The local quickstarts use http://127.0.0.1:3000
export AISIX_PROXY="YOUR_AISIX_GATEWAY_URL"

Request OpenAI's native model list through AISIX:

curl --fail-with-body -sS \
  "$AISIX_PROXY/passthrough/openai/v1/models" \
  -H "Authorization: Bearer $AISIX_API_KEY"

The route strips its path_prefix, avoids duplicating the /v1 segment already present in target_url, injects the OpenAI provider credential, and relays the upstream response.

Compare Cloud and Resources-File References​

Most route fields use the same names in both management paths. References to provider and caller credentials differ:

PurposeAISIX Cloud Admin APIRecommended resources-file fieldExplicit ID accepted in a resources file
Injected provider credentialprovider_key_id (UUID)provider_key (display_name)provider_key_id
Anonymous caller principalanonymous_key_id (UUID)anonymous_key (display_name)anonymous_key_id

Name references in a resources file receive stronger load-time checking and produce candidate names when a reference is unknown. Prefer them over explicit IDs. AISIX Cloud validates that the provider key is visible to the environment and that the anonymous caller key belongs to it.

The route name is fixed after creation in AISIX Cloud. In a resources file, changing the name changes the route identity, so update every caller's allowed_routes entry at the same time.

Configuration Reference​

FieldBehavior
nameRoute identity referenced by callers' allowed_routes patterns and recorded on usage events.
path_prefixGateway path prefix, matched on segment boundaries. A target_url route strips it before joining the remaining path to the target. A preserve_host route keeps the complete path. A path-only route cannot claim /v1, /mcp, /a2a, /admin, /livez, /readyz, or /metrics; a route that also matches hosts may use those upstream-owned paths.
hostsCase-insensitive inbound Host allowlist; a port is ignored. A leading *. wildcard matches one additional label and must retain at least two literal labels.
target_urlExplicit upstream base URL. Configure exactly one target shape: target_url, or preserve_host: true.
preserve_hostDerive https://<matched-host> as the target. It is accepted only with hosts, which bounds the derived destination.
auth_modegateway_key (default), header_key, or anonymous.
auth_header_nameLowercase side-channel header containing the gateway credential in header_key mode. AISIX strips it before forwarding, unless forward_client_headers names it in full; a glob such as x-* does not reach it. authorization, proxy-authorization, cookie, set-cookie, and x-api-key are rejected.
source_cidrsClient source allowlist. It is required and nonempty for anonymous; for other modes it is optional hardening.
credential_modeinject (default) or forward_client.
forward_client_headersInbound client headers relayed upstream even when this route would otherwise strip them, as exact names or single-* globs, matched case-insensitively. Empty by default. A patch replaces the stored list; send null to clear it. In the dashboard it is Forward client headers under Advanced, one entry per line. See Upstream Request Headers.
identity_headerOptional lowercase device-injected identity header. AISIX records a bounded value as client_identity and strips the header before forwarding, unless forward_client_headers names it in full; a glob such as x-* does not reach it. Configure it only behind a trusted device that removes and replaces any client-supplied value. authorization, proxy-authorization, cookie, set-cookie, and x-api-key are rejected.
timeout_msBounds upstream response headers and non-SSE body reads. It does not bound a healthy SSE relay.
enabledA disabled route matches nothing. Default: true.

At least one match dimension, path_prefix or hosts, is required. When both are present, the request must satisfy both.

Gateway Authentication Modes​

  • gateway_key reads the standard gateway credential from Authorization: Bearer or x-api-key.
  • header_key reads the gateway credential from auth_header_name, leaving Authorization available for the caller's upstream credential. This is the standard forward-proxy pairing with forward_client.
  • anonymous accepts no gateway credential. The request runs as the configured caller-key principal and must originate within source_cidrs.

Every resolved principal still needs the route name granted by allowed_routes; * grants every route. A valid key without a matching grant receives 403.

In AISIX Cloud, budgets that already apply to the resolved caller are checked before dispatch. Passthrough usage currently carries no model ID, is recorded with zero cost, and does not add spend to those budgets. The open-source AISIX gateway has no local budget resource.

Upstream Credential Modes​

  • inject strips inbound credential headers and injects the configured provider key. AISIX uses x-api-key plus anthropic-version for Anthropic and Authorization: Bearer for other providers. The provider key's configured header-strip and TLS settings also apply.
  • forward_client forwards the caller's upstream credential when the gateway credential arrived through header_key, or when the route is anonymous. With gateway_key, AISIX removes Authorization and x-api-key because either may contain the gateway credential.

A route relays the caller's other headers by default and strips a small set: hop-by-hop and transport headers, host, content-length, the x-aisix-* namespace, proxy-authorization, and the caller's W3C trace headers. AISIX consumes valid W3C context for its own OTLP trace and sends a gateway request ID upstream.

forward_client_headers overrides that strip, and is the one way to put a header back that the route would otherwise remove. Naming authorization under auth_mode: gateway_key therefore relays the caller's own credential — the very header the gateway consumed to authenticate them — in place of the injected one, which is what lets an internal service that authorizes on the end user keep doing so. host, content-length, hop-by-hop headers, and x-aisix-* are stripped whatever the patterns say, and a credential or trace-context header must be named exactly rather than matched by a wildcard. The route's own auth_header_name and identity_header are read the same way: AISIX consumes both, so a glob does not reach either and naming one in full is what forwards it. For every other name the route strips, a glob is enough — including a provider key strip_headers entry, though three of that list's four defaults (authorization, cookie, x-api-key) are credential slots that still need their own entry, leaving set-cookie the only default a glob restores.

There is no credential fallback: an inject route without a resolvable provider key fails closed, and a forward_client route cannot carry a provider key reference.

Envelope Detection and Usage​

AISIX detects the request shape for extraction only; detection does not change the relayed body. If several recognized carrier fields appear, detection uses this order:

  1. messages for OpenAI-compatible chat or Anthropic Messages traffic.
  2. input as a string or an array for the OpenAI Responses shape.
  3. prompt for legacy completions or fill-in-the-middle traffic.
  4. Opaque handling for every other body, including JSON-RPC, REST, non-JSON, and empty bodies.

Detection selects the text presented to guardrails and the token fields recorded on usage events. If a detected shape yields no text, AISIX scans the complete body instead. Request and response bodies are still relayed without schema translation.

Usage Recorded per Request Shape​

Two opaque request shapes are still read for usage: a rerank body and an Alibaba Cloud DashScope native body. They stay opaque for guardrails, audit capture, and streamed responses, and AISIX reads only their model field and the token counts of a buffered JSON response:

Request bodyBuffered responseSSE response
Chat, Responses, or completions, detected as aboveEvery supported token field of the response usage object.Usage reported by the stream's own events.
Rerank: a top-level query and a documents array. Covers Cohere, Jina, and Alibaba Cloud Model Studio's compatible rerank API.Input tokens from usage.prompt_tokens, usage.input_tokens, or usage.total_tokens, or from Cohere's meta.billed_units.input_tokens. This is the same reading as the rerank endpoint.Opaque stream rules.
DashScope native: a top-level model string and an input object. Covers native rerank, text and multimodal embedding, and native generation.See DashScope Native Token Counts.Opaque stream rules.
Any other bodyNothing. Opaque buffered responses do not receive speculative token extraction.Opaque stream rules.

Under the opaque stream rules, a stream reports usage through a top-level usage object on any frame, read with the standard field names, or through a flat token report on a frame the server labels event: usage or event: token_usage.

Every passthrough usage event has the operation passthrough, whatever the body shape. See Tell Request Kinds Apart. For a recognized shape, requested_model is the request body's model value. That value is the upstream model ID the caller sent, not an AISIX model alias, because the route does not resolve a model. Every other body records no model. model_id is always empty, so passthrough usage records zero cost: pricing is keyed on the AISIX model.

DashScope Native Token Counts​

For a DashScope native body, AISIX reads the response's top-level usage object as follows:

  • Output tokens come from output_tokens, or completion_tokens.
  • When total_tokens is present, input tokens are total_tokens minus the output tokens.
  • Without total_tokens, input tokens are input_tokens, or prompt_tokens, plus the top-level image_tokens. A nested input_tokens_details.image_tokens value is already part of input_tokens and is not added.
  • Cached-input and reasoning token counts are read from the same object with the standard field names, such as prompt_tokens_details.cached_tokens and output_tokens_details.reasoning_tokens.

This counts image tokens once, whether a service reports them inside input_tokens or next to it:

ServiceUpstream usageRecorded inputRecorded output
Native multimodal rerank, qwen3-vl-rerankinput_tokens: 146, image_tokens: 64, total_tokens: 2102100
Multimodal embedding, multimodal-embedding-v1input_tokens: 4, image_tokens: 1281320
Native multimodal generationinput_tokens: 79, image_tokens: 66, output_tokens: 14, total_tokens: 937914

Example: Meter Alibaba Cloud Native Rerank​

The following AISIX Cloud example relays DashScope's native API for the mainland China region through /passthrough/dashscope. Create an alibaba-cn provider key for the DashScope credential. For the international region, use the alibaba provider and https://dashscope-intl.aliyuncs.com/api/v1. The provider key's api_base does not affect this route, because the route's target_url sets the upstream:

export DASHSCOPE_API_KEY="YOUR_DASHSCOPE_API_KEY"

DASHSCOPE_KEY_ID=$(curl --fail-with-body -sS -X POST \
  "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "DashScope native",
    "provider": "alibaba-cn",
    "api_key": "'"${DASHSCOPE_API_KEY}"'",
    "allowed_environments": ["'"${ENV_ID}"'"]
  }' | jq -er '.provider_key.id')

curl --fail-with-body -sS -X POST \
  "$AISIX_CP/environments/$ENV_ID/passthrough_routes" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "dashscope-native",
    "path_prefix": "/passthrough/dashscope",
    "target_url": "https://dashscope.aliyuncs.com/api/v1",
    "credential_mode": "inject",
    "provider_key_id": "'"${DASHSCOPE_KEY_ID}"'"
  }' | jq '.warnings // []'

In a resources file, declare the same route with provider_key set to the display_name of an alibaba-cn provider key. Qwen shows a provider key entry:

resources.yaml (route)
passthrough_routes:
  - name: dashscope-native
    path_prefix: /passthrough/dashscope
    target_url: https://dashscope.aliyuncs.com/api/v1
    provider_key: dashscope-cn

Grant dashscope-native in the caller key's allowed_routes, as in Configure a Provider-Native Route. Then send a native rerank request with the upstream model ID:

curl --fail-with-body -sS \
  "$AISIX_PROXY/passthrough/dashscope/services/rerank/text-rerank/text-rerank" \
  -H "Authorization: Bearer $AISIX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gte-rerank-v2",
    "input": {
      "query": "What does a passthrough route record?",
      "documents": [
        "A passthrough route relays provider-native requests.",
        "Usage events carry token counts and the route name."
      ]
    },
    "parameters": {"top_n": 2}
  }'

AISIX strips the prefix, relays the body to https://dashscope.aliyuncs.com/api/v1/services/rerank/text-rerank/text-rerank with the injected credential, and returns DashScope's response unchanged. The body's top-level model and input object make it a DashScope native body, so the usage event records gte-rerank-v2 as requested_model and the response's usage.total_tokens as input tokens.

Rate Limits​

Caller API-key, team, and member request limits apply before dispatch. On an inject route, a top-level JSON model that resolves to a configured AISIX model of the same provider also reserves that model's request limits. forward_client routes do not perform this model lookup.

Request-count dimensions (rps, rpm, rph, and rpd) are enforced. AISIX also rejects a request when an applicable tpm or tpd counter is already exhausted, but passthrough token usage does not increment those counters. Use recorded usage for telemetry, not passthrough token-quota enforcement.

Concurrency is checked before upstream dispatch. For SSE, the reservation is released when AISIX returns the streaming response, not when the stream ends.

Passthrough routes have no rate-limit field or policy scope. To apply different request limits to different routes, grant them to separate caller keys and configure limits for those caller identities.

Guardrails and Streaming​

In AISIX Cloud, a guardrail can be attached to one passthrough route by selecting the Passthrough routes scope. The attachment uses the route UUID. Environment, caller-key, and team guardrails can also apply.

The open-source resources file declares attachments in its guardrail_attachments collection, so a file-defined guardrail reaches passthrough traffic only if an attachment scopes it there — scope_type: env, or scope_type: passthrough_route naming the route.

Input guardrails run before upstream dispatch. A block returns 422 without contacting the upstream. Buffered responses are checked before delivery. Once an SSE response has started, a block ends the stream with an SSE content_filter error frame; it cannot change the HTTP status to 422.

SSE responses relay incrementally unless a hold-back guardrail buffers frames for inspection. AISIX does not rewrite provider-native bodies to apply redaction. For the built-in pii guardrail, a mask-action-only match is forwarded without masking; use a block action when the matched content must not reach the upstream. Guardrail kinds such as Presidio and Lakera instead block a maskable result when passthrough has no write-back channel.

Audit Capture​

For successfully relayed traffic, an observability exporter with content_mode: full receives the request body as a string, subject to the exporter's content cap. Buffered responses record extracted text when the response matches a supported extraction shape and otherwise record the body as text. Streamed responses record accumulated extracted text; opaque data payloads are retained as text. Captured content is exporter-only and is never sent through the AISIX Cloud telemetry path.

Usage events carry the route name, caller, recorded token counts, and client_identity. External exporters can expose these values. The current AISIX Cloud Request Logs UI shows caller and token metadata, but it does not display passthrough_route_name or client_identity.

Errors​

Status or signalMeaning
401Missing or invalid gateway credential for the route's authentication mode.
403The resolved caller key does not grant the route, or the client source is outside source_cidrs.
404An unclaimed /passthrough/* path reaches the ordinary empty-body not-found path.
422A guardrail blocked the request before dispatch or blocked a buffered response before delivery.
SSE content_filter frameA guardrail blocked content after a stream had started.
429A gateway request limit or budget check rejected the request, or the upstream returned a relayed 429.
Other upstream statusAISIX relays the upstream status and body after filtering response headers.

The unmatched 404 has an empty body. Other AISIX-generated failures use the gateway error envelope; upstream error statuses and bodies are relayed after response-header filtering.

Next Steps​