Skip to main content
API7.ai Logo

Provider Passthrough

Provider passthrough lets an application call a provider-native endpoint through AISIX when the endpoint is not modeled as a first-class gateway route.

AISIX exposes ANY /passthrough/:provider/*rest for these requests. Passthrough keeps gateway authentication and upstream credential injection. It forwards provider-native request bodies without endpoint-specific translation and relays upstream statuses and response bodies unchanged. Errors generated by AISIX use the gateway's OpenAI-style error envelope.

info

Passthrough does not rewrite model identifiers. If the provider-native request identifies a model in the body, path, query string, or a header, send the identifier expected by the provider. A top-level body model can be matched to an AISIX model for rate limiting, but AISIX still forwards the original value instead of replacing a display_name with its configured upstream model_name.

In this guide, you will send a provider passthrough request and review when this fallback path is appropriate.

Prerequisites​

Before starting, prepare the following:

  • A running AISIX gateway that can serve proxy requests.
  • A caller API key allowed to access at least one model for the requested provider.
  • A provider key with an api_base that can reach the provider-native endpoint, unless the provider has a built-in default base URL.

Export the gateway connection and caller API key used by the examples:

# AISIX_PROXY has no trailing slash or endpoint path.
# The local quickstarts use http://127.0.0.1:3000.
export AISIX_PROXY="YOUR_AISIX_GATEWAY_ORIGIN"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"

Understand the Passthrough Flow​

Passthrough preserves the provider-native request schema while AISIX handles gateway-level processing, including caller authentication, upstream URL construction, header management, and gateway controls:

The :provider path value selects a configured provider, not a specific model in the request. AISIX removes the /passthrough/:provider prefix and joins the remaining path to the resolved upstream base URL. It preserves the query string and body without field-level transformation, injects the provider credential, and removes unsafe or explicitly stripped headers. Configured guardrails can inspect and block request and response bodies.

Send a Passthrough Request​

Send passthrough traffic by naming the upstream provider in the path. The following example calls the upstream OpenAI models endpoint through AISIX:

curl -sS -X GET "${AISIX_PROXY}/passthrough/openai/v1/models" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -o aisix-passthrough-models.json

AISIX authenticates the caller API key and selects an accessible model for the requested provider. It injects that model's provider key as upstream authentication, strips unsafe headers, and forwards the provider-native request without endpoint-specific normalization.

The response is the upstream provider response, not a normalized AISIX model list:

{
  "object": "list",
  "data": [
    {
      "id": "model-id-0",
      "object": "model",
      "created": 1686935002,
      "owned_by": "openai"
    }
  ]
}

Check that the response came back as an upstream list:

jq -r '.object' aisix-passthrough-models.json

The command should print:

list

Use Provider Model Identifiers​

Any model identifier in a passthrough request belongs to the provider-native API contract. For rate limiting, AISIX can match a top-level body model against a configured alias or upstream model name. This lookup does not rewrite the request or select its provider key.

For example, assume a direct AISIX model has the following names:

display_name: ali-happyhorse-1.1-t2v
model_name: happyhorse-1.1-t2v

Modeled AISIX routes resolve caller-facing display_name aliases for the capabilities they support. When you call this Alibaba video-synthesis API through passthrough, AISIX does not replace ali-happyhorse-1.1-t2v with the configured upstream name. Send the Alibaba model ID instead:

curl -sS -X POST \
  "${AISIX_PROXY}/passthrough/alibaba/api/v1/services/aigc/video-generation/video-synthesis" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -H "X-DashScope-Async: enable" \
  -d '{
    "model": "happyhorse-1.1-t2v",
    "input": {
      "prompt": "A miniature cardboard city at night"
    },
    "parameters": {
      "duration": 5,
      "ratio": "16:9",
      "resolution": "720P"
    }
  }'

If the request instead contains "model": "ali-happyhorse-1.1-t2v", AISIX forwards that value unchanged. The provider can reject it because it is not a provider model ID.

For supported video providers, prefer the modeled Video Generation endpoint, which accepts the AISIX model alias and resolves the upstream model and provider key. Use passthrough when you need provider-native fields or an endpoint that AISIX does not model.

The same rule applies when a provider identifies the target outside the body. For example, a provider can place the model in a URL path, use a deployment name, or bind the model to an endpoint. Passthrough preserves that provider-native selection mechanism rather than converting it to an AISIX model field.

Choose Passthrough Only When Needed​

Passthrough is suitable for provider-specific APIs that are not exposed as first-class gateway routes. Use it for exploratory integration work or temporary access while evaluating whether a native gateway endpoint is required.

Use a modeled proxy route instead when AISIX already supports the capability. Modeled routes resolve a caller-facing model alias and can apply route-specific behavior such as response normalization, token usage attribution, cache behavior, and endpoint-specific provider checks.

Content guardrails apply on both paths. A modeled route scans the resolved request and, where the route supports output checks, the response. Passthrough scans the raw body as text. See Guardrail Hook Point for route coverage.

Passthrough Behavior​

Passthrough preserves the provider-native request schema and forwarded upstream responses, subject to the gateway-level behavior below:

  • It accepts any HTTP method and preserves the query string and request body without schema or model-name translation.
  • It forwards safe request headers, and strips hop-by-hop headers and the provider key's configured strip_headers.
  • It injects upstream authentication from the selected provider key.
  • It scans the full request and response body as text against any guardrails attached to the selected model. When a guardrail matches the request body, AISIX returns 422 before the request reaches the provider. When a guardrail matches the response body, AISIX returns 422 before relaying the upstream body.
  • It enforces rate limits before the request leaves the gateway. Caller API key limits always apply. When the JSON request body carries a top-level model field that names a configured model of the addressed provider, that model's limits apply as well. See Model Rate Limits on Passthrough.
  • It relays the upstream status, response body, and safe response headers.

Forwarded upstream responses keep the provider's native status and body. Failures generated by AISIX, including guardrail blocks and provider-resolution or transport failures, use the OpenAI-style proxy error envelope.

The provider path segment is used to find a configured model whose provider label matches that value and that the caller API key is allowed to access. This route is provider-scoped, not model-scoped. It does not choose a specific model alias the way a chat-completions request does. If multiple accessible models share the same provider but use different provider keys, avoid relying on implicit selection for production traffic. Credential selection ignores the request body; rate limiting does not — when the body names a configured model, that model's limits are enforced even if a different model of the same provider lends the credentials.

Upstream authentication uses the provider's expected shape. Anthropic passthrough uses an API key header plus the Anthropic version header; most other built-in provider defaults use bearer authentication.

If the selected provider key does not set a base URL, AISIX uses a built-in default only for known providers such as OpenAI, Anthropic, Google, and DeepSeek. For other provider labels, configure the provider key base URL explicitly. When the base URL and passthrough path both include the same API-version segment, AISIX removes the duplicate segment before forwarding the request.

Model Rate Limits on Passthrough​

Passthrough requests count against model rate limits when the JSON request body carries a top-level model field. AISIX matches that value against the configured models of the addressed provider — first as a model alias (display_name), then as the provider-native model name (model_name). The matched model's rate_limit and any model-scope rate-limit policies are checked before the request leaves the gateway, and they draw from the same counters as modeled routes, so passthrough and modeled traffic to one model share one quota.

This match affects rate limiting only. It does not rewrite the forwarded model value or change which accessible model lends the provider key.

When the body is not JSON, has no model field, or names a model that is not configured for the addressed provider, only caller-level limits apply.

This matters most for provider-native endpoints that have no modeled route. For Alibaba, Zhipu, Volcengine Ark, Runway, and OpenAI video generation, the modeled Video Generation endpoint is now the recommended path; the passthrough example below remains valid for tunneled traffic and for provider endpoints AISIX has not modeled. The following example caps an Alibaba Model Studio video model at one submission per minute. Alibaba has no built-in default base URL, so the provider key must set api_base explicitly; the model's rate_limit field defines the cap:

{
  "display_name": "wan-video-prod",
  "provider": "alibaba",
  "model_name": "wan2.7-t2v",
  "provider_key_id": "YOUR_PROVIDER_KEY_ID",
  "rate_limit": {
    "rpm": 1
  }
}

Submit two video-generation tasks through passthrough. The request body names the provider-native model, and the second submission inside the window is rejected before it reaches the provider:

for i in 1 2; do
  printf "request %s: " "${i}"
  curl -sS -o /dev/null -w "%{http_code}\n" -X POST \
    "${AISIX_PROXY}/passthrough/alibaba/api/v1/services/aigc/video-generation/video-synthesis" \
    -H "Authorization: Bearer ${AISIX_API_KEY}" \
    -H "X-DashScope-Async: enable" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "wan2.7-t2v",
      "input": {"prompt": "A miniature city built from cardboard comes alive at night."},
      "parameters": {"resolution": "720P", "duration": 5}
    }'
done

The output should show the second request rejected:

request 1: 200
request 2: 429

The rejected request carries a Retry-After HTTP response header and uses the same rate-limit error envelope as modeled routes in the response body:

{
  "error": {
    "message": "request limit exceeded (requests)",
    "type": "rate_limit_exceeded"
  }
}

Task status polling is excluded from the model quota. A polling request such as GET /passthrough/alibaba/api/v1/tasks/{task_id} has no request body and therefore does not consume the model's submission quota. A client that reaches the model cap after submitting a task can continue polling it to completion. Caller-level request limits still apply to polling.

Passthrough requests do not increment token counters (tpm, tpd). AISIX relays passthrough bodies verbatim and does not parse provider-reported token usage on this route. If modeled traffic to the same model has exhausted a token window, AISIX rejects passthrough requests until the window resets. Passthrough requests do not consume the token quota.

Use request-count fields (rps, rpm, rph, rpd) to cap passthrough traffic. concurrency also applies: a passthrough request holds an in-flight slot until AISIX receives the full upstream response.

Passthrough Checks​

When a passthrough request does not reach the expected upstream, first check the provider segment in the path. Then check which configured model and provider key AISIX can select for that provider label. Passthrough does not use the request body to select upstream credentials, so the selected provider key comes from an accessible model for that provider.

If the provider reports that a model does not exist, compare the forwarded model identifier with the provider's API reference. Do not send the AISIX display_name unless it is also the exact identifier accepted by the provider.

AISIX returns 403 when the caller API key is valid but cannot access any model for the requested provider. It returns 404 when no configured model matches the provider label. If AISIX reports that no default base URL is known, set the base URL on the selected provider key.

Next Steps​

You have now seen when to use provider passthrough instead of a modeled endpoint. Continue with Streaming and Tool Calling when your client depends on request behavior layered onto supported endpoint families.