Skip to main content
API7.ai Logo

Provider Passthrough

Provider passthrough lets an application call a provider-native endpoint through AISIX when the endpoint is not modeled as a first-class gateway route.

AISIX exposes ANY /passthrough/:provider/*rest for these requests. Passthrough keeps gateway authentication and upstream credential injection, but it performs less request and response handling than modeled proxy routes.

In this guide, you will send a provider passthrough request and review when this fallback path is appropriate.

Prerequisites​

Before starting, prepare the following:

  • A running AISIX gateway that can serve proxy requests.
  • A caller API key allowed to access at least one model for the requested provider.
  • A provider key with an api_base that can reach the provider-native endpoint, unless the provider has a built-in default base URL.

Send a Passthrough Request​

Send passthrough traffic by naming the upstream provider in the path. The following example calls the upstream OpenAI models endpoint through AISIX:

curl -sS -X GET "http://127.0.0.1:3000/passthrough/openai/v1/models" \
  -H "Authorization: Bearer YOUR_CALLER_API_KEY" \
  -o aisix-passthrough-models.json

AISIX authenticates the caller API key and selects an accessible model for the requested provider. It injects that model's provider key as upstream authentication, strips unsafe headers, preserves the query string, and forwards the remaining path and body without endpoint-specific normalization.

The response is the upstream provider response, not a normalized AISIX model list:

{
  "object": "list",
  "data": [
    {
      "id": "gpt-4o-mini",
      "object": "model"
    }
  ]
}

Check that the response came back as an upstream list:

jq -r '.object' aisix-passthrough-models.json

The command should print:

list

Choose Passthrough Only When Needed​

Passthrough is suitable for provider-specific APIs that are not exposed as first-class gateway routes. Use it for exploratory integration work or temporary access while evaluating whether a native gateway endpoint is required.

Use a modeled proxy route instead when AISIX already supports the capability. Modeled routes resolve a caller-facing model alias and can apply route-specific behavior such as response normalization, token usage attribution, cache behavior, and endpoint-specific provider checks.

Content guardrails apply on both paths. A modeled route scans the resolved request and, where the route supports output checks, the response. Passthrough scans the raw body as text. See Guardrail Hook Point for route coverage.

Passthrough Behavior​

Passthrough keeps the provider-native request mostly intact:

  • It accepts any HTTP method and preserves the query string and request body.
  • It forwards safe request headers, and strips hop-by-hop headers and the provider key's configured strip_headers.
  • It injects upstream authentication from the selected provider key.
  • It scans the full request and response body as text against any guardrails attached to the selected model. When a guardrail matches the request body, AISIX returns 422 before the request reaches the provider. When a guardrail matches the response body, AISIX returns 422 before relaying the upstream body.
  • It enforces rate limits before the request leaves the gateway. Caller API key limits always apply. When the JSON request body carries a top-level model field that names a configured model of the addressed provider, that model's limits apply as well. See Model Rate Limits on Passthrough.
  • It relays the upstream status, response body, and safe response headers.

Forwarded upstream responses keep the provider's native status and body. Failures generated by AISIX, including guardrail blocks and provider-resolution or transport failures, use the OpenAI-style proxy error envelope.

The provider path segment is used to find a configured model whose provider label matches that value and that the caller API key is allowed to access. This route is provider-scoped, not model-scoped. It does not choose a specific model alias the way a chat-completions request does. If multiple accessible models share the same provider but use different provider keys, avoid relying on implicit selection for production traffic. Credential selection ignores the request body; rate limiting does not — when the body names a configured model, that model's limits are enforced even if a different model of the same provider lends the credentials.

Upstream authentication uses the provider's expected shape. Anthropic passthrough uses an API key header plus the Anthropic version header; most other built-in provider defaults use bearer authentication.

If the selected provider key does not set a base URL, AISIX uses a built-in default only for known providers such as OpenAI, Anthropic, Google, and DeepSeek. For other provider labels, configure the provider key base URL explicitly. When the base URL and passthrough path both include the same API-version segment, AISIX removes the duplicate segment before forwarding the request.

Model Rate Limits on Passthrough​

Passthrough requests count against model rate limits when the JSON request body carries a top-level model field. AISIX matches that value against the configured models of the addressed provider — first as a model alias (display_name), then as the provider-native model name (model_name). The matched model's rate_limit and any model-scope rate-limit policies are checked before the request leaves the gateway, and they draw from the same counters as modeled routes, so passthrough and modeled traffic to one model share one quota.

When the body is not JSON, has no model field, or names a model that is not configured for the addressed provider, only caller-level limits apply.

This matters most for provider-native endpoints that have no modeled route. For Alibaba, Zhipu, Volcengine Ark, Runway, and OpenAI video generation, the modeled Video Generation endpoint is now the recommended path; the passthrough example below remains valid for tunneled traffic and for provider endpoints AISIX has not modeled. The following example caps an Alibaba Model Studio video model at one submission per minute. Alibaba has no built-in default base URL, so the provider key must set api_base explicitly; the model's rate_limit field defines the cap:

{
  "display_name": "wan-video-prod",
  "provider": "alibaba",
  "model_name": "wan2.7-t2v",
  "provider_key_id": "YOUR_PROVIDER_KEY_ID",
  "rate_limit": {
    "rpm": 1
  }
}

Submit two video-generation tasks through passthrough. The request body names the provider-native model, and the second submission inside the window is rejected before it reaches the provider:

for i in 1 2; do
  printf "request %s: " "${i}"
  curl -sS -o /dev/null -w "%{http_code}\n" -X POST \
    "http://127.0.0.1:3000/passthrough/alibaba/api/v1/services/aigc/video-generation/video-synthesis" \
    -H "Authorization: Bearer YOUR_CALLER_API_KEY" \
    -H "X-DashScope-Async: enable" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "wan2.7-t2v",
      "input": {"prompt": "A miniature city built from cardboard comes alive at night."},
      "parameters": {"resolution": "720P", "duration": 5}
    }'
done

The output should show the second request rejected:

request 1: 200
request 2: 429

The rejected request carries a Retry-After HTTP response header and uses the same rate-limit error envelope as modeled routes in the response body:

{
  "error": {
    "message": "request limit exceeded (requests)",
    "type": "rate_limit_exceeded"
  }
}

Task status polling stays outside the model quota. A polling request such as GET /passthrough/alibaba/api/v1/tasks/{task_id} carries no request body, so it never draws from the model's submission budget — a client that submits a task and then hits the cap can still poll that task to completion.

note

Passthrough traffic never adds to token counters (tpm, tpd): AISIX relays passthrough bodies verbatim and does not parse provider-reported token usage on this route. A token window already exhausted by modeled traffic to the same model still rejects passthrough requests until it resets, but passthrough requests themselves cannot fill it. Use request-count fields (rps, rpm, rph, rpd) to cap passthrough traffic; concurrency also applies — a passthrough request holds an in-flight slot until the full upstream response has been received.

Passthrough Checks​

When a passthrough request does not reach the expected upstream, first check the provider segment in the path. Then check which configured model and provider key AISIX can select for that provider label. Passthrough does not use the request body to select upstream credentials, so the selected provider key comes from an accessible model for that provider.

AISIX returns 403 when the caller API key is valid but cannot access any model for the requested provider. It returns 404 when no configured model matches the provider label. If AISIX reports that no default base URL is known, set the base URL on the selected provider key.

Next Steps​

You have now seen when to use provider passthrough instead of a modeled endpoint. Continue with Streaming and Tool Calling when your client depends on request behavior layered onto supported endpoint families.