Proxy Errors and Retries

Handle AISIX proxy failures in a consistent order. Match the response format, decide whether retrying can help, then inspect the request route, gateway policy, routing state, or upstream state when the failure persists.

This page focuses on client behavior. For exact envelopes, headers, and status mappings, use Headers and Error Codes.

Match the Response Format

Start by matching the response to the proxy route your application called. This tells the client which error fields to read before it decides whether to fix the request, back off, or retry.

Route family	Error format	Client handling
OpenAI-compatible routes, including chat completions, embeddings, responses, audio, images, and rerank	OpenAI-style error envelope	Read the OpenAI-style error type and optional error code.
Anthropic-style routes, including Messages and Count Tokens	Anthropic-style error envelope	Read the Anthropic-style error type.
Provider passthrough routes	Forwarded upstream status and body; OpenAI-style envelope for gateway-generated failures	After AISIX authenticates and authorizes the caller and resolves a provider model, use provider-native handling for forwarded upstream responses and OpenAI-style handling for gateway-generated failures.

Keep proxy and admin error handling separate. The Admin API has its own error format and is meant for management clients, not application traffic.

Decide Whether Retrying Helps

After identifying the response format, use the error type as the main decision signal. The status code gives the broad class, while the error type usually identifies the next application action.

Failure class	Typical cause	Retry guidance
Caller or configuration error	The caller API key, model alias, access rule, request body, or endpoint choice is invalid.	Do not retry until the request or gateway configuration changes.
Policy rejection	A guardrail, rate limit, or budget check rejected the request.	Back off when the response includes a retry hint; otherwise change the request or policy.
Unsupported route	The resolved provider adapter does not implement the requested endpoint.	Choose a supported endpoint, provider, adapter, or model.
Upstream failure	The upstream provider returned an error that AISIX rendered through the caller-facing format.	Retry only when the failure is transient and the request is safe to repeat.
Runtime target state	The selected target, or every routing candidate, is temporarily unavailable.	Honor the retry hint when present and check provider health if the condition persists.

Authentication, authorization, model-not-found, invalid-request, and unsupported-route errors are non-retryable until the request or configuration changes. Rate limits, budget denials with a retry hint, transient upstream failures, and unavailable routing candidates can be retryable.

Use Retry Signals

AISIX can return Retry-After for gateway rate limits, budget denials, temporarily unavailable routing candidates, and upstream rate-limit responses that include a retry hint.

When the header is present, use it as the primary delay. If the client SDK also retries automatically, prefer the delay from AISIX or the upstream provider over a fixed local interval.

Streaming changes the retry decision. A request can fail over before the stream starts, but AISIX does not fail over after response bytes have reached the caller. Retry a streaming request only when the application can replay the prompt safely and handle the risk of duplicate output.

Separate Gateway Failover from Client Retries

For a multi-target model, AISIX can make additional attempts before returning an error to the application. This gateway behavior is configured independently from the application or SDK retry policy.

• retries repeats an eligible attempt against the same target, with exponential backoff between attempts. This applies on Messages, Responses, and Chat Completions, streaming and non-streaming alike.
• Streaming can fail over only before AISIX commits response bytes to the caller. Same-target retries and failover both happen while AISIX is still establishing the upstream stream; once the first bytes reach the caller, a failure ends the response.
• max_fallbacks limits how many other targets AISIX can try.
• retry_on_429 makes upstream rate-limit responses eligible for another attempt.
• fallback_on_statuses does the same for configured provider-specific transient statuses.

Count Tokens can fail over across eligible targets, but currently does not perform same-target retries.

The application sees an error only after gateway handling finishes or when the failure is not eligible for gateway retry. Avoid configuring aggressive retries at both layers without considering the combined number of upstream attempts. For gateway configuration, see Routing and Failover.

Understand Upstream Failures

Some failures come from AISIX before an upstream call. Others come from the selected provider and are rendered through the caller-facing proxy format.

For upstream 4xx failures, AISIX keeps the response in the caller-facing format. On OpenAI-style routes, cross-provider failures use a stable upstream-error type and can include a more specific code for rate limits, permissions, or a missing model. Native OpenAI failures can retain more OpenAI-style error fields.

For upstream 5xx failures, AISIX generally returns 502 and does not expose the upstream response body. Provider server errors can contain account, infrastructure, or internal diagnostic details, so use gateway logs for deeper investigation.

Check Common Causes

Use these checks to decide whether the failure starts in the caller request, gateway configuration, policy, routing state, or upstream provider.

Symptom	Check
Authentication failure	Confirm the application sends the plaintext caller API key, not the stored key hash or upstream provider key.
Permission failure	Confirm the caller API key allows the requested model alias.
Model not found	Confirm the application sends the caller-facing model alias, not the upstream provider model ID.
Rate-limit or budget rejection	Check caller-key limits, model limits, matching rate-limit policies, and managed budget state.
Unsupported endpoint	Confirm the selected provider and adapter support the route the application called.
502 response on one model alias	Check the upstream provider path, provider credential, base URL, and provider availability for that alias.
All routing candidates unavailable	Check target model runtime status and provider health for the multi-target model.

Next Steps

Use Headers and Error Codes for exact response fields and Provider Compatibility when a failure can depend on endpoint or adapter support.