Request Lifecycle

AISIX sits between applications and AI providers. Applications send requests to the proxy API with a caller API key and a model alias. AISIX uses that information to apply access control, resolve the upstream target, enforce AI traffic policy, and record what happened.

The lifecycle looks like this:

Caller Authentication

Each proxy request uses a caller API key. AISIX checks the key before it resolves the requested model. The key controls which model aliases the caller can use, so application teams do not need direct provider credentials.

Model Resolution

The model value in the request is the caller-facing alias. AISIX resolves that alias to one of three model shapes:

• A single-target model, which points to one upstream model through one provider credential.
• A multi-target model, which lets AISIX choose one target model by failover, round-robin, or weighted routing.
• An ensemble model, which sends a chat request to panel models and uses a judge model to synthesize the response.

Request Controls

AISIX can stop a request before it reaches a provider. Caller API keys and model aliases can carry rate limits, request budgets can be enforced in managed deployments, guardrails can inspect input, and response caching can return a stored chat completion before an upstream call.

Provider Dispatch

After the request is allowed, AISIX dispatches it to the selected provider using the provider key and adapter configured by the operator. Applications keep their gateway-facing API shape while AISIX handles provider credentials, upstream model names, base URLs, and provider-specific request handling.

Response Handling

Provider responses return through AISIX. Output guardrails can inspect generated text before the response reaches the caller. For successful requests, AISIX records usage and telemetry so operators can understand requested aliases, resolved models, provider attempts, token usage, latency, and errors.

Deployment Boundary

In self-hosted deployments, operators manage gateway resources through the local Admin API. In managed deployments, AISIX Cloud owns resource management and projects accepted configuration to the managed gateway. The proxy request lifecycle remains the same from the caller's perspective: applications call the proxy API, and AISIX applies the configured model access, routing, controls, and observability behavior.