Skip to main content
API7.ai Logo

Request Lifecycle

AISIX sits between applications and AI providers. Applications send requests to the proxy API with a caller credential and a model alias. AISIX uses that information to apply access control, resolve the upstream target, enforce AI traffic policy, and record what happened.

The lifecycle looks like this:

Caller Authentication​

Each proxy request presents either a caller API key or a JWT issued by a configured OIDC provider. AISIX looks up a plaintext key by its hash. For a JWT, AISIX verifies the issuer, signature, and required claims, then maps the external identity to its bound caller API key.

After authentication, both paths use the resolved caller API key as the authorization identity. The key controls which model aliases the caller can use and which traffic controls apply, so application teams do not need direct provider credentials. See Caller API Keys for plaintext gateway credentials and JWT Authentication for external identity-provider credentials.

Model Resolution​

The model value in the request is the caller-facing alias. AISIX resolves that alias to one of four dispatch shapes:

  • A direct model, which points to one upstream model through one provider credential. A direct model can also include embedding metadata when a semantic router uses it for similarity comparisons.
  • A routing model, which lets AISIX choose one target model by failover, weighted round-robin, consistent hashing, cost, latency, or load.
  • A semantic model, which selects a target model by comparing request text with configured route examples.
  • An ensemble model, which sends a chat request to panel models and uses a judge model to synthesize the response.

Only one dispatch shape can be configured on a model resource. For the complete resource relationships, see Models and Providers.

Request Controls​

AISIX can stop a request before it reaches a provider. Caller API keys and model aliases can carry rate limits, while AISIX Cloud deployments can enforce request budgets. Guardrails can inspect input, and response caching can return a stored chat completion before an upstream call.

Provider Dispatch​

After the request is allowed, AISIX dispatches it to the selected provider using the provider key and adapter configured by the operator. Applications keep their gateway-facing API shape while AISIX handles provider credentials, upstream model names, base URLs, and provider-specific request handling.

Response Handling​

Provider responses return through AISIX. Output guardrails can inspect generated text before the response reaches the caller. For successful requests, AISIX records usage and telemetry so operators can understand requested aliases, resolved models, provider attempts, token usage, latency, and errors.

Deployment Boundary​

For the open-source AISIX gateway, operators can declare gateway resources in a resources.yaml file. With AISIX Cloud, the control plane owns resource management and projects accepted configuration to the AISIX gateway. The proxy request lifecycle remains the same from the caller's perspective: applications call the proxy API, and AISIX applies the configured model access, routing, controls, and observability behavior.