AI traffic gateway

AISIX AI Gateway

Put one stable API contract in front of your AI providers

AISIX AI Gateway is an open source, Rust-native gateway for LLM and AI-agent traffic. It runs as a single static binary and lets applications call stable model aliases while the AI platform team controls provider credentials, routing, failover, budgets, rate limits, caching, guardrails, and observability.

Start the quickstartView supported endpoints

OpenAI-compatible requestlocalhost:3000

curl -sS \
http://127.0.0.1:3000/v1/chat/completions \
-H "Authorization: Bearer YOUR_CALLER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "model": "gpt-4o-prod",
  "messages": [
    {"role": "user", "content": "Hello"}
  ]
}'

How requests flow

Applications call AISIX first. AISIX authenticates the caller, resolves the model alias, applies policy, and forwards the request to the selected provider.

ApplicationsApps, agents, and servicesSend OpenAI-compatible requests with gateway-issued caller API keys.

->

AISIX gateway boundaryStable contract, controlled provider access

AISIX keeps client traffic on one API shape while resolving the provider-side target for each request.

Authenticate callerResolve model aliasApply limits, cache, and guardrailsSelect provider route

->

ProvidersOpenAI, Anthropic, Bedrock, Vertex, AzureReceive provider-authenticated requests from the gateway.

Provider responses return through AISIX using the same client-facing contract.

What changes when AI traffic goes through AISIX

Application teams keep calling a familiar API. AI platform teams move provider credentials, model aliases, routing, budgets, and policy into one operator-managed gateway.

AccessCaller API keys and model allowlistsAuthenticate applications and decide which model aliases each key can use.

ProvidersCentralized upstream credentialsKeep provider keys, base URLs, and adapter details out of application code.

RoutingStable aliases with failoverExpose one model name while AISIX selects target models behind it.

PolicyBudgets, limits, cache, guardrails, telemetryApply AI-specific controls and usage visibility before requests leave your gateway boundary.

Why AISIX if you already use APISIX or API7 Gateway?

APISIX and API7 Gateway can add AI behavior to regular gateway routes through AI plugins. That path fits teams whose main workload is API traffic and whose AI calls are part of a broader API gateway deployment.

AISIX is for teams whose main workload is AI traffic. It manages provider keys, model aliases, caller API keys, budgets, routing, policy, and AI usage telemetry as first-class resources instead of route-level plugin settings.

APISIX

Add AI features to existing API routes.Use AI plugins when your existing API gateway route is the natural place to call or transform AI services.

AISIX

Run AI traffic through a dedicated gateway domain.Use AISIX when applications should call stable model names while an AI platform team owns provider keys, budgets, routing, policy, and usage visibility.

Read next

From First Request to Production

Start with one local request, then choose the endpoint, model, routing, controls, and production guidance you need next.

Quickstart

Boot AISIX and send a proxied requestConfigure one provider key, one model alias, and one caller API key in the quickstart.

Endpoints

Choose the caller-facing API surfaceUse OpenAI-compatible, Anthropic-style, specialized endpoint, or passthrough routes for application traffic.

Model access

Expose stable model aliasesKeep application requests stable while operators manage provider credentials, upstream model names, and supported request formats.

Routing

Add resilience behind one aliasUse failover, multi-target models, and ensembles when the best upstream target should be decided by the gateway.

Traffic controls

Control AI request volume and policyStart with rate limits, then add budgets, caching, and guardrails as traffic moves toward production.

Production

Prepare the runtime for productionReview deployment, security, health, metrics, and troubleshooting guidance for production gateways.