Why AI Gateway
An AI gateway treats AI traffic as a first-class workload. In AISIX, models, provider keys, caller API keys, MCP servers, A2A agents, rate-limit policies, cache policies, and guardrails are first-class resources rather than features attached to a general-purpose route. The gateway can apply AI-specific policy and telemetry to token usage, prompts, and streaming responses.
AISIX AI Gateway is purpose-built around this operating model. Applications can call stable model aliases through supported client-facing endpoints, while platform teams manage provider access, routing, traffic controls, guardrails, and telemetry in one AI-specific layer.
Why AI Traffic Needs First-Class Support
AI traffic still needs familiar gateway controls such as authentication, routing, rate limiting, and observability. It also introduces concerns that are specific to models and agents:
- One client-facing model may route to different providers or upstream model IDs over time.
- Usage and capacity depend on tokens, context size, streaming behavior, and provider limits, not only request counts.
- Prompts and model responses may require content inspection, guardrails, and controlled logging.
- Applications and agents may reach models, MCP tools, and A2A agents through the same AI platform boundary.
- Operators need AI-specific telemetry such as the selected model and provider, token usage, and model latency.
When every application handles these concerns independently, provider credentials, routing logic, limits, and telemetry become duplicated across teams. An AI gateway centralizes them without exposing provider-specific details to each application.
What AISIX Makes First Class
AISIX represents the main parts of an AI traffic path directly:
- Models and provider keys are separate resources. Applications call stable model aliases while operators control upstream providers, credentials, and routing.
- Caller API keys and access grants identify which applications may use each model, MCP server, or A2A agent.
- Model configuration and policy resources control routing and fallback, while rate-limit policies, cache policies, and guardrails apply AI-specific controls at the gateway boundary.
- AI-aware telemetry records model and provider context, token usage, latency, and errors. AISIX Enterprise and AISIX Cloud add centralized usage views and budgets.
- MCP and A2A traffic use protocol-aware endpoints, access control, and telemetry rather than being treated as arbitrary HTTP payloads.
This resource model gives an AI platform team one place to change providers, rotate credentials, apply policy, and observe usage without requiring every application team to rebuild those controls.
Choose Based on the Operating Model
APISIX and API7 Gateway already provide AI capabilities such as provider proxying, routing, rate limits, guardrails, and observability. Choose based on how AI traffic should be represented and operated, not a feature checklist:
| Option | Choose it when | How AI is managed |
|---|---|---|
| Apache APISIX | AI calls are part of the APIs already served by an open-source APISIX deployment. | AI plugins attach provider access, model selection, prompt controls, and other behavior to general-purpose gateway resources, typically Routes and Services. |
| API7 Gateway | Traditional APIs and AI-enabled routes should remain in one commercially managed API gateway estate. | AI plugins attach to gateway resources and are managed through API7 Dashboard, the API7 Gateway Admin API, or ADC. |
| AISIX AI Gateway | Models, agents, tools, credentials, policy, and usage need a dedicated owner and lifecycle. | AISIX uses typed models, provider keys, caller API keys, MCP servers, A2A agents, rate-limit policies, cache policies, and guardrails, and emits AI usage telemetry. |
Use APISIX or API7 Gateway when AI is one capability of an existing API platform and the same team should operate both workloads. Choose AISIX when AI traffic is becoming a platform of its own, with dedicated ownership, provider abstraction, token-aware controls, agent and tool access, or an independent release and scaling lifecycle.
Adopting AISIX does not require replacing APISIX or API7 Gateway. They can continue to govern general API traffic while AISIX provides the dedicated path for AI applications and agents.
AISIX does not replace the broader API gateway scope of APISIX or API7 Gateway, including non-AI protocols, service discovery, transformations, and custom plugins.
Choose an AISIX Product
AISIX OSS runs without a control plane. AISIX Enterprise adds a customer-hosted control plane and dashboard, while AISIX Cloud provides an API7-hosted control plane and dashboard. In all three products, the AISIX gateway runs in your infrastructure and handles live AI traffic directly.
Continue with Choose an AISIX AI Gateway Product to compare their management and operational responsibilities.
If AI traffic should remain in your existing gateway, start with Proxy OpenAI Requests for Apache APISIX or Manage and Secure AI Traffic for API7 Gateway.