Manage and Secure AI Traffic
API7 AI Gateway is the AI traffic management layer of API7 Gateway. It provides a unified control point between your applications and LLM providers, enabling platform engineers to centralize authentication, enforce guardrails, control costs, and gain observability across all AI traffic — without modifying application code.
This document provides an overview of AI Gateway capabilities, architecture, and supported providers.
Architecture
AI Gateway runs as a set of plugins on the API7 Data Plane, configured through the Control Plane. Requests flow through a processing pipeline before reaching the LLM provider:
The pipeline stages correspond to AI plugins running on the Data Plane. Each stage is optional — configure only what you need.
Key Capabilities
| Capability | Description | Plugin |
|---|---|---|
| LLM Proxying | Unified API to 8+ providers, including any OpenAI-compatible endpoint | ai-proxy |
| Multi-Model Routing | Load balancing, failover, health checks, and priority-based routing across models and providers | ai-proxy-multi |
| Protocol Conversion | Anthropic to OpenAI protocol translation with automatic response conversion, transparent to clients | ai-proxy |
| Prompt Engineering | Reusable templates and system prompt injection at the gateway layer | ai-prompt-decorator, ai-prompt-template |
| Content Guardrails | PCRE-based prompt filtering and external toxicity detection | ai-prompt-guard, ai-aws-content-moderation, ai-aliyun-content-moderation |
| PII Redaction | Mask sensitive data in prompts before they reach LLM providers | ai-request-rewrite |
| Token Rate Limiting | Budget control per consumer, route, or model instance based on token counts | ai-rate-limiting |
| RAG Integration | Retrieval-Augmented Generation with Azure AI Search and Azure OpenAI embeddings | ai-rag |
| AI-Driven Transformation | Intelligent request rewriting powered by an LLM | ai-request-rewrite |
| AI Observability | Built-in logging (summaries and payloads) and Prometheus metrics for token accounting | ai-proxy |
| MCP Gateway | Expose OpenAPI services as MCP tools for AI agents | openapi-to-mcp |
Supported LLM Providers
API7 AI Gateway supports the following LLM providers with dedicated driver implementations:
| Provider | Provider Key | Guide |
|---|---|---|
| OpenAI | openai | Route Traffic to OpenAI |
| Azure OpenAI | azure-openai | Integrate Azure OpenAI |
| Anthropic | anthropic | Connect to Anthropic Claude |
| DeepSeek | deepseek | Route Traffic to DeepSeek |
| Google Gemini | gemini | Integrate Google Gemini |
| Vertex AI | vertex-ai | Route Traffic to Vertex AI |
| OpenRouter | openrouter | Access LLMs via OpenRouter |
| OpenAI-Compatible | openai-compatible | Connect Any OpenAI-Compatible LLM |
Any LLM API that follows the OpenAI chat completion format can be proxied using the openai-compatible provider type.
AWS Bedrock support with native SigV4 signing is planned for a future release.
Next Steps
- Get Started — Proxy your first LLM request in 5 minutes.
- LLM Providers — Configure a specific provider.
- Use Cases and How-To Guides — Explore advanced capabilities like routing, guardrails, and cost control.