High Availability
AISIX Cloud separates the live AI traffic path from control-plane management. Applications send requests through managed gateways in your runtime environment, while API7 operates the hosted AISIX managed control plane used to manage those gateways.
This separation allows the traffic and management layers to use independent availability strategies. A highly available deployment must account for the application traffic path, managed gateways, upstream services, and each gateway's connection to the managed control plane.
High-Availability Architecture
The diagram below illustrates an active-active reference pattern that keeps live traffic independent from the managed control-plane path. The AISIX managed control plane is not a network hop between applications and upstream services. Choose the number of deployments, gateway instances, and traffic distribution layers according to your availability requirements.
In this pattern, managed gateways run in two active deployments across independent failure domains. A global load balancer steers traffic between the deployments, and a deployment load balancer distributes traffic across multiple gateway instances. Both deployments receive configuration from the same AISIX Cloud environment. Each deployment uses its own gateway certificate, and instances that share a certificate report distinct runtime instances to the managed control plane.
AISIX Cloud exposes separate operator and gateway management endpoints. In this reference pattern, the Control Plane API, dashboard, and Data Plane Manager run as redundant service replicas. Shared state is provided by a replicated PostgreSQL deployment behind a stable endpoint. These elements describe an HA deployment pattern, not the exact topology of the hosted AISIX Cloud service, and the diagram does not prescribe a particular replication or failover implementation.
Managed Control-Plane Components
The hosted AISIX managed control plane separates user management, gateway management, and shared state.
| Component | Role in the architecture |
|---|---|
| AISIX managed control-plane endpoint | Provides the stable public entry point and routes requests to the Control Plane API. |
| Control Plane API | Handles Cloud Admin API operations and reverse-proxies browser requests to the dashboard. It manages organizations, environments, resources, certificates, usage, and budgets. |
| Dashboard | Provides the browser interface and authentication workflows behind the Control Plane API. |
| Gateway management endpoint | Accepts gateway-initiated mTLS connections without requiring inbound access to gateway hosts. |
| Data Plane Manager | Serves projected configuration and receives heartbeats, usage telemetry, and managed budget checks from gateways. |
| PostgreSQL HA cluster | Stores shared control-plane state. In this reference topology, replication and failover operate behind a stable service endpoint. |
The responsibility boundary depends on the operating model. API7 operates the managed control plane for hosted AISIX Cloud. For AISIX Cloud On-Premises, run redundant replicas of the Control Plane API, Data Plane Manager, and dashboard across failure domains, and connect them to an external HA PostgreSQL endpoint. The bundled PostgreSQL chart does not provide the replicated database shown in this reference pattern by default. See On-Premises Configuration.
Availability Across the Request Path
High availability spans the runtime environment, the managed control plane, and upstream services.
| Area | Availability requirement |
|---|---|
| Application traffic | Provide a stable gateway endpoint and health-check each deployment from the global traffic layer. The proxy listener's /readyz endpoint also treats stale control-plane configuration as unready, so use it for traffic eligibility only when management staleness should withdraw an otherwise running instance. |
| Managed gateways | Operate redundant instances across independent failure domains and persist each instance's state directory. |
| Managed control plane | API7 operates the hosted management services. |
| Upstream services | Configure model routes with retries and multiple targets when requests must survive a model or provider outage. Deploy each MCP server and A2A agent behind a resilient service endpoint because AISIX does not automatically select an alternate registered service. |
Failure Behavior
The live traffic path and management path fail independently.
| Failure | Expected behavior |
|---|---|
| One gateway instance fails | The load-balancing layer directs new requests to healthy instances in that deployment. |
| One gateway deployment fails | The global load balancer directs new requests to another healthy deployment. |
| Gateway-to-control-plane connectivity is interrupted | A running gateway can continue from the latest accepted configuration held in memory if traffic continues to reach it. A gateway that restarts during the outage needs its persisted snapshot to recover that configuration. New resource projections stop, and heartbeat reporting resumes after connectivity recovers. /readyz returns an unhealthy response after the configuration watch has been stale for five minutes, or after a cache-only restart until live configuration is applied, so the load-balancer health policy determines whether an otherwise running gateway remains eligible. Managed budget checks follow their configured failure behavior, and failed telemetry batches sent to the control plane can be dropped. |
| A model service fails | AISIX can retry or use another model target only when routing or failover has been configured. |
| An MCP server or A2A agent fails | Requests to that service fail until its configured endpoint recovers. AISIX does not automatically select another registered MCP server or A2A agent. |
| An external guardrail service fails | AISIX follows the configured input and output failure behavior. Fail-open policies allow unscanned traffic, while fail-closed or mandatory policies block it. |
Deployment Guidance
For the active-active pattern shown above, use the following practices for the customer-operated traffic layer:
- Run at least two active gateway deployments behind a global load balancer with active health checks.
- Place each deployment in a separate failure domain. Use a redundant deployment load balancer to distribute traffic across multiple gateway instances. Choose the traffic-eligibility probe according to the intended failure policy:
/readyzprevents traffic before live configuration is applied, including after a cache-only restart, and withdraws an instance after its configuration watch has been stale for five minutes. If the deployment must continue serving the last accepted configuration through a longer control-plane outage, do not use management staleness as the only reason to withdraw every instance. Configure probe intervals, failure and recovery thresholds, and connection draining to avoid traffic flapping and interrupted in-flight requests. - Issue a separate gateway certificate for each deployment so the deployments have independent credential lifecycles. Instances within a deployment can share that deployment's certificate bundle. Protect each private key through your deployment secret system.
- Give each instance its own persistent state directory so its certificate bundle, deployment identity, and latest accepted configuration survive restarts. Do not share a writable state directory between instances.
- Ensure every gateway instance can initiate mTLS connections to the AISIX managed control-plane endpoint.
- Use Redis for rate-limit counters and cache entries that must be shared across instances. In-memory counters and cache entries remain local to one gateway process. When Redis is an availability dependency, use Redis Cluster, Redis Sentinel, or a managed Redis service exposed through a compatible endpoint.
- Configure model routing and failover separately. Gateway redundancy does not make a single model or provider highly available. Deploy MCP servers and A2A agents behind resilient endpoints because AISIX does not fail over between registered services.
- Monitor
/readyz, gateway heartbeat freshness, applied configuration status, rejected resources, and exporter health. A successful live request confirms the traffic path, but it does not prove that management or telemetry paths are healthy.
Network and Security Boundaries
Keep the traffic and management paths distinct:
- Expose the gateway load balancer to applications over HTTPS.
- Preserve TLS from the gateway to upstream services according to each provider or service configuration.
- Permit gateway-initiated mTLS connections from every gateway instance to the AISIX managed control plane. Gateway hosts do not require inbound control-plane access.
- Restrict metrics and health endpoints to the monitoring network used by the load balancer and platform operators.
Live AI requests pass through managed gateways in your runtime environment; they do not pass through the AISIX managed control plane. Usage telemetry sent to the managed control plane excludes prompt and response bodies, but it includes request, caller, routing, usage, and error metadata. External observability exporters can include request and response content when content capture is enabled, so apply the same data-handling controls used for other logging and tracing systems.
Next Steps
Continue with Resource Projection for how control-plane resources reach each gateway instance.
For behavior during temporary control-plane connectivity loss, see Offline Resilience. For upstream continuity, see Routing and Failover.