Skip to main content

High Availability

Both APISIX Ingress Controller and API7 Ingress Controller support high availability through Kubernetes leader election. With multiple replicas, a standby can take over when the active leader becomes unavailable and resume synchronizing Kubernetes resource changes to the gateway.

Understand the Availability Model

Ingress Controller replicas operate in an active-standby model. Only the replica that holds the leader-election Lease reconciles resources and synchronizes configuration to the gateway. Additional replicas remain ready to acquire the Lease; they do not increase reconciliation throughput.

Gateway traffic does not pass through the Ingress Controller. If the leader becomes unavailable, the gateway can continue serving its last applied configuration, but new or changed Kubernetes resources are not synchronized until another replica becomes the leader.

This deployment protects the Ingress Controller reconciliation process. Configure the gateway, Kubernetes control plane, and API7 control plane or APISIX Admin API for their own availability requirements.

Configure Multiple Replicas

Deploy at least two controller replicas. Both controllers provide a dedicated Helm chart with the same replica value path:

Ingress ControllerDedicated Helm chartReplica value path
APISIX Ingress Controllerapisix/apisix-ingress-controllerdeployment.replicas
API7 Ingress Controllerapi7/api7-ingress-controllerdeployment.replicas

When installing either controller from its dedicated chart, add the following setting to the values file:

values.yaml
deployment:
replicas: 2

APISIX Ingress Controller can also be installed by the apisix/apisix chart. In this installation, nest the same deployment setting under ingress-controller:

values.yaml
ingress-controller:
deployment:
replicas: 2

The same paths can be passed as command-line overrides. Use --set deployment.replicas=2 with either dedicated controller chart. Use --set ingress-controller.deployment.replicas=2 when APISIX Ingress Controller is installed by the apisix/apisix chart.

Apply the values with helm upgrade, or update the source repository if a GitOps controller manages the release. See Helm Charts for installation and value-management guidance.

Spread Replicas Across Failure Domains

Multiple replicas can still be scheduled on the same node unless you define placement rules. Use pod anti-affinity so that a single node failure does not stop every controller replica.

The following example requires two replicas installed from either dedicated controller chart to run on different nodes:

values.yaml
deployment:
replicas: 2
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchLabels:
app.kubernetes.io/instance: <release-name>
app.kubernetes.io/name: <controller-app-name>
topologyKey: kubernetes.io/hostname

Replace <release-name> with the Helm release name. Set <controller-app-name> according to the installation:

  • APISIX Ingress Controller dedicated chart: apisix-ingress-controller
  • API7 Ingress Controller dedicated chart: api7-ingress-controller
  • APISIX Ingress Controller installed by the apisix/apisix chart: ingress-controller

If you override the chart's labels, use the labels from the rendered controller Deployment instead.

If APISIX Ingress Controller is installed by the apisix/apisix chart, nest the deployment block under ingress-controller.

Required pod anti-affinity keeps the replicas on separate nodes, but a replica remains Pending if the cluster does not have enough eligible nodes. Confirm that the cluster has sufficient capacity. In a multi-zone cluster, use topology spread constraints with the topology.kubernetes.io/zone key to distribute replicas across zones as well.

Set resource requests and limits so that the scheduler can place the controller reliably during load or node recovery. For APISIX Ingress Controller, deployment.resources applies to both the controller and ADC containers. For API7 Ingress Controller, configure deployment.resources for the controller container and adc.resources for the ADC container.

Verify Disruption Budget Rendering

Before enabling a chart's podDisruptionBudget option, run helm template with the intended values, confirm that rendering succeeds, and verify that the output contains the expected PodDisruptionBudget. Chart templates and supported values can change independently of the Ingress Controller, so the presence of an option in a values file does not guarantee that it renders successfully.

If the chart cannot render the PodDisruptionBudget, leave the option disabled. During voluntary maintenance, drain or upgrade nodes in stages and confirm that one controller replica remains Ready.

A PodDisruptionBudget only limits voluntary disruptions; it does not replace multiple replicas or failure-domain placement.

Configure Leader Election

Helm installations configure and enable leader election by default. Usually, keep the rendered settings.

All replicas in one logical controller deployment must use the same leader-election ID. If you intentionally run separate controller deployments in the same namespace, give each deployment a unique ID and configure it to reconcile a non-overlapping set of resources. Otherwise, the deployments can suppress or conflict with one another.

When you manage config.yaml directly, use snake_case field names. The following example uses valid timing values:

config.yaml
leader_election_id: apisix-ingress-gateway-leader
leader_election:
lease_duration: 30s
renew_deadline: 20s
retry_period: 2s
disable: false

Keep leader election enabled when more than one replica is deployed.

The effective values come from the config.yaml mounted in the running pod, which can differ from both the controller's built-in defaults and the Helm input values. After applying timing overrides through Helm, inspect the mounted file and confirm that it contains the intended snake_case fields and values. If the fields are absent or rendered under different names, the controller uses its built-in timing values. Follow View the Rendered Controller Configuration to inspect the effective file.

The effective lease_duration is the main bound on involuntary failover. After the last successful renewal, a ready standby normally takes over after the Lease expires, plus an acquisition retry interval and Kubernetes API latency. Pod scheduling time affects how quickly the Deployment restores its standby capacity; it does not delay takeover when another replica is already Ready.

Shorter values can reduce the failover interval, but they also increase Lease traffic and the risk of leadership changes during API-server or network latency. Test any tuning under representative failure conditions. The controller requires lease_duration to be greater than renew_deadline, and renew_deadline to be greater than retry_period multiplied by 1.2. Invalid combinations prevent the controller from starting.

Verify High Availability

First, confirm that the Deployment has the intended number of available replicas and that the pods run in different failure domains:

kubectl get deployment <deployment-name> -n <namespace>
kubectl get pods -n <namespace> -o wide

In a namespace with many workloads, add the label selector from the rendered controller Deployment to the second command.

Next, identify the controller Lease. Use the effective leader_election_id from the rendered config.yaml as <leader-election-id>. List the Lease resources rather than assuming a generated name:

kubectl get leases -n <namespace>

Inspect the controller Lease to record its current holder, renewal time, and transition count:

kubectl get lease <leader-election-id> -n <namespace> -o yaml

The relevant fields are spec.holderIdentity, spec.renewTime, and spec.leaseTransitions.

Test failover only in a non-production environment or an approved maintenance window. Watch the Lease in one terminal:

kubectl get lease <leader-election-id> -n <namespace> --watch

In another terminal, resolve the pod name from holderIdentity and delete the leader pod. The value contains the pod name followed by an underscore and a unique suffix:

LEADER_HOLDER=$(kubectl get lease <leader-election-id> -n <namespace> \
-o jsonpath='{.spec.holderIdentity}')
kubectl delete pod "${LEADER_HOLDER%%_*}" -n <namespace>

Confirm that the Lease holder changes, the replacement pod becomes Ready, and the Deployment returns to its desired replica count. Direct pod deletion is a failure simulation and is not restricted by a PodDisruptionBudget.

Finally, create or update a test route and send a request through the gateway. This confirms that the new leader resumed configuration synchronization instead of only acquiring the Lease. See Proxy Requests to a Service for a complete test route and request.

If the Lease changes but gateway configuration does not update, follow Troubleshoot Manifest Translation and Synchronization.

Monitor Availability

Monitor the following signals:

  • The controller Deployment has fewer available replicas than its desired count.
  • Controller pods remain Pending or restart repeatedly.
  • The leader-election Lease stops renewing.
  • Configuration synchronization reports errors.

Together, these signals help distinguish a scheduling or leader-election failure from a downstream gateway or control-plane problem.