Run the Gateway on Kubernetes

Install AISIX gateways on Kubernetes with the api7/aisix Helm chart, then scale them with load. The chart runs the gateway as a data plane of an AISIX Cloud control plane. It takes the data-plane manager endpoint and the gateway certificate bundle; the control plane supplies models, caller API keys, and policies.

Install the control plane first. See On-Premises Installation, or use Hybrid Cloud.

Prerequisites

• A Kubernetes cluster, with kubectl and Helm 3 pointed at it
• An AISIX Cloud control plane reachable from the cluster
• A gateway certificate bundle for the environment this gateway serves

Install the Chart

In the console, open the target environment's Data planes view and issue a gateway certificate. The Kubernetes (Helm) tab generates the commands below with your endpoint and bundle already filled in. The private key is shown only once.

Store the bundle in a Secret so the private key stays out of your values file:

kubectl create namespace aisix

kubectl -n aisix create secret generic aisix-gateway-certificate \
  --from-file=cert.pem=./cert.pem \
  --from-file=key.pem=./key.pem \
  --from-file=ca.pem=./ca.pem

Install the chart against the data-plane manager endpoint from the same view:

helm repo add api7 https://charts.api7.ai
helm repo update

helm install aisix api7/aisix --namespace aisix \
  --set controlPlane.baseURL=https://dp-manager.example.com:7944 \
  --set controlPlane.certificate.existingSecret=aisix-gateway-certificate

The gateway appears in the environment's Data planes view once its first heartbeat lands. Each replica registers as its own instance, and they all share the one certificate.

To see every value the chart accepts:

helm show values api7/aisix

The chart source is published in the api7/api7-helm-chart repository.

Expose the Gateway

The proxy Service is a ClusterIP on port 80 by default. To publish it through a cloud load balancer:

service:
  type: LoadBalancer
  port: 80
  # Preserve the client source IP, which per-model IP allowlists match on.
  externalTrafficPolicy: Local

The gateway binds port 3000 inside the container. It can bind a privileged port directly — the image carries the CAP_NET_BIND_SERVICE file capability, so no root user is required:

containerPorts:
  proxy: 80

See Network and Security for the rest of the exposure model.

Share Rate-Limit Counters across Replicas

Rate-limit counters live in each gateway's own memory by default, so N replicas enforce N times every configured request, token, and concurrency limit. Before running more than one replica — including any replica an autoscaler adds — point every replica at one Redis:

rateLimit:
  backend: redis
  redis:
    url: redis://redis.default.svc:6379

Use rateLimit.redis.existingSecret instead when the connection URL carries a password.

Scale on CPU or Memory

autoscaling creates a HorizontalPodAutoscaler for the gateway Deployment:

autoscaling:
  enabled: true
  minReplicas: 2
  maxReplicas: 20
  targetCPUUtilizationPercentage: 70

The targets are percentages of the pod's resource requests, so the chart sets a CPU request by default. It deliberately sets no CPU limit: throttling adds tail latency to a proxy and suppresses the signal the autoscaler reads. Scaling on CPU requires metrics-server in the cluster.

When autoscaling is enabled, the Deployment omits spec.replicas so that a later helm upgrade cannot reset the replica count the autoscaler chose. The replicaCount value is ignored from then on.

Pass any behavior policy through unchanged, for example to make scale-down gentler than the Kubernetes default:

autoscaling:
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Pods
          value: 1
          periodSeconds: 60

Use autoscaling.extraMetrics for Pods, Object, or External metrics such as a series exposed through a Prometheus adapter.

Scale on Request Load with KEDA

CPU is a proxy for load. To scale on the gateway's own traffic instead, use KEDA and a Prometheus query over the gateway metrics:

metrics:
  serviceMonitor:
    enabled: true

keda:
  enabled: true
  minReplicas: 2
  maxReplicas: 20
  pollingInterval: 15
  cooldownPeriod: 300
  triggers:
    - type: prometheus
      metadata:
        serverAddress: http://prometheus.monitoring.svc:9090
        query: sum(rate(aisix_llm_requests_total[2m]))
        threshold: "100"

autoscaling and keda are mutually exclusive. Enabling both fails the Helm render rather than letting two controllers write spec.replicas.

What Happens during a Scaling Event

A replica the autoscaler adds does not receive traffic before it can serve it. The chart's readiness probe reports 503 until the gateway has applied configuration from the control plane, so Kubernetes keeps the new pod out of the Service endpoints until then. See Health Checks for the readiness contract.

A replica the autoscaler removes leaves the Service endpoints first, then drains. Two chart defaults protect in-flight requests while that happens:

Value	Default	Purpose
preStopSleepSeconds	5	Endpoint removal and SIGTERM are concurrent. The pause holds the container up while the removal reaches every node, so a terminating pod is not handed new connections.
terminationGracePeriodSeconds	120	The gateway drains without a deadline of its own, so this value caps it. A streaming response can run for minutes, which the Kubernetes default of 30 seconds would cut.

Raise terminationGracePeriodSeconds if your workloads stream for longer than two minutes.

To watch scaling decisions and the metrics behind them:

kubectl -n aisix get hpa aisix --watch
kubectl -n aisix describe hpa aisix

Survive Node Disruption

A PodDisruptionBudget keeps voluntary disruptions — node drains, cluster upgrades — from taking every gateway down at once. Spread constraints keep replicas out of a single failure domain:

podDisruptionBudget:
  enabled: true
  minAvailable: 50%

topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: ScheduleAnyway
    labelSelector:
      matchLabels:
        app.kubernetes.io/name: aisix

See High Availability for the wider deployment pattern.

Set Any Other Gateway Configuration

The control plane owns dynamic resources, and the chart owns the pod. For startup configuration the chart does not surface as a value, set the environment variable directly — every configuration field is reachable as AISIX_<SECTION>__<FIELD>:

extraEnvVars:
  - name: AISIX_OBSERVABILITY__LOG_LEVEL
    value: debug
  - name: AISIX_CACHE__BACKEND
    value: redis

See Environment Variables for the naming rules.

Next Steps

Continue with Production Deployment for the checks to run before widening traffic, and Performance and Sizing for the throughput each replica supports.