Deploy AISIX Gateways on Kubernetes

Use the api7/aisix Helm chart to deploy, expose, and scale AISIX gateways in your Kubernetes cluster. The gateways serve live AI traffic in your environment and connect to an existing AISIX Cloud control plane for configuration.

The chart requires the data-plane manager endpoint and a gateway certificate bundle issued for the target environment. Once connected, the gateways receive models, caller API keys, and policies from the control plane.

Prerequisites

• A Kubernetes cluster with kubectl configured to access it and Helm 3 installed.
• Access to an AISIX Cloud environment where you can issue a gateway certificate.
• Network access from the cluster to the AISIX Cloud data-plane manager endpoint. If you use On-Premises, complete On-Premises Installation first.

Install the Chart

In the dashboard, open the target environment's Data planes view and issue a gateway certificate. The Kubernetes (Helm) tab provides the data-plane manager endpoint and certificate bundle used below. The example also pins the initial deployment to one replica. The private key is shown only once.

Store the bundle in a Secret so the private key stays out of your values file:

kubectl create namespace aisix

kubectl -n aisix create secret generic aisix-gateway-certificate \
  --from-file=cert.pem=./cert.pem \
  --from-file=key.pem=./key.pem \
  --from-file=ca.pem=./ca.pem

Install the chart against the data-plane manager endpoint from the same view. The Dev documentation overrides the chart's released gateway image with the dev image:

helm repo add api7 https://charts.api7.ai
helm repo update

helm install aisix api7/aisix --namespace aisix \
  --set image.repository=ghcr.io/api7/aisix \
  --set image.tag=dev \
  --set controlPlane.baseURL=https://dp-manager.example.com:7944 \
  --set controlPlane.certificate.existingSecret=aisix-gateway-certificate \
  --set replicaCount=1

The gateway appears in the environment's Data planes view once its first heartbeat lands. The chart defaults to two replicas, but this initial command starts one because rate-limit counters use per-replica memory until you configure shared Redis. The dashboard Helm snippet omits replicaCount, so installing that snippet unchanged starts two replicas. Add --set replicaCount=1 until Redis is configured, as shown above. Share Rate-Limit Counters across Replicas before increasing the replica count or enabling an autoscaler.

Each replica registers as its own instance, and they all share the one certificate.

To see every value the chart accepts:

helm show values api7/aisix

The chart source is published in the api7/api7-helm-chart repository.

Expose the Gateway

The chart creates a ClusterIP Service for the proxy by default. To publish it through a cloud load balancer:

service:
  type: LoadBalancer
  port: 80
  # Preserve the client source IP, which per-model IP allowlists match on.
  externalTrafficPolicy: Local

By default the chart gives the gateway one plain-HTTP proxy listener: it binds containerPorts.proxy, port 3000, inside the container, and the Service publishes it as service.port. A Service can expose port 80 or 443 without making the process bind a privileged container port.

Serve HTTPS and Plain HTTP Together

Set listeners to serve several proxy ports at once, each with its own TLS:

listeners:
  - name: https                 # Port name, shared by the container port and the Service port.
    containerPort: 3443
    servicePort: 443
    # nodePort: 30443           # Optional; only for non-ClusterIP Service types.
    tls:
      secretName: aisix-proxy-tls
  - name: http
    containerPort: 3000
    servicePort: 80

A non-empty listeners is the complete set of proxy listeners and replaces the single default one. Nothing binds containerPorts.proxy, and service.port and service.nodePort are not read, because each entry carries its own. There is still one proxy Service, and it publishes one port per entry, targeting that entry's container port by name.

Every listener serves the same routes, /livez and /readyz included, so the startup, readiness, and liveness probes target the first entry — over HTTPS when that entry terminates TLS, and the kubelet does not verify the certificate. The chart writes the set to the gateway's configuration file as proxy.listeners. It still writes proxy.addr, which the gateway then ignores and logs an informational line about; no action is needed for that line. This needs a gateway image that supports proxy.listeners.

For the gateway-side rules behind these values, including why a plaintext listener needs a trusted network interface, see Serve HTTPS and HTTP at the Same Time.

The gateway reads TLS material from files, so each TLS listener needs its own kubernetes.io/tls Secret, with the keys tls.crt and tls.key. The chart mounts it read-only at /etc/aisix/tls/<name>. Create it from a certificate and key you already hold:

kubectl -n aisix create secret tls aisix-proxy-tls \
  --cert=./tls.crt --key=./tls.key

Or have cert-manager issue and renew it into the same Secret:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: aisix-proxy-tls
  namespace: aisix
spec:
  secretName: aisix-proxy-tls
  dnsNames:
    - gateway.example.com
  issuerRef:
    name: letsencrypt
    kind: ClusterIssuer

A rotated certificate reaches the gateway as a changed file in that mount. Roll the pods to pick it up, using the Deployment name from the chart's post-install notes. A release name containing aisix is normally used unchanged; name overrides can change the result:

kubectl rollout restart deployment/<deployment-name> -n <namespace>

Bind a Privileged Container Port

Bind a port below 1024 inside the container only when the gateway must listen on that port directly. The published image runs as non-root UID 10001, and the gateway binary carries the effective CAP_NET_BIND_SERVICE file capability:

containerPorts:
  proxy: 80

When listeners is set, put the privileged port on the entry that should bind it, as its containerPort; containerPorts.proxy is not read then.

If you customize the rendered Pod to drop all capabilities, add NET_BIND_SERVICE back to the AISIX container:

spec:
  containers:
    - name: aisix
      securityContext:
        capabilities:
          drop: ["ALL"]
          add: ["NET_BIND_SERVICE"]

The Kubernetes Restricted Pod Security Standard allows this capability. Because the binary's file capability has the effective bit set, the container can fail with exec: Operation not permitted when the runtime prevents the capability from being granted.

Use hostNetwork or hostPort only when the gateway must bind directly on a node and the cluster policy allows it. These options introduce node-port conflicts and reduce network isolation; the Baseline and Restricted Pod Security Standards also disallow them.

See Network and Security for the full exposure and credential model.

Run on OpenShift

The chart installs on OpenShift under the default restricted-v2 security context constraint. No custom SCC and no service-account changes are needed.

The chart pins no UID. Its default pod security context is exactly runAsNonRoot: true plus seccompProfile.type: RuntimeDefault, so restricted-v2 supplies the runAsUser and fsGroup from the namespace's allocated range. The gateway image declares a numeric user and runs under any assigned UID with GID 0; the only path written at runtime is an emptyDir volume. The container security context already satisfies restricted-v2 — all capabilities dropped except NET_BIND_SERVICE, a read-only root filesystem, and no privilege escalation.

To pin a fixed UID instead, set the values back:

podSecurityContext:
  runAsNonRoot: true
  runAsUser: 10001
  runAsGroup: 10001
  fsGroup: 10001
  seccompProfile:
    type: RuntimeDefault

For the control plane on OpenShift, see Install on OpenShift.

Share Rate-Limit Counters across Replicas

Rate-limit counters live in each gateway's own memory by default, so N replicas enforce N times every configured request, token, and concurrency limit. Before running more than one replica — including any replica an autoscaler adds — point every replica at one Redis:

rateLimit:
  backend: redis
  redis:
    url: redis://redis.default.svc:6379

Use rateLimit.redis.existingSecret instead when the connection URL carries a password. To supply the Redis username and password separately from the URL, read them from a Secret through configSecrets, as described in Set Any Other Gateway Configuration. Other settings of the shared Redis connection, such as mode, nodes, or timeout_secs, go under config.ratelimit.redis.

After every replica points to the same Redis deployment, increase replicaCount or enable one of the autoscaling options below. If the deployment does not use request, token, or concurrency limits, you can deliberately accept the per-replica memory backend instead.

Scale on CPU or Memory

autoscaling creates a HorizontalPodAutoscaler for the gateway Deployment:

autoscaling:
  enabled: true
  minReplicas: 2
  maxReplicas: 20
  targetCPUUtilizationPercentage: 70

The targets are percentages of the pod's resource requests, so the chart sets a CPU request by default. It deliberately sets no CPU limit: throttling adds tail latency to a proxy and suppresses the signal the autoscaler reads. Scaling on CPU requires metrics-server in the cluster.

On Linux, proxy.workers defaults to the CPU parallelism available to the gateway process. A Kubernetes CPU request does not constrain that value, but a CPU limit does. Because the chart sets no CPU limit, set config.proxy.workers explicitly when each replica needs a stable worker count, and keep it within the CPU capacity planned for the replica. See Thread-per-Core Workers for worker configuration and sizing considerations.

When autoscaling is enabled, the Deployment omits spec.replicas so that a later helm upgrade cannot reset the replica count the autoscaler chose. The replicaCount value is ignored from then on.

Pass any behavior policy through unchanged, for example to make scale-down gentler than the Kubernetes default:

autoscaling:
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Pods
          value: 1
          periodSeconds: 60

Use autoscaling.extraMetrics for Pods, Object, or External metrics such as a series exposed through a Prometheus adapter.

Scale on Request Load with KEDA

CPU is a proxy for load. To scale on the gateway's own traffic instead, use KEDA and a Prometheus query over the gateway metrics.

Before enabling the example, prepare these cluster components:

• Install KEDA, including the ScaledObject CRD and controller.
• Provide a Prometheus server that can query the gateway metrics.
• If the chart should create the scrape configuration shown below, install Prometheus Operator or another controller that consumes ServiceMonitor objects. Installing only the CRD lets Helm create the object, but nothing turns it into scrape configuration. Set metrics.serviceMonitor.labels when the Prometheus instance selects ServiceMonitors by label. If Prometheus discovers the metrics Service another way, leave metrics.serviceMonitor.enabled set to false and omit that block.

Confirm the CRDs required by the values you enable:

kubectl get crd scaledobjects.keda.sh
# Required only when metrics.serviceMonitor.enabled is true.
kubectl get crd servicemonitors.monitoring.coreos.com

Configure the chart after those prerequisites are available:

metrics:
  serviceMonitor:
    enabled: true

keda:
  enabled: true
  minReplicas: 2
  maxReplicas: 20
  pollingInterval: 15
  cooldownPeriod: 300
  triggers:
    - type: prometheus
      metadata:
        serverAddress: http://prometheus.monitoring.svc:9090
        query: sum(rate(aisix_llm_requests_total[2m]))
        threshold: "100"

aisix_llm_requests_total counts model-inference requests, such as /v1/chat/completions. It does not include MCP or A2A calls. Use aisix_proxy_requests_total if that is the load you want to scale on.

A streamed request is counted when its stream ends, not when it starts. While a burst of long streams is in progress, a rate() over either counter therefore lags behind the incoming load by roughly the length of one stream; steady-state rates are unchanged. Account for that delay when you choose pollingInterval and the query window. A gateway at 1.5.0 or earlier counts a streamed request when its response starts.

autoscaling and keda are mutually exclusive. Enabling both fails the Helm render rather than letting two controllers write spec.replicas.

What Happens during a Scaling Event

A replica the autoscaler adds does not receive traffic before it can serve it. Until the gateway has applied configuration from the control plane it does not bind the proxy listener at all, so every probe the chart points at that listener is refused rather than answered.

The chart's startupProbe is what covers that wait. Kubernetes holds the readiness and liveness probes back until a startup probe succeeds, so during the wait the readiness probe does not run and the pod simply never becomes ready — Kubernetes keeps it out of the Service endpoints, and liveness does not restart a pod that is still reaching the control plane. The startup probe's budget is periodSeconds x failureThreshold, and it has to cover reaching the control plane and applying what it holds. A pod whose control-plane connection stays unreachable for longer than that budget has its container killed and restarted, and repeated restarts show as CrashLoopBackOff — the intended outcome for an instance that has never had anything to serve. See Startup and the First Configuration for the readiness contract and how to size the budget.

When a replica scales down, Kubernetes starts removing it from Service endpoints and terminating the pod concurrently. Two chart values protect in-flight requests while that happens.

preStopSleepSeconds pauses before the container receives SIGTERM, so endpoint removal reaches every node before the gateway begins shutting down. That covers a balancer watching the Kubernetes API; one that polls a health check instead sees a still-ready pod throughout the pause and is covered by the gateway's own drain window, shutdown.min_drain_secs.

terminationGracePeriodSeconds caps the pause, the drain window, and the in-flight drain that follows. When snapshot persistence is enabled, it also covers up to five more seconds for an outstanding cache write. The gateway drains in-flight requests without a deadline of its own. The Kubernetes default of 30 seconds would cut a streaming response, so the chart ships a longer one.

Both values ship with defaults sized for endpoint withdrawal and long-running requests; helm show values api7/aisix reports what they currently are. Raise terminationGracePeriodSeconds for workloads that stream beyond the remaining budget after the pause and drain window. When snapshot persistence is enabled, also leave five seconds for the cache-write drain.

To watch scaling decisions and the metrics behind them:

kubectl -n aisix get hpa aisix --watch
kubectl -n aisix describe hpa aisix

Survive Node Disruption

A PodDisruptionBudget keeps voluntary disruptions — node drains, cluster upgrades — from taking every gateway down at once. Spread constraints keep replicas out of a single failure domain:

podDisruptionBudget:
  enabled: true
  minAvailable: 50%

topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: ScheduleAnyway
    labelSelector:
      matchLabels:
        app.kubernetes.io/name: aisix

The chart uses an emptyDir volume for /var/lib/aisix. It survives a container restart within the same Pod but is lost when the Pod is replaced. The chart supplies the certificate bundle from a Secret at every start. A replacement Pod therefore reconstructs its identity from that bundle and downloads its configuration from the control plane before receiving traffic. Keep multiple replicas across failure domains so an existing replica can continue serving during a replacement.

To recover cached configuration after a Pod replacement while the control plane is unavailable, customize the workload in two ways. Give each replica its own persistent state directory, and set config.managed.snapshot_cache_enabled to true. Persistent storage alone does not enable the cache. See Restart from Cached Configuration for the full recovery and security requirements.

See High Availability for the wider deployment pattern.

Set Any Other Gateway Configuration

The control plane owns dynamic resources, and the chart owns the pod. The gateway reads its startup configuration from a file that the chart renders into a ConfigMap and mounts read-only. The config value mirrors that file key for key, with the same section and key names as the Startup Configuration Reference, and lists every setting with the gateway's default. Set only the keys you need to change:

config:
  observability:
    log_level: debug
  upstream:
    pool_max_idle_per_host: 32
  proxy:
    real_ip:
      trusted_proxies: ["10.0.0.0/8"]
      recursive: true

A key set to null is left out of the file, so the gateway uses its own default. A change to config rolls the pods on helm upgrade. Run helm show values api7/aisix to see every key.

The chart sets some keys from its other values and refuses them under config. The render fails with a message naming the value to use instead:

Configuration key	Chart value
proxy.addr, observability.metrics.prometheus.addr	containerPorts
proxy.listeners, proxy.tls	listeners
ratelimit.backend, ratelimit.redis.url	rateLimit
managed.enabled and the control-plane connection under managed	controlPlane
etcd.endpoints, etcd.prefix	Set from the control-plane connection.
etcd.env_id, etcd.user, etcd.password_env, etcd.tls	Not configurable through this chart.
resources_file	standalone
admin	admin and containerPorts.admin, in standalone mode only. See Enable the Admin API.

Settings that hold a credential never enter the ConfigMap. Store the value in a Secret and reference it under configSecrets, keyed by the configuration path. The chart hands it to the gateway as an environment variable, which overrides the configuration file:

config:
  cache:
    backend: redis
    redis:
      mode: single
configSecrets:
  cache.redis.url:
    secretName: aisix-cache-redis
    key: url
  cache.redis.password:
    secretName: aisix-cache-redis
    key: password

configSecrets accepts cache.redis.url, cache.redis.username, cache.redis.password, ratelimit.redis.username, and ratelimit.redis.password. The chart also refuses these keys under config. The rate-limit Redis URL comes from rateLimit.redis.

Use extraEnvVars only for system-level environment variables, such as TZ, and for variables that a standalone resources file references. An AISIX_* variable set there still overrides the configuration file. See Environment Variables for the naming rules.

Overriding a variable that the chart sets itself still works, and your extraEnvVars value still wins, but only as a compatibility path for existing deployments. For a name that chart 1.5.0 rendered, such as AISIX_MANAGED__CP_BASE_URL or AISIX_MANAGED__HEARTBEAT_INTERVAL_SECS, the chart keeps its own entry ahead of yours, so the container lists the name twice. Helm 4 server-side apply rejects that. On a new installation, set the value through its own chart value or under config instead, and move existing overrides there as well. See Move Application Settings into Chart Values, including why removing such an override takes two upgrades.