Skip to main content
Version: 3.21.0

Autoscale APISIX Gateway (K8s)

Autoscaling is a mechanism that automatically adjusts the resources available to the gateway, ensuring consistent performance under varying traffic loads.

This document guides you through setting CPU and memory requests, creating a Horizontal Pod Autoscaler (HPA) for APISIX, and verifying scaling behavior with a simple load test, allowing your deployment to respond dynamically to workload changes.

The instructions below assume that APISIX is already installed in a Kubernetes cluster.

Provide Resource Metrics​

The Horizontal Pod Autoscaler (HPA) requires CPU and memory metrics from the resource metrics API. Metrics Server is one implementation that collects these metrics from kubelets and exposes them through the Kubernetes API. Check whether your cluster already provides the resource metrics API:

kubectl get --raw /apis/metrics.k8s.io/v1beta1/nodes

If the API is unavailable, use the resource metrics integration supported by your Kubernetes provider or install Metrics Server. Before installing Metrics Server, select a released version from its compatibility matrix that supports your Kubernetes version. Pin the selected release instead of using a moving latest URL:

export METRICS_SERVER_VERSION="vX.Y.Z"
kubectl apply -f "https://github.com/kubernetes-sigs/metrics-server/releases/download/${METRICS_SERVER_VERSION}/components.yaml"

Metrics Server should verify the kubelet serving certificates. Use kubelet certificates signed by the cluster CA, or mount the CA certificate in Metrics Server and configure --kubelet-certificate-authority with its path. See the Metrics Server security guidance.

Disposable test clusters only

If a disposable test cluster uses self-signed kubelet serving certificates, you can temporarily disable certificate verification:

kubectl patch deployment metrics-server \
-n kube-system \
--type='json' \
-p='[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value":"--kubelet-insecure-tls"}]'

This flag is for testing only. Do not use it in production.

To verify that the Metrics Server is functioning correctly, you can inspect the CPU and memory usage of pods within the current namespace:

kubectl top pods

Assuming that the APISIX pod is running in the current namespace, you should see its CPU and memory metrics:

NAME CPU(cores) MEMORY(bytes)
apisix-595698f9df-hmbwn 19m 296Mi

This confirms that metrics are being collected and available for the HPA.

Configure Resource Requests and Limits​

To ensure proper resource allocation and enable HPA to scale APISIX effectively, you should set CPU and memory requests and limits for the APISIX Gateway pods.

To do so, set the release name and namespace, then export all computed values in YAML format:

export APISIX_RELEASE="apisix"
export APISIX_NAMESPACE="<namespace>"
helm get values "${APISIX_RELEASE}" -n "${APISIX_NAMESPACE}" --all -o yaml > values.yaml

Update the resources section to include CPU and memory requests and limits. This ensures the Kubernetes scheduler can allocate sufficient resources for each pod and that HPA can monitor pod usage accurately.

The values below are example settings. In production, you should adjust CPU and memory requests and limits based on your expected traffic, workload characteristics, and cluster resources.

values.yaml
resources:
requests:
cpu: "200m" # Minimum CPU guaranteed (0.2 CPU cores)
memory: "256Mi" # Minimum memory guaranteed (256 MiB)
limits:
cpu: "1" # Maximum CPU allowed (1 CPU core)
memory: "512Mi" # Maximum memory allowed (512 MiB)

Find the installed chart version:

helm list -n "${APISIX_NAMESPACE}"

In the CHART column, the value includes the chart name and version, such as apisix-2.18.3. Set the installed version using only the version suffix, then apply the updated configuration. Pinning the chart version keeps this configuration change from also upgrading the chart or its default image.

export APISIX_CHART_VERSION="<installed-chart-version>"
helm upgrade "${APISIX_RELEASE}" apisix/apisix \
-n "${APISIX_NAMESPACE}" \
--version "${APISIX_CHART_VERSION}" \
-f values.yaml

Create an HPA​

Create a HorizontalPodAutoscaler for the APISIX deployment that automatically scales the number of pods between 2 and 10. Scaling is based on average CPU usage, with a target of 50%, to accommodate varying traffic loads. Adjust the number of replicas and the CPU target as needed to match your workload and cluster resources.

List the workloads and Services installed by the selected Helm release, then set their actual names. The defaults for a release named apisix are apisix and apisix-gateway, but name overrides change them:

kubectl get deployment,service \
-n "${APISIX_NAMESPACE}" \
-l "app.kubernetes.io/instance=${APISIX_RELEASE}"

export APISIX_DEPLOYMENT="<apisix-deployment-name>"
export APISIX_GATEWAY_SERVICE="<apisix-gateway-service-name>"
export APISIX_HPA="${APISIX_RELEASE}-hpa"

Create the HPA in the same namespace and target the selected deployment:

kubectl apply -f - <<EOF
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: ${APISIX_HPA}
namespace: ${APISIX_NAMESPACE}
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: ${APISIX_DEPLOYMENT}
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 50
EOF

View the current status of HPA in the cluster:

kubectl get hpa "${APISIX_HPA}" -n "${APISIX_NAMESPACE}"

You should see an output similar to the following:

NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
apisix-hpa Deployment/apisix cpu: 6%/50% 2 10 2 10m

Verify Autoscaling with Load Test​

To verify autoscaling, you will start 20 load pods to generate traffic against the gateway. Alternatively, you can also use load-testing tools such as k6 or wrk.

for i in $(seq 1 20); do
kubectl run "load-$i" -n "${APISIX_NAMESPACE}" \
--image=busybox:1.28 --restart=Never \
--labels=app=loadgen \
-- /bin/sh -c "while true; do wget -q -O- http://${APISIX_GATEWAY_SERVICE}/ >/dev/null 2>&1; done" &
done

In a separate terminal, watch the status of the HPA, including current metrics, replica count, and scaling activity in real time:

kubectl get hpa "${APISIX_HPA}" -n "${APISIX_NAMESPACE}" -w

You should see the CPU utilization rising. When the load is sustained and high enough, HPA starts to scale up the APISIX deployment:

NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
apisix-hpa Deployment/apisix cpu: 14%/50% 2 10 2 54m
apisix-hpa Deployment/apisix cpu: 37%/50% 2 10 2 55m
apisix-hpa Deployment/apisix cpu: 365%/50% 2 10 2 56m
apisix-hpa Deployment/apisix cpu: 73%/50% 2 10 4 56m
apisix-hpa Deployment/apisix cpu: 21%/50% 2 10 7 56m
apisix-hpa Deployment/apisix cpu: 39%/50% 2 10 7 57m

To observe the deployment scale down, delete all load-generating pods:

kubectl delete pod -n "${APISIX_NAMESPACE}" -l app=loadgen

You should see the CPU utilization decreasing and APISIX deployment scales down the number of replicas:

NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
apisix-hpa Deployment/apisix cpu: 26%/50% 2 10 7 62m
apisix-hpa Deployment/apisix cpu: 17%/50% 2 10 5 63m
apisix-hpa Deployment/apisix cpu: 6%/50% 2 10 4 64m
apisix-hpa Deployment/apisix cpu: 6%/50% 2 10 3 64m
apisix-hpa Deployment/apisix cpu: 6%/50% 2 10 2 64m

Next Steps​

Kubernetes provides several autoscaling strategies beyond the CPU-based approach shown in this guide. Depending on your workload characteristics, APISIX can be scaled based on memory utilization, custom metrics, or a combination of multiple metrics to achieve more accurate scaling decisions. Additionally, Kubernetes allows the configuration of scaling policies to control the rate of scale-up and scale-down events, which is important for maintaining stability in production environments.

For a full overview of supported metrics, scaling behaviors, and advanced configuration options, please consult the official Kubernetes documentation: