# Autoscale Data Plane on Kubernetes Autoscaling automatically adjusts the number of gateway pods available to serve traffic, helping maintain consistent performance under changing load. This guide focuses on Kubernetes-based automatic scaling for the data plane. Use it after you already have a working deployment and want the cluster to add or remove pods based on observed load. The instructions below assume that API7 Gateway is already installed in a Kubernetes cluster. ## Deploy a Metrics Server[​](#deploy-a-metrics-server "Direct link to Deploy a Metrics Server") The Horizontal Pod Autoscaler (HPA) requires CPU and memory metrics from the cluster. The Kubernetes Metrics Server collects these metrics from kubelets and exposes them through the Kubernetes API. To deploy the Metrics Server, run: ``` kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml ``` Patch the Metrics Server deployment to allow insecure TLS connections to kubelets, which may be required in some cluster configurations: caution The `--kubelet-insecure-tls` flag disables TLS certificate verification between the Metrics Server and kubelets. This is **not recommended for production environments**. In production, configure proper kubelet serving certificates or use a kubelet CA bundle instead. ``` kubectl patch deployment metrics-server \ -n kube-system \ --type='json' \ -p='[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value":"--kubelet-insecure-tls"}]' ``` To verify that the Metrics Server is functioning correctly, you can inspect the CPU and memory usage of pods within the current namespace: ``` kubectl top pods ``` Assuming that the gateway pod is running in the current namespace, you should see its CPU and memory metrics: ``` NAME CPU(cores) MEMORY(bytes) api7-gateway-7998bf6dc6-2kh7q 16m 291Mi ``` This confirms that metrics are being collected and available for the HPA. ## Configure CPU Requests and Worker Processes[​](#configure-cpu-requests-and-worker-processes "Direct link to Configure CPU Requests and Worker Processes") When an HPA uses a CPU utilization target, Kubernetes calculates utilization relative to each pod's CPU request. Set a CPU request for the gateway pods so that the HPA has a stable baseline for scaling. For a standard workload with a fixed worker count, plan at least one requested CPU core for each worker process. A CPU limit is not required for HPA. If your cluster policy allows it, leave the CPU limit unset to avoid throttling gateway processes. If a CPU limit is required, set it no lower than the worker count and use benchmark results and runtime monitoring to determine the required headroom. See [CPU Resources and Worker Processes](https://docs.api7.ai/api7-gateway/install/deploy-on-kubernetes.md#cpu-resources-and-worker-processes) for detailed sizing guidance. To do so, export the current Helm values of your API7 Gateway release: ``` # update with your gateway name helm get values api7-gateway --all > values.yaml ``` Update the Helm values with the worker count and corresponding CPU request. The following example configures one fixed worker with one requested CPU core, intentionally leaves the CPU limit unset, and sets example memory requests and limits. Adjust the memory values based on your workload and monitoring data. values.yaml ``` nginx: workerProcesses: 1 # One fixed NGINX worker apisix: resources: requests: cpu: "1" # Requested CPU baseline for one worker (1 CPU core) memory: "256Mi" # Minimum memory guaranteed (256 MiB) limits: memory: "512Mi" # Maximum memory allowed (512 MiB) ``` If you configure memory-based scaling, also set a memory request so that Kubernetes can calculate memory utilization relative to the requested amount. Apply the updated configuration by upgrading the release: ``` # update with your gateway name helm upgrade api7-gateway api7/gateway --version "~3.10.0" -f values.yaml ``` ## Create an HPA[​](#create-an-hpa "Direct link to Create an HPA") Create a `HorizontalPodAutoscaler` for the gateway deployment that automatically scales the number of pods between 2 and 10. Scaling is based on average CPU usage, with a target of 50%, to accommodate varying traffic loads. Adjust the number of replicas and the CPU target as needed to match your workload and cluster resources. gateway-hpa.yaml ``` apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: gateway-hpa spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: api7-gateway # update with your gateway deployment name minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 50 ``` Apply the configuration to create the HPA: ``` kubectl apply -f gateway-hpa.yaml ``` View the current status of HPA in the cluster: ``` kubectl get hpa ``` You should see an output similar to the following: ``` NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE gateway-hpa Deployment/api7-gateway cpu: 7%/50% 2 10 2 30s ``` ## Verify Autoscaling with Load Test[​](#verify-autoscaling-with-load-test "Direct link to Verify Autoscaling with Load Test") To verify autoscaling, you will start 20 load pods to generate traffic against the gateway. Alternatively, you can also use load-testing tools such as k6 or wrk. ``` # update with your gateway service name for i in $(seq 1 20); do kubectl run load-$i --image=busybox:1.28 --restart=Never \ --labels=app=loadgen \ -- /bin/sh -c "while true; do wget -q -O- http://api7-gateway-gateway/ >/dev/null 2>&1; done" & done ``` In a separate terminal, watch the status of the HPA, including current metrics, replica count, and scaling activity in real time: ``` kubectl get hpa gateway-hpa -w ``` You should see the CPU utilization rising. When the load is sustained and high enough, HPA starts to scale up the gateway deployment: ``` NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE gateway-hpa Deployment/api7-gateway cpu: 59%/50% 2 10 2 4m30s gateway-hpa Deployment/api7-gateway cpu: 37%/50% 2 10 3 7m gateway-hpa Deployment/api7-gateway cpu: 60%/50% 2 10 3 9m16s gateway-hpa Deployment/api7-gateway cpu: 62%/50% 2 10 4 9m46s gateway-hpa Deployment/api7-gateway cpu: 52%/50% 2 10 5 11m ``` To observe the deployment scale down, delete all load-generating pods: ``` kubectl delete pod -l app=loadgen ``` You should see the CPU utilization decreasing and the gateway deployment scales down the number of replicas: ``` NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE gateway-hpa Deployment/api7-gateway cpu: 6%/50% 2 10 6 17m gateway-hpa Deployment/api7-gateway cpu: 8%/50% 2 10 6 17m gateway-hpa Deployment/api7-gateway cpu: 9%/50% 2 10 5 17m gateway-hpa Deployment/api7-gateway cpu: 7%/50% 2 10 4 18m gateway-hpa Deployment/api7-gateway cpu: 7%/50% 2 10 2 18m ``` ## Next Steps[​](#next-steps "Direct link to Next Steps") Kubernetes provides several autoscaling strategies beyond the CPU-based approach shown in this guide. Depending on your workload characteristics, API7 Gateway can be scaled based on memory utilization, custom metrics, or a combination of multiple metrics to achieve more accurate scaling decisions. Additionally, Kubernetes allows the configuration of scaling policies to control the rate of scale-up and scale-down events, which is important for maintaining stability in production environments. * [Scale Data Plane](https://docs.api7.ai/api7-gateway/configure-and-manage/scale-data-plane.md) * [Data Plane High Availability](https://docs.api7.ai/api7-gateway/configure-and-manage/high-availability-data-plane.md) * [Deploy for High Availability](https://docs.api7.ai/api7-gateway/install/deploy-high-availability.md) For a full overview of supported metrics, scaling behaviors, and advanced configuration options, please consult the official Kubernetes documentation: * [Horizontal Pod Autoscaling](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale) * [HorizontalPodAutoscaler Walkthrough](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough)