Load Balancing

Load balancing is a technique used to distribute network request loads. It is a key consideration in designing systems that need to handle a large volume of traffic, allowing for improved system performance, scalability, and reliability.

Apache APISIX supports a number of load balancing algorithms, one of which is the weighted round-robin algorithm. This algorithm distributes incoming requests over a set of servers in a cyclical pattern.

In this tutorial, you will create a route with two upstream services and uses the round-robin load balancing algorithm to load balance requests.

Prerequisite(s)

1. Complete Get APISIX to install APISIX in Docker or on Kubernetes.
2. Understand APISIX routes and upstreams.
3. Install ADC or APISIX-MCP if you are using these tools.

Enable Load Balancing

Admin API

Create a route with two upstream services, httpbin.org and mock.api7.ai, to distribute requests across. Both services respond with the request headers when receiving request at /headers:

curl -i "http://127.0.0.1:9180/apisix/admin/routes" -X PUT -d '
{
  "id": "getting-started-headers",
  "uri": "/headers",
  "upstream" : {
    "type": "roundrobin",
    "nodes": {
      "httpbin.org:443": 1,
      "mock.api7.ai:443": 1
    },
    "pass_host": "node",
    "scheme": "https"
  }
}'

❶ type: use roundrobin as the load balancing algorithm.

❷ nodes: upstream services.

❸ pass_host: use node to pass the host header to the upstream.

❹ scheme: use https to enable TLS with upstream.

You should receive an HTTP/1.1 201 Created response if the route was created successfully.

ADC

Create an ADC configuration file containing a route pointing to two upstream services, httpbin.org and mock.api7.ai. Both services will with the request headers when receiving request at /headers.

adc.yaml

services:
  - name: httpbin Service
    routes:
      - uris:
          - /headers
        name: getting-started-headers
    upstream:
      type: roundrobin
      nodes:
        - host: mock.api7.ai
          port: 443
          weight: 1
        - host: httpbin.org
          port: 443
          weight: 1
      pass_host: node
      scheme: https

❶ type: use roundrobin as the load balancing algorithm.

❷ nodes: upstream services.

❸ pass_host: use node to pass the host header to the upstream.

❹ scheme: use https to enable TLS with upstream.

Synchronize the configuration to APISIX:

adc sync -f adc.yaml

You will receive a similar response if the configuration was synchronized successfully:

[11:25:49 AM] [ADC] › ✔  success   Sync configuration
[11:25:49 AM] [ADC] › ★  star      All is well, see you next time!

Ingress Controller

For demonstration purpose, you will be creating a route to the publicly hosted httpbin services and mock.api7.ai. If you would like to proxy requests to services on Kubernetes, please modify accordingly.

important

If you are using Gateway API, you should first configure the GatewayClass and Gateway resources:

Show configuration

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: apisix
spec:
  controllerName: apisix.apache.org/apisix-ingress-controller
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: apisix
spec:
  gatewayClassName: apisix
  listeners:
  - name: http
    protocol: HTTP
    port: 80
  infrastructure:
    parametersRef:
      group: apisix.apache.org
      kind: GatewayProxy
      name: apisix-config

If you are using Ingress or APISIX custom resources, you can proceed without additional configuration.

Gateway API

APISIX Ingress controller installed with the current helm chart version (apisix-2.11.2) has a bug in load balancing, which is actively being fixed.

APISIX CRD

Create a Kubernetes manifest file for a route that proxy requests to two upstream services for load balancing:

lb-route.yaml

apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
  name: httpbin-external-domain
spec:
  scheme: https
  passHost: node
  externalNodes:
  - type: Domain
    name: httpbin.org
    weight: 1
    port: 443
---
apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
  name: mockapi7-external-domain
spec:
  scheme: https
  passHost: node
  externalNodes:
  - type: Domain
    name: mock.api7.ai
    weight: 1
    port: 443
---
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: lb-route
spec:
  ingressClassName: apisix
  http:
    - name: lb-route
      match:
        paths:
          - /headers
      upstreams:
      - name: httpbin-external-domain
      - name: mockapi7-external-domain

Apply the configuration to your cluster:

kubectl apply -f lb-route.yaml

APISIX-MCP

Enter the following prompt in your AI client:

Create a route with the ID getting-started-headers that matches requests to the URI /headers, and forward them to two upstream nodes — httpbin.org and mock.api7.ai — both on 443. Use roundrobin algorithm for load balancing. The route should use HTTPS scheme and passes the original host of each upstream node in the request.

You should see a response similar to the following:

Successfully created route "getting-started-headers" with:

URI: /headers
Two upstream nodes:
https://httpbin.org:443
https://mock.api7.ai:443
Round-robin load balancing
Original host headers preserved (pass_host: node)
Route ID: getting-started-headers
Status: Active (1)
Created at: 1744189246 (2025-04-09 09:00:46 UTC)
The route is now ready to distribute requests between the two upstream endpoints while maintaining the original host headers.

Verify

Admin API

Generate 50 consecutive requests to APISIX /headers route to see the load-balancing effect:

resp=$(seq 50 | xargs -I{} curl "http://127.0.0.1:9080/headers" -sL) && \
  count_httpbin=$(echo "$resp" | grep "httpbin.org" | wc -l) && \
  count_mockapi7=$(echo "$resp" | grep "mock.api7.ai" | wc -l) && \
  echo httpbin.org: $count_httpbin, mock.api7.ai: $count_mockapi7

The command keeps count of the number of requests that was handled by the two services respectively. The output shows that requests were distributed over to the two services:

httpbin.org: 23, mock.api7.ai: 27

ADC

Generate 50 consecutive requests to APISIX /headers route to see the load-balancing effect:

resp=$(seq 50 | xargs -I{} curl "http://127.0.0.1:9080/headers" -sL) && \
  count_httpbin=$(echo "$resp" | grep "httpbin.org" | wc -l) && \
  count_mockapi7=$(echo "$resp" | grep "mock.api7.ai" | wc -l) && \
  echo httpbin.org: $count_httpbin, mock.api7.ai: $count_mockapi7

The command keeps count of the number of requests that was handled by the two services respectively. The output shows that requests were distributed over to the two services:

httpbin.org: 23, mock.api7.ai: 27

Ingress Controller

First, expose the service port to your local machine by port forwarding:

kubectl port-forward svc/apisix-gateway 9080:80 &

The command above runs in the background and maps port 80 of the apisix-gateway service to port 9080 on the local machine.

Alternatively, you can use a load balancer to expose the service on the kind cluster. See load balancer kind documentation for more details.

Generate 50 consecutive requests to the route to see the load-balancing effect:

resp=$(seq 50 | xargs -I{} curl "http://127.0.0.1:9080/headers" -sL) && \
  count_httpbin=$(echo "$resp" | grep "httpbin.org" | wc -l) && \
  count_mockapi7=$(echo "$resp" | grep "mock.api7.ai" | wc -l) && \
  echo httpbin.org: $count_httpbin, mock.api7.ai: $count_mockapi7

The command keeps count of the number of requests that was handled by the two services respectively. The output shows that requests were distributed over to the two services:

httpbin.org: 23, mock.api7.ai: 27

APISIX-MCP

Enter the following prompt in your AI client:

Generate 50 consecutive requests to the APISIX /headers route to observe the load-balancing behavior, then count how many responses came from httpbin.org and how many came from mock.api7.ai.

You should see a response similar to the following:

Load balancing test results for 50 requests to /headers route:

httpbin.org responses: 23 (46%)
mock.api7.ai responses: 27 (54%)
The round-robin load balancing is working as expected, distributing requests between the two upstream nodes. The slight imbalance (46/54 split) could be due to connection timing or other network factors, but overall demonstrates effective load distribution.

The distribution of requests across services should be close to 1:1 but might not always result in a perfect 1:1 ratio. The slight deviation is due to APISIX operates with multiple workers.

What's Next

You have learned how to configure load balancing. In the next tutorial, you will learn how to configure key authentication.