Load Balancing

Load balancing is a technique used to distribute network request loads. It is a key consideration in designing systems that need to handle a large volume of traffic, allowing for improved system performance, scalability, and reliability

Apache APISIX supports a number of load balancing algorithms, one of which is the weighted round-robin algorithm. This algorithm distributes incoming requests over a set of servers in a cyclical pattern.

In this tutorial, you will create a route with two upstream services and uses the round-robin load balancing algorithm to load balance requests.

Prerequisite(s)

1. Complete Get APISIX to install APISIX in Docker or on Kubernetes.
2. Understand APISIX routes and upstreams.

Start Sample Upstream Services

If you are running APISIX in Kubernetes, you will be deploying two services to your cluster in this section for load balancing. Otherwise, skip to the next section where you will be using the hosted upstream services as the load balancing target nodes.

Start and expose a sample httpbin service:

kubectl run httpbin --image kennethreitz/httpbin --port 80
kubectl expose pod httpbin --port 80

Similarly, start and expose a sample NGINX service:

kubectl run nginx --image nginx --port 80
kubectl expose pod nginx --port 80

To see if services are running, check for these resources in the namespace:

kubectl get all | grep -e httpbin -e nginx

You should see a similar output:

pod/httpbin                                      1/1     Running   0          8m
pod/nginx                                        1/1     Running   0          9m
service/httpbin                                    ClusterIP   10.96.189.45    <none>        80/TCP                       8m
service/nginx                                      ClusterIP   10.96.121.40    <none>        80/TCP                       9m

Enable Load Balancing

Admin API

Create a route with two upstream services, httpbin.org and mock.api7.ai, to distribute requests across. Both services respond with the request headers when receiving request at /headers:

curl -i "http://127.0.0.1:9180/apisix/admin/routes" -X PUT -d '
{
  "id": "getting-started-headers",
  "uri": "/headers",
  "upstream" : {
    "type": "roundrobin",
    "nodes": {
      "httpbin.org:443": 1,
      "mock.api7.ai:443": 1
    },
    "pass_host": "node",
    "scheme": "https"
  }
}'

❶ type: use roundrobin as the load balancing algorithm.

❷ nodes: upstream services.

❸ pass_host: use node to pass the host header to the upstream.

❹ scheme: use https to enable TLS with upstream.

You should receive an HTTP/1.1 201 Created response if the route was created successfully.

ADC

Create an ADC configuration file containing a route pointing to two upstream services, httpbin.org and mock.api7.ai. Both services will with the request headers when receiving request at /headers.

adc.yaml

services:
  - name: httpbin Service
    routes:
      - uris:
          - /headers
        name: getting-started-headers
    upstream:
      type: roundrobin
      nodes:
        - host: mock.api7.ai
          port: 443
          weight: 1
        - host: httpbin.org
          port: 443
          weight: 1
      pass_host: node
      scheme: https

❶ type: use roundrobin as the load balancing algorithm.

❷ nodes: upstream services.

❸ pass_host: use node to pass the host header to the upstream.

❹ scheme: use https to enable TLS with upstream.

Synchronize the configuration to APISIX:

adc sync -f adc.yaml

You will receive a similar response if the configuration was synchronized successfully:

[11:25:49 AM] [ADC] › ✔  success   Sync configuration
[11:25:49 AM] [ADC] › ★  star      All is well, see you next time!

Ingress Controller

Create a Kubernetes manifest file to configure a route pointing to two upstream services using the ApisixRoute custom resource:

lb-route.yaml

apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: lb-route
  namespace: ingress-apisix
spec:
  http:
    - name: lb-route
      match:
        paths:
          - /*
      backends:
        - serviceName: httpbin
          servicePort: 80
        - serviceName: nginx
          servicePort: 80

Apply the configuration to your cluster:

kubectl apply -f lb-route.yaml

You should see a response similar to the following:

apisixroute.apisix.apache.org/lb-route created

Verify

Admin API

Generate 50 consecutive requests to APISIX /headers route to see the load-balancing effect:

resp=$(seq 50 | xargs -I{} curl "http://127.0.0.1:9080/headers" -sL) && \
  count_httpbin=$(echo "$resp" | grep "httpbin.org" | wc -l) && \
  count_mockapi7=$(echo "$resp" | grep "mock.api7.ai" | wc -l) && \
  echo httpbin.org: $count_httpbin, mock.api7.ai: $count_mockapi7

The command keeps count of the number of requests that was handled by the two services respectively. The output shows that requests were distributed over to the two services:

httpbin.org: 23, mock.api7.ai: 27

The distribution of requests across services should be close to 1:1 but might not always result in a perfect 1:1 ratio. The slight deviation is due to APISIX operates with multiple workers.

ADC

Generate 50 consecutive requests to APISIX /headers route to see the load-balancing effect:

resp=$(seq 50 | xargs -I{} curl "http://127.0.0.1:9080/headers" -sL) && \
  count_httpbin=$(echo "$resp" | grep "httpbin.org" | wc -l) && \
  count_mockapi7=$(echo "$resp" | grep "mock.api7.ai" | wc -l) && \
  echo httpbin.org: $count_httpbin, mock.api7.ai: $count_mockapi7

The command keeps count of the number of requests that was handled by the two services respectively. The output shows that requests were distributed over to the two services:

httpbin.org: 23, mock.api7.ai: 27

The distribution of requests across services should be close to 1:1 but might not always result in a perfect 1:1 ratio. The slight deviation is due to APISIX operates with multiple workers.

Ingress Controller

First, expose the service port to your local machine by port forwarding:

kubectl port-forward svc/apisix-gateway 9080:80 &

The command above runs in the background and maps port 80 of the apisix-gateway service to port 9080 on the local machine.

Alternatively, you can use a load balancer to expose the service on the kind cluster. See load balancer kind documentation for more details.

Generate 50 consecutive requests to the route to see the load-balancing effect:

resp=$(seq 50 | xargs -I{} curl "http://127.0.0.1:9080/" -sL) && \
  count_httpbin=$(echo "$resp" | grep "kennethreitz/httpbin" | wc -l) && \
  count_nginx=$(echo "$resp" | grep "nginx web server" | wc -l) && \
  echo httpbin: $count_httpbin, nginx: $count_nginx

The command keeps count of the number of requests that was handled by the two services respectively. The output shows that requests were distributed over to the two services:

httpbin: 23, nginx: 27

The distribution of requests across services should be close to 1:1 but might not always result in a perfect 1:1 ratio. The slight deviation is due to APISIX operates with multiple workers.

What's Next

You have learned how to configure load balancing. In the next tutorial, you will learn how to configure key authentication.