Configure mTLS Between APISIX and Upstream

Mutual TLS (mTLS) is a two-way TLS where the client and the server authenticate each other. It is typically implemented in high-security environments to prevent unauthorized access and harden security.

This guide will walk you through how to configure mTLS between APISIX and an upstream service, using NGINX as a sample upstream service.

Prerequisite(s)

• Install Docker.
• Install cURL to send requests to the service for testing.
• Follow the Getting Started tutorial to start a new APISIX instance in Docker.

Generate Certificates and Keys

Generate the Certificate Authority (CA) key and certificate:

openssl genrsa -out ca.key 2048
openssl req -new -x509 -days 36500 -sha256 \
  -key ca.key \
  -out ca.crt \
  -subj "/CN=MyTestCA" \
  -extensions v3_ca \
  -config <(printf "[req]\ndistinguished_name=req\n[ v3_ca ]\nbasicConstraints=critical,CA:TRUE\nkeyUsage=critical,keyCertSign,cRLSign\nsubjectKeyIdentifier=hash\nauthorityKeyIdentifier=keyid:always,issuer")

Generate the key and certificate signing request (CSR):

openssl genrsa -out server.key 2048
openssl req -new -sha256 \
  -key server.key \
  -out server.csr \
  -subj "/CN=test.com"

Sign the server CSR with the CA certificate to generate the server certificate:

openssl x509 -req -days 36500 -sha256 \
  -in server.csr \
  -CA ca.crt -CAkey ca.key -CAcreateserial \
  -out server.crt \
  -extensions v3_req \
  -extfile <(printf "[v3_req]\nbasicConstraints=CA:FALSE\nkeyUsage=digitalSignature,keyEncipherment\nextendedKeyUsage=serverAuth")

Generate the key and certificate signing request (CSR) for the client:

openssl genrsa -out client.key 2048
openssl req -new -sha256 \
  -key client.key \
  -out client.csr \
  -subj "/CN=CLIENT"

Sign the client CSR with the CA certificate to generate the client certificate:

openssl x509 -req -days 36500 -sha256 \
  -in client.csr \
  -CA ca.crt -CAkey ca.key -CAcreateserial \
  -out client.crt \
  -extensions v3_req \
  -extfile <(printf "[v3_req]\nbasicConstraints=CA:FALSE\nkeyUsage=digitalSignature,keyEncipherment\nextendedKeyUsage=clientAuth")

Configure Upstream Service

Docker

Start an NGINX server as a sample upstream service in the same Docker network as APISIX:

docker run -d \
  --name quickstart-nginx \
  --network=apisix-quickstart-net \
  -p 8443:8443 \
  nginx

Copy CA certificate, server certificate public and private keys into NGINX:

docker cp ca.crt quickstart-nginx:/var/ca.crt
docker cp server.crt quickstart-nginx:/var/server.crt
docker cp server.key quickstart-nginx:/var/server.key

Configure an HTTPS server listening on /hello and port 8443 in NGINX configuration file:

/etc/nginx/nginx.conf

http {
    # ...
    server {
        listen 8443 ssl;
        server_name            test.com;
        ssl_certificate        /var/server.crt;
        ssl_certificate_key    /var/server.key;
        ssl_client_certificate /var/ca.crt;
        ssl_verify_client on;
        location /hello {
            return 200 "Hello APISIX!";
        }
    }
}

❶ server_name: set to test.com to be consistent with the server certificate CN value.

❷ ssl_certificate: configure the path to the server certificate public key server.crt.

❸ ssl_certificate_key: configure the path to the server certificate private key server.key.

❹ ssl_client_certificate: configure the path to the CA certificate public key ca.crt.

❺ ssl_verify_client: set to on to verify the client certificate.

Reload the NGINX server to apply the configuration changes:

docker exec quickstart-nginx nginx -s reload

Kubernetes

Create a Kubernetes secret for the NGINX sample upstream service:

kubectl create secret generic nginx-certs \
  --from-file=server.crt=server.crt \
  --from-file=server.key=server.key \
  --from-file=ca.crt=ca.crt

Create a ConfigMap for the NGINX sample upstream service:

nginx-cm.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-config
data:
  default.conf: |
    server {
        listen 8443 ssl;
        server_name test.com;
        ssl_certificate        /etc/nginx/ssl/server.crt;
        ssl_certificate_key    /etc/nginx/ssl/server.key;
        ssl_client_certificate /etc/nginx/ssl/ca.crt;
        ssl_verify_client on;
        location /hello {
            return 200 "Hello APISIX!";
        }
    }

❶ server_name: set to test.com to be consistent with the server certificate CN value.

❷ ssl_certificate: configure the path to the server certificate public key server.crt.

❸ ssl_certificate_key: configure the path to the server certificate private key server.key.

❹ ssl_client_certificate: configure the path to the CA certificate public key ca.crt.

❺ ssl_verify_client: set to on to verify the client certificate.

Create a Kubernetes manifest file for the NGINX sample upstream service deployment and service:

nginx.yaml

apiVersion: v1
kind: Service
metadata:
  name: quickstart-nginx
spec:
  type: NodePort
  selector:
    app: quickstart-nginx
  ports:
    - name: https
      port: 8443
      targetPort: 8443
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: quickstart-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: quickstart-nginx
  template:
    metadata:
      labels:
        app: quickstart-nginx
    spec:
      containers:
        - name: nginx
          image: nginx
          ports:
            - containerPort: 8443
          volumeMounts:
            - name: config
              mountPath: /etc/nginx/conf.d
            - name: certs
              mountPath: /etc/nginx/ssl
      volumes:
        - name: config
          configMap:
            name: nginx-config
        - name: certs
          secret:
            secretName: nginx-certs

Apply the configuration to your cluster:

kubectl apply -f nginx-cm.yaml -f nginx.yaml

Port forward the NGINX service port to your local machine's port for verification:

kubectl port-forward service/quickstart-nginx 8443:8443 &

To verify that the NGINX instance is properly configured, send a request to the Nginx service's route with client certificate and key:

curl -ik "https://127.0.0.1:8443/hello" --cert client.crt --key client.key

You should receive an HTTP/1.1 200 OK response and see the following message:

Hello APISIX!

If you send a request to the Nginx service's route without any client certificate or key:

curl -ik "https://127.0.0.1:8443/hello"

You should receive an HTTP/1.1 400 Bad Request response.

Configure mTLS for APISIX

You can optionally load the content of client.crt and client.key into environment variables:

client_cert=$(cat client.crt)
client_key=$(cat client.key)

Admin API

Create a route to the NGINX server:

curl -i "http://127.0.0.1:9180/apisix/admin/routes" -X PUT -d '
{
  "id": "mtls-nginx",
  "uri": "/hello",
  "upstream": {
    "scheme": "https",
    "nodes": {
      "quickstart-nginx:8443":1
    },
    "tls": {
      "client_cert": "'"${client_cert}"'",
      "client_key": "'"${client_key}"'"
    },
    "type": "roundrobin"
  }
}'

❶ scheme: set to https.

❷ nodes: set to the NGINX server hostname quickstart-nginx and port 8443.

❸ tls.client_cert: configure the certificate public key client.crt.

❹ tls.client_key: configure the certificate private key client.key.

Ingress Controller

important

You should enable SSL for the gateway if you have not already:

helm upgrade apisix apisix/apisix \
  --namespace ingress-apisix \
  --set apisix.deployment.role=traditional \
  --set apisix.deployment.role_traditional.config_provider=yaml \
  --set etcd.enabled=false \
  --set ingress-controller.enabled=true \
  --set ingress-controller.config.provider.type=apisix-standalone \
  --set ingress-controller.apisix.adminService.namespace=ingress-apisix \
  --set ingress-controller.gatewayProxy.createDefault=true \
  --set apisix.ssl.enabled=true

Port forward the SSL port to your host machine:

kubectl port-forward svc/apisix-gateway 9443:443 &

Gateway API

Gateway API does not currently support the configuration of upstream mTLS.

APISIX CRD

Create a Kubernetes secret manifest file:

kubectl create secret tls test-mtls-secret \
  --cert=client.crt \
  --key=client.key \
  --namespace=ingress-apisix \
  --dry-run=client -o yaml > client-secret.yaml

Apply the secret to your cluster:

kubectl apply -f client-secret.yaml

Create a Kubernetes manifest file for a route to the NGINX server:

upstream-mtls-route.yaml

apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
  name: quickstart-nginx
spec:
  ingressClassName: apisix
  scheme: https
  tlsSecret:
    name: test-mtls-secret
    namespace: ingress-apisix
---
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: nginx
spec:
  ingressClassName: apisix
  http:
  - name: nginx
    match:
      paths:
      - /hello
    backends:
    - serviceName: quickstart-nginx
      servicePort: 8443

Apply the configuration to your cluster:

kubectl apply -f upstream-mtls-route.yaml

Verify mTLS between APISIX and Upstream Service

Send a request to the route:

curl -ikv "http://127.0.0.1:9080/hello"

You should receive an HTTP/1.1 200 OK response and see the following message:

Hello APISIX!

This verifies the successful establishment of mTLS between APISIX and the upstream service.

Next Steps

You have learned how to set up mTLS between APISIX and upstream services. APISIX also supports mTLS between clients and APISIX. See Configure mTLS between Client and APISIX to learn more.