Skip to main content
API7.ai Logo

Integrate with Kubernetes Service Discovery

Kubernetes tracks the ready pod addresses behind each Service through Endpoints or EndpointSlices. APISIX can watch these resources and route traffic directly to changing pod addresses instead of using a Service virtual IP or cluster DNS name.

The integration supports one Kubernetes cluster or multiple named clusters. APISIX authenticates to each Kubernetes API with a service-account bearer token and makes the discovered addresses available to upstreams.

Prerequisites​

  • An APISIX deployment that can connect to the Kubernetes API server and discovered pod addresses.
  • A Kubernetes ServiceAccount token, supplied directly with client.token or through client.token_file.
  • get, list, and watch permissions for endpoints. Add the same permissions for endpointslices in the discovery.k8s.io API group when watch_endpoint_slices is true.
  • The Kubernetes API server CA certificate in a PEM bundle available to APISIX when TLS verification is enabled.

Grant Kubernetes API Access​

The following manifest creates the apisix namespace and grants the required discovery permissions. If APISIX runs in another namespace, replace apisix in the Namespace, ServiceAccount, and ClusterRoleBinding subject.

kubernetes-discovery-rbac.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: apisix
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: apisix-discovery
  namespace: apisix
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: apisix-discovery
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["discovery.k8s.io"]
    resources: ["endpointslices"]
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: apisix-discovery
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: apisix-discovery
subjects:
  - kind: ServiceAccount
    name: apisix-discovery
    namespace: apisix

Apply the RBAC resources:

kubectl apply -f kubernetes-discovery-rbac.yaml

When APISIX runs in Kubernetes and reads the mounted service-account token, set spec.template.spec.serviceAccountName in the APISIX Deployment or StatefulSet to apisix-discovery. Applying the RBAC manifest does not change the identity of existing pods. Roll out the workload after updating its pod specification.

Configure a Single Kubernetes Cluster​

When APISIX runs in a pod, the default configuration reads the Kubernetes API address and mounted service-account token from the pod environment:

config.yaml
discovery:
  kubernetes: {}

For production, explicitly enable certificate verification. Create and mount a PEM bundle that contains the Kubernetes API CA together with every system or custom CA that APISIX already trusts. The apisix.ssl.ssl_trusted_certificate setting is global, so replacing an existing bundle with the Kubernetes CA alone can break other TLS connections. Keep the API server hostname consistent with its certificate.

config.yaml
apisix:
  ssl:
    ssl_trusted_certificate: /usr/local/apisix/conf/trusted-ca-bundle.pem

discovery:
  kubernetes:
    service:
      schema: https
      host: ${KUBERNETES_SERVICE_HOST}
      port: ${KUBERNETES_SERVICE_PORT}
      ssl_verify: true
    client:
      token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
    namespace_selector:
      equal: default
    watch_endpoint_slices: true

service.ssl_verify defaults to false for compatibility, including when service.schema is https. This accepts an untrusted API server certificate and is not recommended for production. Set it to true and configure apisix.ssl.ssl_trusted_certificate with the combined trust bundle.

When APISIX runs outside Kubernetes, set service.host and service.port explicitly. Supply either client.token or a readable client.token_file; HTTPS API servers require a nonempty token.

Configure Multiple Kubernetes Clusters​

Use an array to configure multiple clusters. Each item requires a unique id containing 1–64 lowercase letters or digits, an explicit API server address, and client credentials. Defaults from the single-cluster service and client fields are not applied to array items.

config.yaml
apisix:
  ssl:
    ssl_trusted_certificate: /usr/local/apisix/conf/trusted-ca-bundle.pem

discovery:
  kubernetes:
    - id: prod
      service:
        schema: https
        host: prod-api.example.com
        port: "6443"
        ssl_verify: true
      client:
        token_file: /usr/local/apisix/conf/prod-service-account.token
      namespace_selector:
        match:
          - ^prod-
      watch_endpoint_slices: true
    - id: staging
      service:
        schema: https
        host: staging-api.example.com
        port: "6443"
        ssl_verify: true
      client:
        token_file: /usr/local/apisix/conf/staging-service-account.token
      namespace_selector:
        equal: staging
      watch_endpoint_slices: true

The CA bundle must trust every configured API server and retain the roots required by other HTTPS connections from APISIX. After changing discovery or trust settings, reload APISIX.

Deploy a Sample Service​

Deploy an HTTPBin workload and a Service with a named http port in the default namespace:

httpbin.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpbin
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: httpbin
  template:
    metadata:
      labels:
        app: httpbin
    spec:
      containers:
        - name: httpbin
          image: mccutchen/go-httpbin:v2.15.0
          ports:
            - name: http
              containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: httpbin
  namespace: default
spec:
  selector:
    app: httpbin
  ports:
    - name: http
      port: 80
      targetPort: http

Apply the resources and wait for both pods to become ready:

kubectl apply -f httpbin.yaml
kubectl rollout status deployment/httpbin -n default

Route to a Discovered Service​

For a single cluster, use namespace/service:port-name as service_name. For multiple clusters, prefix the name with the cluster ID: cluster-id/namespace/service:port-name. If the Kubernetes resource defines a named port, use the port name rather than its number.

Create a route that sends requests to the httpbin Service in the default namespace through its http port:

curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
  -H "X-API-KEY: ${ADMIN_API_KEY}" \
  -d '{
    "id": "kubernetes-discovery-route",
    "uri": "/anything",
    "upstream": {
      "type": "roundrobin",
      "discovery_type": "kubernetes",
      "service_name": "default/httpbin:http"
    }
  }'

For the prod item in the multiple-cluster example, use prod/default/httpbin:http instead.

Send a request to verify that APISIX resolves a ready endpoint and proxies the request:

curl -i "http://127.0.0.1:9080/anything"

An HTTP/1.1 200 OK response confirms that the route reached a discovered endpoint.

Verify Discovery and TLS Trust​

If the Control API is enabled, inspect the discovery cache:

curl "http://127.0.0.1:9090/v1/discovery/kubernetes/dump"

Confirm that the output contains the expected namespace, service, port name, and ready endpoint addresses. With multiple clusters, also confirm the expected cluster id.

To verify trusted-certificate behavior:

  1. Configure service.ssl_verify: true and the correct CA bundle, reload APISIX, and confirm the expected endpoints appear in the discovery dump.
  2. In an isolated test deployment, point apisix.ssl.ssl_trusted_certificate to a bundle that does not trust the API server, or use a host that does not match the certificate. Reload APISIX and confirm the error log reports certificate verification failure and no fresh endpoint updates are received. Previously discovered entries can remain in shared memory during a reload, so do not use the presence of an old entry as proof that the new TLS connection succeeded.
  3. Restore the correct CA bundle and host, reload APISIX, and confirm discovery recovers.

Do not use ssl_verify: false as the fix for a production trust failure. Correct the CA bundle, certificate chain, API server hostname, or system time instead.

Next Steps​

Kubernetes discovery supports namespace selectors, Kubernetes label-selector expressions, configurable endpoint weights, shared-memory sizing, and either Endpoints or EndpointSlices. For the complete discovery state and troubleshooting endpoints, see the Control API reference.