Implement Traffic Mirroring

Traffic mirroring is a mechanism that duplicates the traffic flowing through the API gateway and forwards the duplicated traffic to a designated upstream, without interrupting the regular services. The mechanism benefits a few use cases, including troubleshooting, security inspection, analytics, and more.

This guide will walk you through the process of implementing traffic mirroring in APISIX using the proxy-mirror plugin.

Prerequisite(s)

• Install Docker.
• Install cURL to send requests to the services for validation.
• Follow the Getting Started tutorial to start a new APISIX instance in Docker.

Start Sample Service to Receive Mirrored Traffic

Start a sample NGINX server for receiving mirrored traffic:

Docker

docker run -p 8081:80 --name nginx nginx

You should see NGINX access log and error log on the terminal session.

Kubernetes

Create a Kubernetes manifest file for a sample NGINX server deployment and service:

nginx.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:latest
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  type: NodePort
  selector:
    app: nginx
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

Apply the configuration to your cluster:

kubectl apply -f nginx.yaml

Configure APISIX

Create a route with the proxy-mirror plugin and configure the address for mirrored traffic. Update the address accordingly.

Admin API

curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT -d '
{
  "id": "traffic-mirror-route",
  "uri": "/get",
  "plugins": {
    "proxy-mirror": {
      "host": "http://192.168.42.145:8081"
    }
  },
  "upstream": {
    "nodes": {
      "httpbin.org": 1
    },
    "type": "roundrobin"
  }
}'

ADC

adc.yaml

services:
  - name: httpbin Service
    routes:
      - uris:
          - /get
        name: traffic-mirror-route
        plugins:
          proxy-mirror:
            host: http://192.168.42.145:8081
    upstream:
      type: roundrobin
      nodes:
        - host: httpbin.org
          port: 80
          weight: 1

Synchronize the configuration to APISIX:

adc sync -f adc.yaml

Ingress Controller

Gateway API

traffic-mirror-route.yaml

apiVersion: v1
kind: Service
metadata:
  name: httpbin-external-domain
spec:
  type: ExternalName
  externalName: httpbin.org
---
apiVersion: apisix.apache.org/v1alpha1
kind: PluginConfig
metadata:
  name: mirror-plugin-config
spec:
  plugins:
    - name: proxy-mirror
      config:
        host: http://nginx-service:80
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: traffic-mirror-route
spec:
  parentRefs:
  - name: apisix
  rules:
  - matches: 
    - path:
        type: Exact
        value: /get
    filters:
    - type: ExtensionRef
      extensionRef:
        group: apisix.apache.org
        kind: PluginConfig
        name: mirror-plugin-config
    backendRefs:
    - name: httpbin-external-domain
      port: 80

APISIX CRD

traffic-mirror-route.yaml

apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
  name: httpbin-external-domain
spec:
  externalNodes:
  - type: Domain
    name: httpbin.org
---
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: traffic-mirror-route
spec:
  ingressClassName: apisix
  http:
    - name: traffic-mirror-route
      match:
        paths:
          - /get
      upstreams:
      - name: httpbin-external-domain
      plugins:
      - name: proxy-mirror
        enable: true
        config:
          host: http://nginx-service:80

Apply the configuration to your cluster:

kubectl apply -f traffic-mirror-route.yaml

Verify Mirroring

Send a request to the route:

curl -i "http://127.0.0.1:9080/get"

You should receive an HTTP/1.1 200 OK response.

To verify, return to the NGINX terminal session (if running in Docker) or check the NGINX pod logs (if running on Kubernetes); you should see the corresponding access log entry:

172.17.0.1 - - [29/Jan/2024:23:11:01 +0000] "GET /get HTTP/1.1" 404 153 "-" "curl/7.64.1" "-"

The HTTP response status is 404 and expected, since the sample NGINX server does not implement the route. This verifies that APISIX has mirrored the request to the NGINX server.

Next Steps

You have now learned how to mirror all ingress traffic to a different upstream. The proxy-mirror plugin also supports mirroring partial traffic and customization of the mirroring timeout values. See plugin doc for more information.