datadog

The datadog plugin supports the integration with Datadog, one of the most used observability service for cloud applications. When enabled, the plugin pushes metrics to DogStatsD server, which comes bundled with the Datadog agent, over UDP protocol.

Metrics

The plugin exports the following metrics by default.

All metrics will be prefixed by the namespace configured in metadata. For example, if the namespace is configured to be apisix, you will see the request.counter metric exported as apisix.request.counter in Datadog.

Name	Type	Description
request.counter	counter	Number of requests received.
request.latency	histogram	Time taken to process the request, in milliseconds.
upstream.latency	histogram	Time taken to proxy the request to the upstream server until a response is received, in milliseconds.
apisix.latency	histogram	Time taken by APISIX agent to process the request, in milliseconds.
ingress.size	timer	Request body size in bytes.
egress.size	timer	Response body size in bytes.

Name	Description
route_name	Name of the route. If not present or if the attribute `prefer_name` is set to false, fall back to the route ID.
service_name	Name of the service. If not present or if the attribute `prefer_name` is set to false, fall back to the service ID.
consumer	Username of the consumer if the route is connected to a consumer.
balancer_ip	IP address of the upstream balancer that processes the current request.
response_status	HTTP response status code.
scheme	Request scheme, such as HTTP and gRPC.

Examples

The examples below demonstrate how you can configure datadog plugin for different scenarios.

Before proceeding, please make sure you have installed Datadog agent which collects events and metrics from monitored objects and sends them to Datadog.

Start the Datadog agent in Docker:

docker run -d \
  --name dogstatsd-agent \
  -e DD_API_KEY=35ebe12345678dec56218930b79fdb4cf \
  -e DD_SITE="us5.datadoghq.com" \
  -e DD_HOSTNAME=apisix.quickstart \
  -e DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true \
  -p 8125:8125/udp \
  datadog/dogstatsd:latest

❶ DD_API_KEY: replace with your API key.

❷ DD_SITE: replace with your Datadog site.

❸ DD_HOSTNAME: replace with your hostname.

❹ DD_DOGSTATSD_NON_LOCAL_TRAFFIC: set to true to listen to DogStatsD packets from other containers.

You can configure most options in the agent’s main configuration file datadog.yaml through environment variables, prefixed with DD_. For more information, see agent environment variables.

Update Datadog Agent Address and Other Metadata

By default, the plugin expects the DogStatsD server to be available at 127.0.0.1:8125. To customize the address and other metadata, update the plugin metadata as such:

curl "http://127.0.0.1:9180/apisix/admin/plugin_metadata/datadog" -X PUT \
  -H "X-API-KEY: ${ADMIN_API_KEY}" \
  -d '{
    "host": "192.168.0.90",
    "port": 8125,
    "namespace": "apisix",
    "constant_tags": [
      "source:apisix",
      "service:custom"
    ]
  }'

❶ Replace with your private IP address if you are running both APISIX and Datadog agent in Docker.

❷ Set to Datadog agent listening port.

❸ Set namespace which prefixes all metrics.

❹ Configure constant tags.

To reset to default configuration, send a request to the datadog plugin metadata with an empty body:

curl "http://127.0.0.1:9180/apisix/admin/plugin_metadata/datadog" -X PUT -d '{}'

Monitor Route Metrics

The example below shows how you can send the metrics of a particular route to Datadog.

Create a route with the datadog plugin and a few optional configuration options:

curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
  -H "X-API-KEY: ${ADMIN_API_KEY}" \
  -d '{
    "id": "datadog-route",
    "uri": "/anything",
    "plugins": {
      "datadog": {
        "batch_max_size" : 1,
        "max_retry_count": 0
      }
    },
    "upstream": {
      "type": "roundrobin",
      "nodes": {
        "httpbin.org": 1
      }
    }
  }'

❶ batch_max_size: set to 1 to send the metric immediately.

❷ max_retry_count: set to 0 to disallow retries if metrics were unsuccessfully sent.

Generate a few requests to the previously created route:

curl "http://127.0.0.1:9080/anything"

In Datadog, Select Metrics from the left menu and go to Explorer. Select apisix.ingress.size.count as the metric. You should see the count reflecting the number of requests generated:

apisix-datadog-ingress-size-count

datadog

Metrics​

Tags​

Examples​

Update Datadog Agent Address and Other Metadata​

Monitor Route Metrics​

Metrics

Tags

Examples

Update Datadog Agent Address and Other Metadata

Monitor Route Metrics