# Monitor APISIX Metrics with Datadog

Monitoring gateway traffic helps operators identify changes in request volume, latency, and payload size that can affect API reliability. Sending these metrics to [Datadog](https://www.datadoghq.com/) makes APISIX activity available for dashboards, monitors, and operational analysis alongside other application and infrastructure telemetry.

The [`datadog`](https://docs.api7.ai/hub/datadog.md) plugin sends APISIX request, latency, and traffic-size metrics to a DogStatsD agent over UDP. The agent then forwards the metrics to Datadog.

In this example, the DogStatsD agent runs in the APISIX quickstart network. APISIX addresses the agent by container name, and the resulting route metrics are verified in Datadog.

<br />

![APISIX sends metrics to a DogStatsD agent, which forwards them to Datadog](https://static.api7.ai/uploads/2024/01/19/LUWRybZv_dd-sad.png)

## Prerequisite(s)[​](#prerequisites "Direct link to Prerequisite(s)")

* Install [Docker](https://docs.docker.com/get-docker/).
* Install [cURL](https://curl.se/) to send test requests.
* Follow the [Getting Started tutorial](https://docs.api7.ai/apisix/getting-started.md) to start APISIX in Docker. The commands below use the `apisix-quickstart-net` network created by that tutorial.
* Create a [Datadog account](https://www.datadoghq.com/) and obtain an API key.
* If Admin API authentication is enabled, export an admin-role key as `ADMIN_API_KEY`. The Docker quickstart disables authentication for local evaluation; keep it enabled in production.

## Start a DogStatsD Agent[​](#start-a-dogstatsd-agent "Direct link to Start a DogStatsD Agent")

Export your Datadog API key, then set the [Datadog site](https://docs.datadoghq.com/getting_started/site/) and the hostname that identifies this example:

```
export DD_API_KEY=replace-with-datadog-api-key

export DD_SITE=datadoghq.com

export DD_HOSTNAME=apisix-quickstart
```

Start DogStatsD in the APISIX quickstart network:

```
docker rm -f quickstart-dogstatsd 2>/dev/null || true



docker run -d --rm \

  --name quickstart-dogstatsd \

  --network apisix-quickstart-net \

  -e DD_API_KEY \

  -e DD_SITE \

  -e DD_HOSTNAME \

  -e DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true \

  datadog/dogstatsd:7.81.2@sha256:475f87872f2847e1fe4e60370ad0b3a075726aa5b81f5f507f90ce53cecc7f6d
```

`DD_DOGSTATSD_NON_LOCAL_TRAFFIC` allows the agent to receive metrics from the APISIX container. Because APISIX and DogStatsD share `apisix-quickstart-net`, APISIX can send metrics to the agent by container name. You do not need a host-specific IP address or published UDP port.

## Configure the DogStatsD Endpoint[​](#configure-the-dogstatsd-endpoint "Direct link to Configure the DogStatsD Endpoint")

The plugin sends metrics to `127.0.0.1:8125` by default. Configure plugin metadata to use the DogStatsD container instead:

* Admin API
* ADC

```
curl "http://127.0.0.1:9180/apisix/admin/plugin_metadata/datadog" -X PUT \

  -H "X-API-KEY: ${ADMIN_API_KEY}" \

  -H "Content-Type: application/json" \

  -d '{

    "host": "quickstart-dogstatsd",

    "port": 8125,

    "namespace": "apisix"

  }'
```

Export the complete plugin-metadata collection:

```
adc dump -o adc-plugin-metadata.yaml \

  --include-resource-type plugin_metadata
```

caution

ADC dumps can contain decrypted sensitive fields returned by the Admin API. Restrict access to exported files and delete them after completing the change.

Edit the exported file to add the Datadog metadata while preserving metadata for every other plugin:

adc-plugin-metadata.yaml (plugin\_metadata excerpt)

```
plugin_metadata:

  # Keep all other plugin metadata from the exported file.

  datadog:

    host: quickstart-dogstatsd

    port: 8125

    namespace: apisix
```

Preview the complete plugin-metadata collection and confirm that the diff contains no unintended changes or deletions:

```
adc diff -f adc-plugin-metadata.yaml \

  --include-resource-type plugin_metadata
```

Synchronize the reviewed plugin metadata:

```
adc sync -f adc-plugin-metadata.yaml \

  --include-resource-type plugin_metadata
```

## Configure Metric Collection[​](#configure-metric-collection "Direct link to Configure Metric Collection")

Enable `datadog` globally, then create a sample route for monitored traffic:

* Admin API
* ADC

Create a global rule that enables the plugin:

```
curl "http://127.0.0.1:9180/apisix/admin/global_rules/datadog" -X PUT \

  -H "X-API-KEY: ${ADMIN_API_KEY}" \

  -H "Content-Type: application/json" \

  -d '{

    "plugins": {

      "datadog": {}

    }

  }'
```

Create the sample route:

```
curl "http://127.0.0.1:9180/apisix/admin/routes/dd-route" -X PUT \

  -H "X-API-KEY: ${ADMIN_API_KEY}" \

  -H "Content-Type: application/json" \

  -d '{

    "uri": "/anything",

    "upstream": {

      "type": "roundrobin",

      "scheme": "https",

      "pass_host": "node",

      "nodes": {

        "httpbin.org:443": 1

      }

    }

  }'
```

ADC exports global rules as one map keyed by plugin name. That export does not keep the original rule ID or which plugins shared a rule. If more than one rule configures the same plugin, the export keeps only one configuration. Use the Admin API tab in this section when you need to keep the original rule ID or grouping.

Export the current global rules:

```
adc dump -o adc-global-rules.yaml \

  --include-resource-type global_rule
```

caution

ADC dumps can contain decrypted sensitive fields returned by the Admin API. Restrict access to exported files and delete them after completing the change.

Edit the exported file to add the Datadog plugin while preserving every other plugin entry:

adc-global-rules.yaml (global\_rules excerpt)

```
global_rules:

  # Keep every other global plugin entry from the exported file.

  datadog: {}
```

Create the sample service in a separate file:

adc-route.yaml

```
services:

  - name: datadog-httpbin

    labels:

      docs-example: datadog

    routes:

      - name: dd-route

        uris:

          - /anything

    upstream:

      type: roundrobin

      scheme: https

      pass_host: node

      nodes:

        - host: httpbin.org

          port: 443

          weight: 1
```

Preview the global rules and confirm that the diff contains no unrelated plugin changes or deletions:

```
adc diff -f adc-global-rules.yaml \

  --include-resource-type global_rule
```

Preview the service-owned resources:

```
adc diff -f adc-route.yaml \

  --include-resource-type service \

  --label-selector docs-example=datadog
```

Synchronize the reviewed global rules and service:

```
adc sync -f adc-global-rules.yaml \

  --include-resource-type global_rule



adc sync -f adc-route.yaml \

  --include-resource-type service \

  --label-selector docs-example=datadog
```

## Verify Metrics[​](#verify-metrics "Direct link to Verify Metrics")

Send several requests through the route to produce metric data:

```
for _ in 1 2 3; do

  curl -sS "http://127.0.0.1:9080/anything" > /dev/null

done
```

The plugin submits metrics in batches, so they might not appear immediately. In Datadog, open **Metrics → Explorer** and search for metrics under the `apisix` namespace, including:

* `apisix.request.counter`
* `apisix.request.latency`
* `apisix.upstream.latency`
* `apisix.apisix.latency`
* `apisix.ingress.size`
* `apisix.egress.size`

Use these metrics to build [monitors](https://docs.datadoghq.com/monitors/) and [dashboards](https://docs.datadoghq.com/dashboards/). See the [`datadog` Plugin Hub reference](https://docs.api7.ai/hub/datadog.md) for metric tags, batching options, and additional configuration.
