Skip to main content

Static Configurations

By default, prometheus configurations are pre-configured in the default configuration.

The prompt- and completion-token histograms default to buckets at 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 50000, 100000, 200000, 500000, and 1000000 tokens. Configure llm_prompt_tokens_buckets and llm_completion_tokens_buckets when different boundaries better fit the workload.

The file to update depends on how the gateway is deployed:

For host or Docker deployments, configure the following settings:

config.yaml
plugin_attr:
prometheus: # Plugin: prometheus attributes
export_uri: /apisix/prometheus/metrics # Set the URI for the Prometheus metrics endpoint.
metric_prefix: apisix_ # Set the prefix for Prometheus metrics generated by APISIX.
enable_export_server: true # Enable the Prometheus export server.
export_addr: # Set the address for the Prometheus export server.
ip: 127.0.0.1 # Set the IP.
port: 9091 # Set the port.
refresh_interval: 15 # Only available in APISIX.
# Set the interval for refreshing cached metric data, in seconds.
fetch_metric_timeout: 5 # Only available in API7 Enterprise.
# Timeout for fetching metrics in seconds. If exceeded, the API only returns
# basic metrics, including nginx_http_current_connections, http_requests_total,
# etcd_reachable, prometheus_disable, node_info, etcd_modify_indexes,
# shared_dict_capacity_bytes, and shared_dict_free_space_bytes.
allow_degradation: false # Only available in API7 Enterprise.
# If true, allow degradation when shared memory is insufficient.
degradation_pause_steps: [ 60 ] # Only available in API7 Enterprise.
# Time to skip the execution of the plugin in seconds when the plugin is in
# degradation while reclaiming the shared memory used by the plugin.
# metrics: # Create extra labels for metrics.
# http_status: # These metrics will be prefixed with `apisix_`.
# extra_labels: # Set the extra labels for http_status metrics.
# - upstream_addr: $upstream_addr
# - status: $upstream_status
# expire: 0 # The expiration time of metrics in seconds.
# 0 means the metrics will not expire.
# http_latency:
# extra_labels: # Set the extra labels for http_latency metrics.
# - upstream_addr: $upstream_addr
# expire: 0 # The expiration time of metrics in seconds.
# 0 means the metrics will not expire.
# bandwidth:
# extra_labels: # Set the extra labels for bandwidth metrics.
# - upstream_addr: $upstream_addr
# expire: 0 # The expiration time of metrics in seconds.
# 0 means the metrics will not expire.
# default_buckets: # Built-in `http_latency` histogram buckets in milliseconds when this key is omitted.
# Uncomment the list only to override those defaults.
# - 1
# - 2
# - 5
# - 10
# - 20
# - 50
# - 100
# - 200
# - 500
# - 1000
# - 2000
# - 5000
# - 10000
# - 30000
# - 60000
# llm_latency_buckets: # Set buckets for `apisix_llm_latency`, in milliseconds.
# # Introduced in API7 Enterprise 3.9.7 and APISIX 3.17.0.
# # Applies to both `type=total` and `type=ttft`.
# # Introduced in API7 Enterprise 3.9.14 and 3.10.1, and APISIX 3.18.0.
# - 100
# - 500
# - 1000
# - 5000
# llm_prompt_tokens_buckets: # Introduced in API7 Enterprise 3.9.14 and 3.10.1, and APISIX 3.18.0.
# # Set the buckets for `apisix_llm_prompt_tokens_dist` histogram, in tokens.
# - 100
# - 1000
# - 10000
# llm_completion_tokens_buckets: # Introduced in API7 Enterprise 3.9.14 and 3.10.1, and APISIX 3.18.0.
# # Set the buckets for `apisix_llm_completion_tokens_dist` histogram, in tokens.
# - 100
# - 1000
# - 10000

Then reload the gateway for changes to take effect.

You can use built-in variables to create extra_labels. See add extra labels for more details.

Parameters

See plugin common configurations for configuration options available to all plugins.

  • prefer_name

    boolean

    default: false


    If true, export route/service name instead of their ID in Prometheus metrics.

Plugin Metadata

APISIX and API7 Enterprise both support plugin metadata. The available fields, metadata keys, and version boundaries are described in the table below.

Plugin metadata is configured through the Admin API or declarative configuration. It is separate from the Helm values that render plugin_attr in config.yaml.

  • disabled_labels

    object


    Labels to disable to reduce the number of metrics and prevent resource bottlenecks. APISIX uses http_status and http_latency as the metadata keys for the corresponding metrics, while API7 Enterprise uses status and latency. Labels that define a metric's identity cannot be disabled, because collapsing them would merge distinct measurements into a single series. These are code on the HTTP status metric, type on the HTTP latency, bandwidth, and LLM latency metrics, and layer on ai_cache_hits_total. Structural-label validation was introduced in API7 Enterprise 3.9.17 and 3.10.4, and APISIX 3.18.0. See Reduce Metric Cardinality by Disabling Labels for product-specific keys and label sets.

    • http_status

      array[string]

      vaild vaule:

      Any combination of route, matched_uri, matched_host, service, consumer, node, request_type, request_llm_model, llm_model, and response_source


      Labels to disable for apisix_http_status. API7 Enterprise uses status instead. Introduced in APISIX 3.18.0.

    • http_latency

      array[string]

      vaild vaule:

      Any combination of route, service, consumer, node, request_type, request_llm_model, and llm_model


      Labels to disable for apisix_http_latency. API7 Enterprise uses latency instead. Introduced in APISIX 3.18.0.

    • status

      array[string]

      vaild vaule:

      Any combination of route, route_id, matched_uri, matched_host, service, service_id, consumer, node, request_type, request_llm_model, llm_model, mcp_request_type, and mcp_tool_name


      Labels to disable for apisix_http_status metrics. Available in API7 Enterprise; APISIX uses http_status instead. The request_type, request_llm_model, and llm_model labels were introduced in API7 Enterprise 3.9.7. The mcp_request_type and mcp_tool_name labels were introduced in API7 Enterprise 3.9.14.

    • latency

      array[string]

      vaild vaule:

      Any combination of route, route_id, service, service_id, consumer, node, request_type, request_llm_model, llm_model, mcp_request_type, and mcp_tool_name


      Labels to disable for apisix_http_latency metrics. Available in API7 Enterprise; APISIX uses http_latency instead. The request_type, request_llm_model, and llm_model labels were introduced in API7 Enterprise 3.9.7. The mcp_request_type and mcp_tool_name labels were introduced in API7 Enterprise 3.9.14.

    • bandwidth

      array[string]

      vaild vaule:

      APISIX: Any combination of route, service, consumer, node, request_type, request_llm_model, and llm_model
      API7 Enterprise: APISIX values plus route_id, service_id, mcp_request_type, and mcp_tool_name


      Labels to disable for apisix_bandwidth metrics. Available in API7 Enterprise and introduced in APISIX 3.18.0. The request_type, request_llm_model, and llm_model labels were introduced in API7 Enterprise 3.9.7. The mcp_request_type and mcp_tool_name labels were introduced in API7 Enterprise 3.9.14.

    • stream_status

      array[string]

      vaild vaule:

      node


      Labels to disable for apisix_stream_status metrics. The code and listen_addr labels define the metric's identity and cannot be disabled. Available in API7 Enterprise from version 3.9.19 on the 3.9 line and from version 3.10.6 on the 3.10 line.

    • llm_latency

      array[string]

      vaild vaule:

      APISIX: Any combination of route_id, service_id, consumer, node, request_type, request_llm_model, and llm_model
      API7 Enterprise: APISIX values plus route and service


      Labels to disable for apisix_llm_latency metrics. Introduced in API7 Enterprise 3.9.7 and APISIX 3.18.0.

    • llm_prompt_tokens

      array[string]

      vaild vaule:

      APISIX: Any combination of route_id, service_id, consumer, node, request_type, request_llm_model, and llm_model
      API7 Enterprise: APISIX values plus route, matched_uri, matched_host, and service


      Labels to disable for apisix_llm_prompt_tokens metrics. Introduced in API7 Enterprise 3.9.7 and APISIX 3.18.0.

    • llm_completion_tokens

      array[string]

      vaild vaule:

      APISIX: Any combination of route_id, service_id, consumer, node, request_type, request_llm_model, and llm_model
      API7 Enterprise: APISIX values plus route, matched_uri, matched_host, and service


      Labels to disable for apisix_llm_completion_tokens metrics. Introduced in API7 Enterprise 3.9.7 and APISIX 3.18.0.

    • llm_active_connections

      array[string]

      vaild vaule:

      Any combination of route, route_id, matched_uri, matched_host, service, service_id, consumer, node, request_type, request_llm_model, and llm_model


      Labels to disable for apisix_llm_active_connections metrics. Introduced in API7 Enterprise 3.9.7 and APISIX 3.18.0.

    • llm_prompt_tokens_dist

      array[string]

      vaild vaule:

      APISIX: Any combination of route_id, service_id, consumer, node, request_type, request_llm_model, and llm_model
      API7 Enterprise: APISIX values plus route, matched_uri, matched_host, and service


      Labels to disable for apisix_llm_prompt_tokens_dist metrics. Introduced in API7 Enterprise 3.9.14 and 3.10.1, and APISIX 3.18.0.

    • llm_completion_tokens_dist

      array[string]

      vaild vaule:

      APISIX: Any combination of route_id, service_id, consumer, node, request_type, request_llm_model, and llm_model
      API7 Enterprise: APISIX values plus route, matched_uri, matched_host, and service


      Labels to disable for apisix_llm_completion_tokens_dist metrics. Introduced in API7 Enterprise 3.9.14 and 3.10.1, and APISIX 3.18.0.

    • ai_cache_hits_total

      array[string]

      vaild vaule:

      APISIX: Any combination of route, route_id, service, service_id, consumer, node, request_type, request_llm_model, and llm_model
      API7 Enterprise: APISIX values plus matched_uri and matched_host


      Labels to disable for apisix_ai_cache_hits_total metrics. The layer label is structural and cannot be disabled, because collapsing it would merge exact-match and semantic cache hits into a single series. Introduced in API7 Enterprise 3.9.16 and 3.10.3, and APISIX 3.18.0.

    • ai_cache_misses_total

      array[string]

      vaild vaule:

      APISIX: Any combination of route, route_id, service, service_id, consumer, node, request_type, request_llm_model, and llm_model
      API7 Enterprise: APISIX values plus matched_uri and matched_host


      Labels to disable for apisix_ai_cache_misses_total metrics. Introduced in API7 Enterprise 3.9.16 and 3.10.3, and APISIX 3.18.0.

    • ai_cache_bypasses_total

      array[string]

      vaild vaule:

      APISIX: Any combination of route, route_id, service, service_id, consumer, node, request_type, request_llm_model, and llm_model
      API7 Enterprise: APISIX values plus matched_uri and matched_host


      Labels to disable for apisix_ai_cache_bypasses_total metrics. Introduced in API7 Enterprise 3.9.16 and 3.10.3, and APISIX 3.18.0.

    • ai_cache_embedding_latency

      array[string]

      vaild vaule:

      APISIX: Any combination of route, route_id, service, service_id, consumer, node, request_type, request_llm_model, and llm_model
      API7 Enterprise: APISIX values plus matched_uri and matched_host


      Labels to disable for apisix_ai_cache_embedding_latency metrics. Introduced in API7 Enterprise 3.9.16 and 3.10.3, and APISIX 3.18.0.