Static Configurations
By default, prometheus configurations are pre-configured in the default configuration.
The prompt- and completion-token histograms default to buckets at 1, 10, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000, 50000, 100000, 200000, 500000, and 1000000 tokens. Configure llm_prompt_tokens_buckets and llm_completion_tokens_buckets when different boundaries better fit the workload.
The file to update depends on how the gateway is deployed:
- Host or Docker
- Kubernetes (Helm)
For host or Docker deployments, configure the following settings:
plugin_attr:
prometheus: # Plugin: prometheus attributes
export_uri: /apisix/prometheus/metrics # Set the URI for the Prometheus metrics endpoint.
metric_prefix: apisix_ # Set the prefix for Prometheus metrics generated by APISIX.
enable_export_server: true # Enable the Prometheus export server.
export_addr: # Set the address for the Prometheus export server.
ip: 127.0.0.1 # Set the IP.
port: 9091 # Set the port.
refresh_interval: 15 # Only available in APISIX.
# Set the interval for refreshing cached metric data, in seconds.
fetch_metric_timeout: 5 # Only available in API7 Enterprise.
# Timeout for fetching metrics in seconds. If exceeded, the API only returns
# basic metrics, including nginx_http_current_connections, http_requests_total,
# etcd_reachable, prometheus_disable, node_info, etcd_modify_indexes,
# shared_dict_capacity_bytes, and shared_dict_free_space_bytes.
allow_degradation: false # Only available in API7 Enterprise.
# If true, allow degradation when shared memory is insufficient.
degradation_pause_steps: [ 60 ] # Only available in API7 Enterprise.
# Time to skip the execution of the plugin in seconds when the plugin is in
# degradation while reclaiming the shared memory used by the plugin.
# metrics: # Create extra labels for metrics.
# http_status: # These metrics will be prefixed with `apisix_`.
# extra_labels: # Set the extra labels for http_status metrics.
# - upstream_addr: $upstream_addr
# - status: $upstream_status
# expire: 0 # The expiration time of metrics in seconds.
# 0 means the metrics will not expire.
# http_latency:
# extra_labels: # Set the extra labels for http_latency metrics.
# - upstream_addr: $upstream_addr
# expire: 0 # The expiration time of metrics in seconds.
# 0 means the metrics will not expire.
# bandwidth:
# extra_labels: # Set the extra labels for bandwidth metrics.
# - upstream_addr: $upstream_addr
# expire: 0 # The expiration time of metrics in seconds.
# 0 means the metrics will not expire.
# default_buckets: # Built-in `http_latency` histogram buckets in milliseconds when this key is omitted.
# Uncomment the list only to override those defaults.
# - 1
# - 2
# - 5
# - 10
# - 20
# - 50
# - 100
# - 200
# - 500
# - 1000
# - 2000
# - 5000
# - 10000
# - 30000
# - 60000
# llm_latency_buckets: # Set buckets for `apisix_llm_latency`, in milliseconds.
# # Introduced in API7 Enterprise 3.9.7 and APISIX 3.17.0.
# # Applies to both `type=total` and `type=ttft`.
# # Introduced in API7 Enterprise 3.9.14 and 3.10.1, and APISIX 3.18.0.
# - 100
# - 500
# - 1000
# - 5000
# llm_prompt_tokens_buckets: # Introduced in API7 Enterprise 3.9.14 and 3.10.1, and APISIX 3.18.0.
# # Set the buckets for `apisix_llm_prompt_tokens_dist` histogram, in tokens.
# - 100
# - 1000
# - 10000
# llm_completion_tokens_buckets: # Introduced in API7 Enterprise 3.9.14 and 3.10.1, and APISIX 3.18.0.
# # Set the buckets for `apisix_llm_completion_tokens_dist` histogram, in tokens.
# - 100
# - 1000
# - 10000
Then reload the gateway for changes to take effect.
For Helm deployments, update the chart values that render plugin_attr.prometheus. Keep the rest of your values file unchanged.
For the APISIX Helm chart, set the following values:
apisix:
pluginAttrs:
prometheus:
export_uri: /apisix/prometheus/metrics
metric_prefix: apisix_
enable_export_server: true
export_addr:
ip: 127.0.0.1
port: 9091
refresh_interval: 15
# Add the remaining prometheus plugin_attr fields here.
For the API7 Gateway Helm chart, set the following values:
pluginAttrs:
prometheus:
export_uri: /apisix/prometheus/metrics
metric_prefix: apisix_
enable_export_server: true
export_addr:
ip: 127.0.0.1
port: 9091
fetch_metric_timeout: 5
allow_degradation: false
degradation_pause_steps: [ 60 ]
# Add the remaining prometheus plugin_attr fields here.
Then apply the values file with the chart used for this gateway release:
helm upgrade <release-name> <chart-name> -n <namespace> -f values.yaml
You can use built-in variables to create extra_labels. See add extra labels for more details.
Parameters
See plugin common configurations for configuration options available to all plugins.
prefer_name
If true, export route/service name instead of their ID in Prometheus metrics.
Plugin Metadata
APISIX and API7 Enterprise both support plugin metadata. The available fields, metadata keys, and version boundaries are described in the table below.
Plugin metadata is configured through the Admin API or declarative configuration. It is separate from the Helm values that render plugin_attr in config.yaml.
disabled_labels
Labels to disable to reduce the number of metrics and prevent resource bottlenecks. APISIX uses
http_statusandhttp_latencyas the metadata keys for the corresponding metrics, while API7 Enterprise usesstatusandlatency. Labels that define a metric's identity cannot be disabled, because collapsing them would merge distinct measurements into a single series. These arecodeon the HTTP status metric,typeon the HTTP latency, bandwidth, and LLM latency metrics, andlayeronai_cache_hits_total. Structural-label validation was introduced in API7 Enterprise 3.9.17 and 3.10.4, and APISIX 3.18.0. See Reduce Metric Cardinality by Disabling Labels for product-specific keys and label sets.http_status
vaild vaule:
Any combination of
route,matched_uri,matched_host,service,consumer,node,request_type,request_llm_model,llm_model, andresponse_sourceLabels to disable for
apisix_http_status. API7 Enterprise usesstatusinstead. Introduced in APISIX 3.18.0.http_latency
vaild vaule:
Any combination of
route,service,consumer,node,request_type,request_llm_model, andllm_modelLabels to disable for
apisix_http_latency. API7 Enterprise useslatencyinstead. Introduced in APISIX 3.18.0.status
vaild vaule:
Any combination of
route,route_id,matched_uri,matched_host,service,service_id,consumer,node,request_type,request_llm_model,llm_model,mcp_request_type, andmcp_tool_nameLabels to disable for
apisix_http_statusmetrics. Available in API7 Enterprise; APISIX useshttp_statusinstead. Therequest_type,request_llm_model, andllm_modellabels were introduced in API7 Enterprise 3.9.7. Themcp_request_typeandmcp_tool_namelabels were introduced in API7 Enterprise 3.9.14.latency
vaild vaule:
Any combination of
route,route_id,service,service_id,consumer,node,request_type,request_llm_model,llm_model,mcp_request_type, andmcp_tool_nameLabels to disable for
apisix_http_latencymetrics. Available in API7 Enterprise; APISIX useshttp_latencyinstead. Therequest_type,request_llm_model, andllm_modellabels were introduced in API7 Enterprise 3.9.7. Themcp_request_typeandmcp_tool_namelabels were introduced in API7 Enterprise 3.9.14.bandwidth
vaild vaule:
APISIX: Any combination of
route,service,consumer,node,request_type,request_llm_model, andllm_model
API7 Enterprise: APISIX values plusroute_id,service_id,mcp_request_type, andmcp_tool_nameLabels to disable for
apisix_bandwidthmetrics. Available in API7 Enterprise and introduced in APISIX 3.18.0. Therequest_type,request_llm_model, andllm_modellabels were introduced in API7 Enterprise 3.9.7. Themcp_request_typeandmcp_tool_namelabels were introduced in API7 Enterprise 3.9.14.stream_status
vaild vaule:
nodeLabels to disable for
apisix_stream_statusmetrics. Thecodeandlisten_addrlabels define the metric's identity and cannot be disabled. Available in API7 Enterprise from version 3.9.19 on the 3.9 line and from version 3.10.6 on the 3.10 line.llm_latency
vaild vaule:
APISIX: Any combination of
route_id,service_id,consumer,node,request_type,request_llm_model, andllm_model
API7 Enterprise: APISIX values plusrouteandserviceLabels to disable for
apisix_llm_latencymetrics. Introduced in API7 Enterprise 3.9.7 and APISIX 3.18.0.llm_prompt_tokens
vaild vaule:
APISIX: Any combination of
route_id,service_id,consumer,node,request_type,request_llm_model, andllm_model
API7 Enterprise: APISIX values plusroute,matched_uri,matched_host, andserviceLabels to disable for
apisix_llm_prompt_tokensmetrics. Introduced in API7 Enterprise 3.9.7 and APISIX 3.18.0.llm_completion_tokens
vaild vaule:
APISIX: Any combination of
route_id,service_id,consumer,node,request_type,request_llm_model, andllm_model
API7 Enterprise: APISIX values plusroute,matched_uri,matched_host, andserviceLabels to disable for
apisix_llm_completion_tokensmetrics. Introduced in API7 Enterprise 3.9.7 and APISIX 3.18.0.llm_active_connections
vaild vaule:
Any combination of
route,route_id,matched_uri,matched_host,service,service_id,consumer,node,request_type,request_llm_model, andllm_modelLabels to disable for
apisix_llm_active_connectionsmetrics. Introduced in API7 Enterprise 3.9.7 and APISIX 3.18.0.llm_prompt_tokens_dist
vaild vaule:
APISIX: Any combination of
route_id,service_id,consumer,node,request_type,request_llm_model, andllm_model
API7 Enterprise: APISIX values plusroute,matched_uri,matched_host, andserviceLabels to disable for
apisix_llm_prompt_tokens_distmetrics. Introduced in API7 Enterprise 3.9.14 and 3.10.1, and APISIX 3.18.0.llm_completion_tokens_dist
vaild vaule:
APISIX: Any combination of
route_id,service_id,consumer,node,request_type,request_llm_model, andllm_model
API7 Enterprise: APISIX values plusroute,matched_uri,matched_host, andserviceLabels to disable for
apisix_llm_completion_tokens_distmetrics. Introduced in API7 Enterprise 3.9.14 and 3.10.1, and APISIX 3.18.0.ai_cache_hits_total
vaild vaule:
APISIX: Any combination of
route,route_id,service,service_id,consumer,node,request_type,request_llm_model, andllm_model
API7 Enterprise: APISIX values plusmatched_uriandmatched_hostLabels to disable for
apisix_ai_cache_hits_totalmetrics. Thelayerlabel is structural and cannot be disabled, because collapsing it would merge exact-match and semantic cache hits into a single series. Introduced in API7 Enterprise 3.9.16 and 3.10.3, and APISIX 3.18.0.ai_cache_misses_total
vaild vaule:
APISIX: Any combination of
route,route_id,service,service_id,consumer,node,request_type,request_llm_model, andllm_model
API7 Enterprise: APISIX values plusmatched_uriandmatched_hostLabels to disable for
apisix_ai_cache_misses_totalmetrics. Introduced in API7 Enterprise 3.9.16 and 3.10.3, and APISIX 3.18.0.ai_cache_bypasses_total
vaild vaule:
APISIX: Any combination of
route,route_id,service,service_id,consumer,node,request_type,request_llm_model, andllm_model
API7 Enterprise: APISIX values plusmatched_uriandmatched_hostLabels to disable for
apisix_ai_cache_bypasses_totalmetrics. Introduced in API7 Enterprise 3.9.16 and 3.10.3, and APISIX 3.18.0.ai_cache_embedding_latency
vaild vaule:
APISIX: Any combination of
route,route_id,service,service_id,consumer,node,request_type,request_llm_model, andllm_model
API7 Enterprise: APISIX values plusmatched_uriandmatched_hostLabels to disable for
apisix_ai_cache_embedding_latencymetrics. Introduced in API7 Enterprise 3.9.16 and 3.10.3, and APISIX 3.18.0.