Metric Labels and Variables

Configure observability.metrics.labels to select the complete label list for each Prometheus metric family. This supports adding available variables and removing default labels. Gateways running 1.1.0 or earlier do not support this option.

Configure Labels

config.yaml

observability:
  metrics:
    labels:
      aisix_request_ttft_seconds:
        - env_id
        - endpoint
        - model
        - provider
        - status_class
        - streaming
        - provider_key_name
      aisix_proxy_requests_total:
        - endpoint
        - status
      aisix_requests_total: []

This example adds the selected provider credential's name to TTFT. It keeps only endpoint and status on the detailed request counter. It removes all business labels from the legacy request counter.

• An omitted metric keeps its existing defaults. labels: {} preserves the entire default metric surface.
• Each configured list replaces that metric's default list. An empty list removes all business labels where no identity labels are required.
• Use the metric family name, such as aisix_request_ttft_seconds, without _bucket, _sum, or _count. Histogram components always share the selected labels. Prometheus still adds le; summaries still add quantile.
• side on aisix_request_ttft_seconds, aisix_llm_time_to_first_token_seconds, and aisix_request_e2e_latency_seconds is always emitted. A replacement list cannot remove it, because its two values are different measurements that would otherwise merge into one series. Listing it is accepted.
• Unknown metric names, unsupported variables, duplicate variables, and removal of required identity labels cause startup to fail with a configuration error.
• Labels are fixed at startup. Restart the gateway after changing the selection. Existing stored series remain in Prometheus according to its retention policy; update queries and alerts that use removed labels.

For deployments configured through environment variables, set the entire map as one JSON object:

export AISIX_OBSERVABILITY__METRICS__LABELS='{"aisix_request_ttft_seconds":["model","provider_key_name"]}'

Label selection is part of a gateway's own configuration, so it is applied by whoever operates the gateway. AISIX Cloud generates the basic deployment template only; add observability.metrics.labels to the gateway's configuration file, or set the environment variable above, on each gateway that needs it. Apply it only to a gateway version that supports the option.

Aggregation and Cardinality

Removing labels merges counter observations and histogram distributions before they are accumulated. For example, removing model combines requests across models; it does not select one model's last sample. The configuration does not change which requests or attempts a metric counts.

Gauge values describe current state, such as one credential's remaining quota or one deployment's health. The identity labels listed below must remain, so independent objects cannot overwrite one another. Optional gauge labels can still be added or removed.

Each distinct combination of selected label values creates a time series. Credential names, API key IDs, and member identities can multiply series counts. A classic histogram creates a series for each finite bucket, the +Inf bucket, _sum, and _count for every combination. Select only dimensions needed by your queries.

Removing a categorical label also combines its categories. In particular, token_type="total" already includes input and output; removing token_type sums all three categories and must not be interpreted as the original total.

Large Deployments

Default labels stay enabled unless you configure a replacement. For deployments with many API keys or members, select dimensions from the queries and alerts you need. Use request logs or exported usage events for individual-call investigations when per-member Prometheus series are unnecessary.

Start with these choices:

• Keep model, provider, protocol, status and outcome dimensions needed by service-level queries. Keep is_fallback when distinguishing recovered requests. Keep category labels such as token_type; removing them changes what a sum means.
• Remove api_key_id, user_id and user_name from request, duration, token and usage-event families when you do not query individual callers. Keep team_id only on families where team-level monitoring is needed.
• Prefer provider_key_id to a credential display name when monitoring credential health. A rename changes a name-based series. Add env_id if your deployment and queries need that dimension.
• Keep the required identities of budget, remaining-quota, deployment-state and other gauges. These are independent current values, so dropping their identities would allow one object's value to overwrite another. The gateway rejects such configurations.

The following example groups detailed traffic metrics by model and provider credential, removing caller identities and display names. Add these entries to your existing observability.metrics.labels map; preserve settings for other families. Each line replaces the named family's complete label list, so adapt it to your dashboards before restarting the gateway.

config.yaml (observability excerpt)

observability:
  metrics:
    labels:
      aisix_proxy_requests_total: [endpoint, upstream_protocol, provider, model, provider_key_id, inbound_protocol, stream, is_fallback, status, outcome]
      aisix_proxy_failed_requests_total: [endpoint, upstream_protocol, provider, model, provider_key_id, inbound_protocol, stream, is_fallback, status, outcome]
      aisix_llm_requests_total: [endpoint, upstream_protocol, provider, model, provider_key_id, inbound_protocol, stream, is_fallback, status, outcome]
      aisix_proxy_request_duration_seconds: [endpoint, upstream_protocol, provider, model, provider_key_id, inbound_protocol, stream, status, outcome]
      aisix_llm_request_duration_seconds: [endpoint, upstream_protocol, provider, model, provider_key_id, inbound_protocol, stream, status, outcome]
      aisix_llm_time_to_first_token_seconds: [endpoint, upstream_protocol, provider, model, provider_key_id]
      aisix_llm_spend_micro_usd_total: [endpoint, upstream_protocol, provider, model, provider_key_id]
      aisix_llm_input_tokens_total: [endpoint, upstream_protocol, provider, model, provider_key_id]
      aisix_llm_output_tokens_total: [endpoint, upstream_protocol, provider, model, provider_key_id]
      aisix_llm_total_tokens_total: [endpoint, upstream_protocol, provider, model, provider_key_id]
      aisix_llm_cached_input_tokens_total: [endpoint, upstream_protocol, provider, model, provider_key_id]
      aisix_llm_cache_read_input_tokens_total: [endpoint, upstream_protocol, provider, model, provider_key_id]
      aisix_llm_cache_creation_input_tokens_total: [endpoint, upstream_protocol, provider, model, provider_key_id]
      aisix_usage_events_emitted_total: [handler, status_code, status, inbound_protocol, upstream_protocol, model, provider_key_id]
      aisix_usage_event_drops_total: [reason, model, provider_key_id, upstream_protocol]

This example leaves the SLO latency histograms at their defaults and keeps budget and quota gauges unchanged. Their per-object series still contribute to total size. Metrics not listed retain all their default labels, so review the other families in the table below against your workload.

Size for observed label combinations, not only configured objects or current request rate. Once many distinct callers have sent requests, their cumulative series remain even while business traffic is quiet. Each default duration summary exports seven quantiles plus sum and count: two such families with 50,000 observed combinations already produce 900,000 sample lines, before counters and other metrics. A histogram with N finite bucket boundaries exports N + 3 series per combination. Reducing labels before recording reduces gateway storage, maintenance, serialization and transfer costs. Prometheus-side metric_relabel_configs runs after the gateway has generated the response and does not remove that work; dropping labels there also does not aggregate samples.

Scrape each gateway instance directly, rather than putting a load-balanced service in front of several independent counters. Start with a 30-second interval and a 10-second timeout, then measure at expected peak cardinality, including the first scrape after warm-up. Keep the timeout shorter than the interval and choose the interval according to alert freshness requirements. Raising a timeout alone does not reduce series count or CPU use.

Check scrape_duration_seconds, scrape_samples_scraped, up, gateway CPU, RSS and request latency together. For example, find the largest ingested AISIX families with:

topk(10, count by (__name__) ({job="aisix", __name__=~"aisix_.+"}))

Replace job="aisix" with your scrape job. After changing labels, verify the new scrape, update affected dashboards and alerts, and allow old Prometheus series to age out under the configured retention policy.

Supported Variables

Variable names are also the emitted label names. Availability depends on the metric: the next table lists its defaults and additional supported variables. Missing request attribution uses unknown; labels that already define a different missing-value category keep that category. Variables read from request context are unavailable on background metrics that have no such context. Arbitrary request headers and expressions are not supported.

Variable	Meaning
env_id	The gateway's AISIX Cloud environment ID; unknown when not connected to AISIX Cloud. Available on every metric.
trigger	What caused a configuration apply: watch for a coalesced batch of watch events, full for a read of every watched prefix.
endpoint	The matched route template, without caller-supplied path parameters.
inbound_protocol	The protocol used by the caller, derived from the matched endpoint.
upstream_protocol	The wire protocol of the selected provider credential; unknown before upstream selection or for non-LLM traffic.
provider	The provider kind attributed to this observation; ensemble when no single provider owns the result.
model	The configured gateway model identity attributed to the observation. Wildcard requests use the configured model pattern. See the metric reference for its request or attempt scope.
upstream_model	The upstream model identity, bounded to the configured model pattern for wildcard models.
provider_key_id	The identifier of the selected provider credential. This is an identifier, never its secret.
provider_key_name	The display name of that same provider credential. Renaming a credential starts a new series when this variable is selected.
api_key_id	The identifier of the authenticating gateway API key, never its plaintext value.
team_id	The team associated with the authenticating API key.
user_id	The member associated with the authenticating API key.
user_name	The member display name carried by the API key's configuration snapshot.
stream	Whether the request asked for streaming, encoded as true or false. Used by the detailed request metrics.
streaming	Whether the observed request is streaming, encoded as true or false. Used by the request latency histograms.
is_fallback	Whether request attribution identifies a fallback attempt, encoded as true or false.
status	The HTTP status code on request and usage-event metrics; the HTTP status class on A2A request metrics.
status_class	The HTTP status class: 2xx, 3xx, 4xx, 5xx, or other.
side	Which interval a latency observation measures. On the two TTFT metrics: upstream, from the start of the attempt that served the request to the first upstream frame, or downstream, from the gateway receiving the request to the first frame written to the client. On aisix_request_e2e_latency_seconds: upstream, the upstream duration of the attempt that produced the response, or downstream, the whole request as the client experienced it. Always emitted on the metrics that carry it and cannot be removed by a label selection. Gateways running 1.4.0 or earlier emit no side label on the TTFT metrics, and gateways running 1.5.0 or earlier emit none on aisix_request_e2e_latency_seconds.
status_code	The HTTP status class on usage-event metrics: 2xx, 3xx, 4xx, 5xx, or other. The status variable retains the exact code.
outcome	The outcome defined by the metric, such as the request result, cache decision, or request-body-limit result.
client_type	The classified client name from built-in or configured client_type_rules; never the raw User-Agent.
token_type	The token count category: input, output, or total. Total already includes the input and output categories.
fallback_model	The configured fallback model attributed to a routing fallback.
scope	The scope of the rate-limit decision.
layer	The layer of the rate-limit decision.
policy_id	The rate-limit policy identifier, or the metric's missing-policy value.
reason	The bounded reason code defined by the emitting authentication, guardrail, configuration, or exporter metric.
method	The authentication method used for the credential decision.
result	The authentication or guardrail execution result, as defined by the metric.
guardrail	The configured guardrail name.
kind	The guardrail kind on guardrail metrics, or the resource kind on configuration metrics.
phase	The guardrail execution phase.
error_type	The bounded guardrail failure category, or none when no error occurred.
handler	The usage event's handler family, such as chat, messages, embeddings, or mcp.
policy	The configured cache policy name.
cause	The bounded semantic-cache embedding failure cause.
op	The semantic-cache storage operation.
operation	The Redis operation or A2A operation, as defined by the metric.
exporter	The configured observability exporter name.
agent	The registered A2A agent name.
state	The A2A task state reported by the upstream agent.
hash	The hash of the applied gateway resource configuration.

Variables Available per Metric

env_id is available on every metric. All default labels are selectable variables. The additional column lists variables that can be added beyond those defaults; a dash means none. Required identities must be included in a replacement list.

Metric family	Default labels	Additional variables	Required identities
aisix_requests_total	provider, model, status, outcome	env_id	—
aisix_request_duration_seconds	provider, model, status	env_id	—
aisix_ratelimit_rejections_total	scope, layer, policy_id	env_id	—
aisix_tokens_consumed_total	provider, model	env_id	—
aisix_llm_spend_micro_usd_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	env_id	—
aisix_llm_input_tokens_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	env_id	—
aisix_llm_output_tokens_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	env_id	—
aisix_llm_total_tokens_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	env_id	—
aisix_llm_cached_input_tokens_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	env_id	—
aisix_llm_cache_read_input_tokens_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	env_id	—
aisix_llm_cache_creation_input_tokens_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	env_id	—
aisix_llm_requests_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name, stream, is_fallback, status, outcome	env_id	—
aisix_llm_request_duration_seconds	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name, stream, status, outcome	is_fallback, env_id	—
aisix_llm_time_to_first_token_seconds	side (always emitted), endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	env_id	—
aisix_llm_tokens_by_client_total	client_type, model, token_type	env_id	—
aisix_proxy_in_flight_requests	endpoint, inbound_protocol	env_id	endpoint
aisix_proxy_requests_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name, stream, is_fallback, status, outcome	env_id	—
aisix_proxy_failed_requests_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name, stream, is_fallback, status, outcome	env_id	—
aisix_proxy_request_duration_seconds	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name, stream, status, outcome	is_fallback, env_id	—
aisix_proxy_client_cancelled_requests_total	endpoint, model, provider_key_id, provider_key_name	env_id	—
aisix_proxy_request_body_limit_rejections_total	endpoint, inbound_protocol, outcome	env_id	—
aisix_deployment_requests_total	provider, model, upstream_model, provider_key_id	env_id	—
aisix_deployment_success_responses_total	provider, model, upstream_model, provider_key_id	env_id	—
aisix_deployment_failure_responses_total	provider, model, upstream_model, provider_key_id	env_id	—
aisix_deployment_state	provider, model, upstream_model, provider_key_id	env_id	provider, model, upstream_model, provider_key_id
aisix_deployment_cooled_down_total	provider, model, upstream_model, provider_key_id	env_id	—
aisix_routing_successful_fallbacks_total	model, fallback_model	env_id	—
aisix_routing_failed_fallbacks_total	model, fallback_model	env_id	—
aisix_ratelimit_remaining_requests	api_key_id, model	env_id	api_key_id, model
aisix_ratelimit_remaining_tokens	api_key_id, model	env_id	api_key_id, model
aisix_budget_limit_usd	api_key_id, team_id, user_id, user_name	env_id	api_key_id, team_id, user_id, user_name
aisix_budget_spent_usd	api_key_id, team_id, user_id, user_name	env_id	api_key_id, team_id, user_id, user_name
aisix_budget_remaining_usd	api_key_id, team_id, user_id, user_name	env_id	api_key_id, team_id, user_id, user_name
aisix_budget_reset_seconds	api_key_id, team_id, user_id, user_name	env_id	api_key_id, team_id, user_id, user_name
aisix_budget_details_present	api_key_id, team_id, user_id, user_name	env_id	api_key_id, team_id, user_id, user_name
aisix_redis_failures_total	operation	env_id	—
aisix_usage_event_drops_total	reason, model, provider_key_id, provider_key_name, user_id, user_name, upstream_protocol	env_id	—
aisix_usage_events_rejected_total	—	env_id	—
aisix_guardrail_blocks_total	—	env_id	—
aisix_guardrail_bypasses_total	reason	env_id	—
aisix_auth_decisions_total	method, result, reason	env_id	—
aisix_guardrail_latency_seconds	env_id, guardrail, kind, phase, result, error_type	—	—
aisix_usage_events_emitted_total	handler, status_code, status, inbound_protocol, upstream_protocol, model, provider_key_id, provider_key_name, user_id, user_name	env_id	—
aisix_cache_requests_total	policy, outcome	env_id	—
aisix_cache_semantic_embedding_seconds	policy	env_id	—
aisix_cache_semantic_embedding_failures_total	policy, cause	env_id	—
aisix_cache_semantic_store_failures_total	policy, op	env_id	—
aisix_otlp_fanout_drops_total	exporter, reason	env_id	—
aisix_otlp_fanout_failures_total	exporter	env_id	—
aisix_request_e2e_latency_seconds	side (always emitted), env_id, endpoint, model, provider, status_class, streaming	inbound_protocol, upstream_protocol, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	—
aisix_request_ttft_seconds	side (always emitted), env_id, endpoint, model, provider, status_class, streaming	inbound_protocol, upstream_protocol, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	—
aisix_a2a_requests_total	agent, operation, status	env_id	—
aisix_a2a_ttfb_seconds	agent, operation	env_id	—
aisix_a2a_stream_events_total	agent, operation	env_id	—
aisix_a2a_task_state_total	agent, state	env_id	—
aisix_config_last_reload_successful	—	env_id	—
aisix_config_last_reload_success_timestamp_seconds	—	env_id	—
aisix_config_reloads_total	—	env_id	—
aisix_config_reload_failures_total	reason	env_id	—
aisix_config_rejected_resources	kind	env_id	kind
aisix_config_partially_compatible_resources	kind	env_id	kind
aisix_config_stale_served_resources	kind	env_id	kind
aisix_config_unknown_kind_resources	kind	env_id	kind
aisix_config_observed_revision	—	env_id	—
aisix_config_applied_revision	—	env_id	—
aisix_config_hash_info	hash	env_id	hash
aisix_config_source_connected	—	env_id	—