Metric Labels and Variables

Configure observability.metrics.labels to select the complete label list for each Prometheus metric family. This supports adding available variables and removing default labels. Gateways running 1.1.0 or earlier do not support this option.

Configure Labels

config.yaml

observability:
  metrics:
    labels:
      aisix_request_ttft_seconds:
        - env_id
        - endpoint
        - model
        - provider
        - status_class
        - streaming
        - provider_key_name
      aisix_proxy_requests_total:
        - endpoint
        - status
      aisix_requests_total: []

This example adds the selected provider credential's name to TTFT. It keeps only endpoint and status on the detailed request counter. It removes all business labels from the legacy request counter.

• An omitted metric keeps its existing defaults. labels: {} preserves the entire default metric surface.
• Each configured list replaces that metric's default list. An empty list removes all business labels where no identity labels are required.
• Use the metric family name, such as aisix_request_ttft_seconds, without _bucket, _sum, or _count. Histogram components always share the selected labels. Prometheus still adds le; summaries still add quantile.
• Unknown metric names, unsupported variables, duplicate variables, and removal of required identity labels cause startup to fail with a configuration error.
• Labels are fixed at startup. Restart the gateway after changing the selection. Existing stored series remain in Prometheus according to its retention policy; update queries and alerts that use removed labels.

For deployments configured through environment variables, set the entire map as one JSON object:

export AISIX_OBSERVABILITY__METRICS__LABELS='{"aisix_request_ttft_seconds":["model","provider_key_name"]}'

In AISIX Cloud, open Data Planes → Metric labels in the installation card to select metrics and labels. The generated Docker, Compose, Helm, and systemd instructions include this startup setting. Apply the generated configuration to a gateway version that supports it.

Aggregation and Cardinality

Removing labels merges counter observations and histogram distributions before they are accumulated. For example, removing model combines requests across models; it does not select one model's last sample. The configuration does not change which requests or attempts a metric counts.

Gauge values describe current state, such as one credential's remaining quota or one deployment's health. The identity labels listed below must remain, so independent objects cannot overwrite one another. Optional gauge labels can still be added or removed.

Each distinct combination of selected label values creates a time series. Credential names, API key IDs, and member identities can multiply series counts. A classic histogram creates a series for each finite bucket, the +Inf bucket, _sum, and _count for every combination. Select only dimensions needed by your queries.

Removing a categorical label also combines its categories. In particular, token_type="total" already includes input and output; removing token_type sums all three categories and must not be interpreted as the original total.

Supported Variables

Variable names are also the emitted label names. Availability depends on the metric: the next table lists its defaults and additional supported variables. Missing request attribution uses unknown; labels that already define a different missing-value category keep that category. Variables read from request context are unavailable on background metrics that have no such context. Arbitrary request headers and expressions are not supported.

Variable	Meaning
env_id	The gateway's AISIX Cloud environment ID; unknown when not connected to AISIX Cloud. Available on every metric.
endpoint	The matched route template, without caller-supplied path parameters.
inbound_protocol	The protocol used by the caller, derived from the matched endpoint.
upstream_protocol	The wire protocol of the selected provider credential; unknown before upstream selection or for non-LLM traffic.
provider	The provider kind attributed to this observation; ensemble when no single provider owns the result.
model	The configured gateway model identity attributed to the observation. Wildcard requests use the configured model pattern. See the metric reference for its request or attempt scope.
upstream_model	The upstream model identity, bounded to the configured model pattern for wildcard models.
provider_key_id	The identifier of the selected provider credential. This is an identifier, never its secret.
provider_key_name	The display name of that same provider credential. Renaming a credential starts a new series when this variable is selected.
api_key_id	The identifier of the authenticating gateway API key, never its plaintext value.
team_id	The team associated with the authenticating API key.
user_id	The member associated with the authenticating API key.
user_name	The member display name carried by the API key's configuration snapshot.
stream	Whether the request asked for streaming, encoded as true or false. Used by the detailed request metrics.
streaming	Whether the observed request is streaming, encoded as true or false. Used by the request latency histograms.
is_fallback	Whether request attribution identifies a fallback attempt, encoded as true or false.
status	The HTTP status code on request and usage-event metrics; the HTTP status class on A2A request metrics.
status_class	The HTTP status class: 2xx, 3xx, 4xx, 5xx, or other.
status_code	The HTTP status class on usage-event metrics: 2xx, 3xx, 4xx, 5xx, or other. The status variable retains the exact code.
outcome	The outcome defined by the metric, such as the request result, cache decision, or request-body-limit result.
client_type	The classified client name from built-in or configured client_type_rules; never the raw User-Agent.
token_type	The token count category: input, output, or total. Total already includes the input and output categories.
fallback_model	The configured fallback model attributed to a routing fallback.
scope	The scope of the rate-limit decision.
layer	The layer of the rate-limit decision.
policy_id	The rate-limit policy identifier, or the metric's missing-policy value.
reason	The bounded reason code defined by the emitting authentication, guardrail, configuration, or exporter metric.
method	The authentication method used for the credential decision.
result	The authentication or guardrail execution result, as defined by the metric.
guardrail	The configured guardrail name.
kind	The guardrail kind on guardrail metrics, or the resource kind on configuration metrics.
phase	The guardrail execution phase.
error_type	The bounded guardrail failure category, or none when no error occurred.
handler	The usage event's handler family, such as chat, messages, embeddings, or mcp.
policy	The configured cache policy name.
cause	The bounded semantic-cache embedding failure cause.
op	The semantic-cache storage operation.
operation	The Redis operation or A2A operation, as defined by the metric.
exporter	The configured observability exporter name.
agent	The registered A2A agent name.
state	The A2A task state reported by the upstream agent.
hash	The hash of the applied gateway resource configuration.

Variables Available per Metric

env_id is available on every metric. All default labels are selectable variables. The additional column lists variables that can be added beyond those defaults; a dash means none. Required identities must be included in a replacement list.

Metric family	Default labels	Additional variables	Required identities
aisix_requests_total	provider, model, status, outcome	env_id	—
aisix_request_duration_seconds	provider, model, status	env_id	—
aisix_ratelimit_rejections_total	scope, layer, policy_id	env_id	—
aisix_tokens_consumed_total	provider, model	env_id	—
aisix_llm_spend_micro_usd_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	env_id	—
aisix_llm_input_tokens_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	env_id	—
aisix_llm_output_tokens_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	env_id	—
aisix_llm_total_tokens_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	env_id	—
aisix_llm_cached_input_tokens_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	env_id	—
aisix_llm_cache_read_input_tokens_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	env_id	—
aisix_llm_cache_creation_input_tokens_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	env_id	—
aisix_llm_requests_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name, stream, is_fallback, status, outcome	env_id	—
aisix_llm_request_duration_seconds	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name, stream, status, outcome	is_fallback, env_id	—
aisix_llm_time_to_first_token_seconds	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	env_id	—
aisix_llm_tokens_by_client_total	client_type, model, token_type	env_id	—
aisix_proxy_in_flight_requests	endpoint, inbound_protocol	env_id	endpoint
aisix_proxy_requests_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name, stream, is_fallback, status, outcome	env_id	—
aisix_proxy_failed_requests_total	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name, stream, is_fallback, status, outcome	env_id	—
aisix_proxy_request_duration_seconds	endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name, stream, status, outcome	is_fallback, env_id	—
aisix_proxy_client_cancelled_requests_total	endpoint, model, provider_key_id, provider_key_name	env_id	—
aisix_proxy_request_body_limit_rejections_total	endpoint, inbound_protocol, outcome	env_id	—
aisix_deployment_requests_total	provider, model, upstream_model, provider_key_id	env_id	—
aisix_deployment_success_responses_total	provider, model, upstream_model, provider_key_id	env_id	—
aisix_deployment_failure_responses_total	provider, model, upstream_model, provider_key_id	env_id	—
aisix_deployment_state	provider, model, upstream_model, provider_key_id	env_id	provider, model, upstream_model, provider_key_id
aisix_deployment_cooled_down_total	provider, model, upstream_model, provider_key_id	env_id	—
aisix_routing_successful_fallbacks_total	model, fallback_model	env_id	—
aisix_routing_failed_fallbacks_total	model, fallback_model	env_id	—
aisix_ratelimit_remaining_requests	api_key_id, model	env_id	api_key_id, model
aisix_ratelimit_remaining_tokens	api_key_id, model	env_id	api_key_id, model
aisix_budget_limit_usd	api_key_id, team_id, user_id, user_name	env_id	api_key_id, team_id, user_id, user_name
aisix_budget_spent_usd	api_key_id, team_id, user_id, user_name	env_id	api_key_id, team_id, user_id, user_name
aisix_budget_remaining_usd	api_key_id, team_id, user_id, user_name	env_id	api_key_id, team_id, user_id, user_name
aisix_budget_reset_seconds	api_key_id, team_id, user_id, user_name	env_id	api_key_id, team_id, user_id, user_name
aisix_budget_details_present	api_key_id, team_id, user_id, user_name	env_id	api_key_id, team_id, user_id, user_name
aisix_redis_failures_total	operation	env_id	—
aisix_usage_event_drops_total	reason, model, provider_key_id, provider_key_name, user_id, user_name, upstream_protocol	env_id	—
aisix_guardrail_blocks_total	—	env_id	—
aisix_guardrail_bypasses_total	reason	env_id	—
aisix_auth_decisions_total	method, result, reason	env_id	—
aisix_guardrail_latency_seconds	env_id, guardrail, kind, phase, result, error_type	—	—
aisix_usage_events_emitted_total	handler, status_code, status, inbound_protocol, upstream_protocol, model, provider_key_id, provider_key_name, user_id, user_name	env_id	—
aisix_cache_requests_total	policy, outcome	env_id	—
aisix_cache_semantic_embedding_seconds	policy	env_id	—
aisix_cache_semantic_embedding_failures_total	policy, cause	env_id	—
aisix_cache_semantic_store_failures_total	policy, op	env_id	—
aisix_otlp_fanout_drops_total	exporter, reason	env_id	—
aisix_otlp_fanout_failures_total	exporter	env_id	—
aisix_request_e2e_latency_seconds	env_id, endpoint, model, provider, status_class, streaming	inbound_protocol, upstream_protocol, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	—
aisix_request_ttft_seconds	env_id, endpoint, model, provider, status_class, streaming	inbound_protocol, upstream_protocol, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name	—
aisix_a2a_requests_total	agent, operation, status	env_id	—
aisix_a2a_ttfb_seconds	agent, operation	env_id	—
aisix_a2a_stream_events_total	agent, operation	env_id	—
aisix_a2a_task_state_total	agent, state	env_id	—
aisix_config_last_reload_successful	—	env_id	—
aisix_config_last_reload_success_timestamp_seconds	—	env_id	—
aisix_config_reloads_total	—	env_id	—
aisix_config_reload_failures_total	reason	env_id	—
aisix_config_rejected_resources	kind	env_id	kind
aisix_config_partially_compatible_resources	kind	env_id	kind
aisix_config_stale_served_resources	kind	env_id	kind
aisix_config_observed_revision	—	env_id	—
aisix_config_applied_revision	—	env_id	—
aisix_config_hash_info	hash	env_id	hash
aisix_config_source_connected	—	env_id	—