Metric Labels and Variables
Configure observability.metrics.labels to select the complete label list for each Prometheus metric family. This supports adding available variables and removing default labels. Gateways running 1.1.0 or earlier do not support this option.
Configure Labels
observability:
metrics:
labels:
aisix_request_ttft_seconds:
- env_id
- endpoint
- model
- provider
- status_class
- streaming
- provider_key_name
aisix_proxy_requests_total:
- endpoint
- status
aisix_requests_total: []
This example adds the selected provider credential's name to TTFT. It keeps only endpoint and status on the detailed request counter. It removes all business labels from the legacy request counter.
- An omitted metric keeps its existing defaults.
labels: {}preserves the entire default metric surface. - Each configured list replaces that metric's default list. An empty list removes all business labels where no identity labels are required.
- Use the metric family name, such as
aisix_request_ttft_seconds, without_bucket,_sum, or_count. Histogram components always share the selected labels. Prometheus still addsle; summaries still addquantile. sideonaisix_request_ttft_seconds,aisix_llm_time_to_first_token_seconds, andaisix_request_e2e_latency_secondsis always emitted. A replacement list cannot remove it, because its two values are different measurements that would otherwise merge into one series. Listing it is accepted.- Unknown metric names, unsupported variables, duplicate variables, and removal of required identity labels cause startup to fail with a configuration error.
- Labels are fixed at startup. Restart the gateway after changing the selection. Existing stored series remain in Prometheus according to its retention policy; update queries and alerts that use removed labels.
For deployments configured through environment variables, set the entire map as one JSON object:
export AISIX_OBSERVABILITY__METRICS__LABELS='{"aisix_request_ttft_seconds":["model","provider_key_name"]}'
For a gateway deployed with the AISIX Helm chart, set the same map under config.observability.metrics.labels in your values file:
config:
observability:
metrics:
labels:
aisix_request_ttft_seconds: [model, provider_key_name]
Label selection is part of a gateway's own configuration, so it is applied by whoever operates the gateway. AISIX Cloud generates the basic deployment template only; add observability.metrics.labels to the gateway's configuration file, the Helm values, or the environment variable above, on each gateway that needs it. Apply it only to a gateway version that supports the option.
Name Labels
Identifier labels such as api_key_id and team_id are stable but hard to read on a dashboard. Each of them has a matching name label that carries the display name of the same object, so a query can group or filter by a readable name without looking the ID up elsewhere. Gateways running 1.5.0 or earlier support none of the selections in the table below.
| Name label | Display name of | Metric families that accept it |
|---|---|---|
api_key_name | The authenticating API key. | Every family that carries api_key_id: the detailed request counters, the request duration summaries, aisix_llm_time_to_first_token_seconds, the token, cache-token, and spend counters, the budget gauges, aisix_request_e2e_latency_seconds, aisix_request_ttft_seconds, and both aisix_ratelimit_remaining_* gauges. |
team_name | The API key's team. | The same families, except the two aisix_ratelimit_remaining_* gauges, which carry no team. |
provider_key_name | The selected provider key. | aisix_deployment_requests_total, aisix_deployment_success_responses_total, aisix_deployment_failure_responses_total, and aisix_deployment_cooled_down_total, in addition to the families that already list it as a default. |
policy_name | The rate-limit policy. | aisix_ratelimit_rejections_total. |
aisix_deployment_state does not accept provider_key_name. That gauge is written only when a target's health changes, so a renamed provider key would keep reporting its old name there until the next transition. There is no environment name label: one gateway serves one environment, so env_id is constant per gateway, and a Prometheus target label covers it. Variables Available per Metric lists the exact labels each family accepts.
Enable Name Labels
api_key_name, team_name, and policy_name are never default labels, and neither is provider_key_name on the deployment counters. user_name, and provider_key_name on the families that already list it, remain default labels. To add a name label, list it in the family's entry in observability.metrics.labels. Each entry is the family's complete label list, so repeat the defaults you want to keep:
observability:
metrics:
labels:
aisix_llm_spend_micro_usd_total: [endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, api_key_name, team_id, team_name, user_id, user_name]
aisix_ratelimit_rejections_total: [scope, layer, policy_id, policy_name]
aisix_deployment_failure_responses_total: [provider, model, upstream_model, provider_key_id, provider_key_name]
Families you do not list keep their defaults. The gateway validates the selection at startup and refuses to start on an unknown family or an unsupported label, and a change takes effect only after a restart.
Set the same selection through an environment variable as one JSON object:
export AISIX_OBSERVABILITY__METRICS__LABELS='{"aisix_ratelimit_rejections_total":["scope","layer","policy_id","policy_name"]}'
Or through the Helm chart:
config:
observability:
metrics:
labels:
aisix_ratelimit_rejections_total: [scope, layer, policy_id, policy_name]
Name Values
- A name that cannot be resolved reports
unknown: an API key without a display name, or a team whose name the gateway has not received. policy_namefollowspolicy_id. Both are empty when the rejection did not come from a rate-limit policy.- A name is one-to-one with its ID at any moment, but a rename starts a new series under the new name. The old series stops receiving samples; counters and summaries keep it until Prometheus ages it out. The budget gauges and the
aisix_ratelimit_remaining_*gauges retire the old series the same way they do when an API key is deleted.
Name Sources
With AISIX Cloud, the control plane projects every team's and member's name to the gateway, alongside the rest of its configuration. Renaming or deleting a team or member reaches the gateway like any other configuration change. user_name is read from the member's own name document, so a member rename is reflected without rewriting the member's API keys. api_key_name and provider_key_name come from the API key's and provider key's own display names.
In standalone mode, the resources file and the Admin API have no team or member name documents:
team_namealways reportsunknown. To make team series readable, use a human-readableteam_idon your API keys instead.user_namecomes from the API key's ownuser_namefield.
aisix export warns when the configuration it exports contains team or member name documents, because the resources file it writes cannot carry them.
Cardinality and Privacy
Name labels are opt-in for two reasons. A name label adds one more label to every series of the families it is added to. Where the matching ID is not selected, as on the latency histograms by default, it splits each series by every distinct name, and every rename starts another series. Name labels also publish display names, such as a team or API key name, to anyone who can scrape /metrics. Add them only to the families your dashboards need, and only where everyone with access to the metrics endpoint may see those names.
Aggregation and Cardinality
Removing labels merges counter observations and histogram distributions before they are accumulated. For example, removing model combines requests across models; it does not select one model's last sample. The configuration does not change which requests or attempts a metric counts.
Gauge values describe current state, such as one credential's remaining quota or one deployment's health. The identity labels listed below must remain, so independent objects cannot overwrite one another. Optional gauge labels can still be added or removed.
Each distinct combination of selected label values creates a time series. Credential names, API key IDs, and member identities can multiply series counts. A classic histogram creates a series for each finite bucket, the +Inf bucket, _sum, and _count for every combination. Select only dimensions needed by your queries.
Removing a categorical label also combines its categories. In particular, token_type="total" already includes input and output; removing token_type sums all three categories and must not be interpreted as the original total.
Large Deployments
Default labels stay enabled unless you configure a replacement. For deployments with many API keys or members, select dimensions from the queries and alerts you need. Use request logs or exported usage events for individual-call investigations when per-member Prometheus series are unnecessary.
Start with these choices:
- Keep
model,provider, protocol, status and outcome dimensions needed by service-level queries. Keepis_fallbackwhen distinguishing recovered requests. Keep category labels such astoken_type; removing them changes what a sum means. - Remove
api_key_id,user_idanduser_namefrom request, duration, token and usage-event families when you do not query individual callers. Keepteam_idonly on families where team-level monitoring is needed. - Prefer
provider_key_idto a credential display name when monitoring credential health. A rename changes a name-based series. Addenv_idif your deployment and queries need that dimension. - Keep the required identities of budget, remaining-quota, deployment-state and other gauges. These are independent current values, so dropping their identities would allow one object's value to overwrite another. The gateway rejects such configurations.
The following example groups detailed traffic metrics by model and provider credential, removing caller identities and display names. Add these entries to your existing observability.metrics.labels map; preserve settings for other families. Each line replaces the named family's complete label list, so adapt it to your dashboards before restarting the gateway.
observability:
metrics:
labels:
aisix_proxy_requests_total: [endpoint, upstream_protocol, provider, model, provider_key_id, inbound_protocol, stream, is_fallback, status, outcome]
aisix_proxy_failed_requests_total: [endpoint, upstream_protocol, provider, model, provider_key_id, inbound_protocol, stream, is_fallback, status, outcome]
aisix_llm_requests_total: [endpoint, upstream_protocol, provider, model, provider_key_id, inbound_protocol, stream, is_fallback, status, outcome]
aisix_proxy_request_duration_seconds: [endpoint, upstream_protocol, provider, model, provider_key_id, inbound_protocol, stream, status, outcome]
aisix_llm_request_duration_seconds: [endpoint, upstream_protocol, provider, model, provider_key_id, inbound_protocol, stream, status, outcome]
aisix_llm_time_to_first_token_seconds: [endpoint, upstream_protocol, provider, model, provider_key_id]
aisix_llm_spend_micro_usd_total: [endpoint, upstream_protocol, provider, model, provider_key_id]
aisix_llm_input_tokens_total: [endpoint, upstream_protocol, provider, model, provider_key_id]
aisix_llm_output_tokens_total: [endpoint, upstream_protocol, provider, model, provider_key_id]
aisix_llm_total_tokens_total: [endpoint, upstream_protocol, provider, model, provider_key_id]
aisix_llm_cached_input_tokens_total: [endpoint, upstream_protocol, provider, model, provider_key_id]
aisix_llm_cache_read_input_tokens_total: [endpoint, upstream_protocol, provider, model, provider_key_id]
aisix_llm_cache_creation_input_tokens_total: [endpoint, upstream_protocol, provider, model, provider_key_id]
aisix_usage_events_emitted_total: [handler, status_code, status, inbound_protocol, upstream_protocol, model, provider_key_id]
aisix_usage_event_drops_total: [reason, model, provider_key_id, upstream_protocol]
This example leaves the SLO latency histograms at their defaults and keeps budget and quota gauges unchanged. Their per-object series still contribute to total size. Metrics not listed retain all their default labels, so review the other families in the table below against your workload.
Size for observed label combinations, not only configured objects or current request rate. Once many distinct callers have sent requests, their cumulative series remain even while business traffic is quiet. Each default duration summary exports seven quantiles plus sum and count: two such families with 50,000 observed combinations already produce 900,000 sample lines, before counters and other metrics. A histogram with N finite bucket boundaries exports N + 3 series per combination. Reducing labels before recording reduces gateway storage, maintenance, serialization and transfer costs. Prometheus-side metric_relabel_configs runs after the gateway has generated the response and does not remove that work; dropping labels there also does not aggregate samples.
Scrape each gateway instance directly, rather than putting a load-balanced service in front of several independent counters. Start with a 30-second interval and a 10-second timeout, then measure at expected peak cardinality, including the first scrape after warm-up. Keep the timeout shorter than the interval and choose the interval according to alert freshness requirements. Raising a timeout alone does not reduce series count or CPU use.
Check scrape_duration_seconds, scrape_samples_scraped, up, gateway CPU, RSS and request latency together. For example, find the largest ingested AISIX families with:
topk(10, count by (__name__) ({job="aisix", __name__=~"aisix_.+"}))
Replace job="aisix" with your scrape job. After changing labels, verify the new scrape, update affected dashboards and alerts, and allow old Prometheus series to age out under the configured retention policy.
Supported Variables
Variable names are also the emitted label names. Availability depends on the metric: the next table lists its defaults and additional supported variables. Missing request attribution uses unknown; labels that already define a different missing-value category keep that category. Variables read from request context are unavailable on background metrics that have no such context. Arbitrary request headers and expressions are not supported.
| Variable | Meaning |
|---|---|
env_id | The gateway's AISIX Cloud environment ID; unknown when not connected to AISIX Cloud. Available on every metric. |
trigger | What caused a configuration apply: watch for a coalesced batch of watch events, full for a read of every watched prefix. On heap-profile dumps: auto for a memory threshold, manual for a request to the debug listener. |
endpoint | The matched route template, without caller-supplied path parameters. |
inbound_protocol | The protocol used by the caller, derived from the matched endpoint. |
upstream_protocol | The wire protocol of the selected provider credential; unknown before upstream selection or for non-LLM traffic. |
provider | The provider kind attributed to this observation; ensemble when no single provider owns the result. |
model | The configured gateway model identity attributed to the observation. Wildcard requests use the configured model pattern. See the metric reference for its request or attempt scope. |
upstream_model | The upstream model identity, bounded to the configured model pattern for wildcard models. |
provider_key_id | The identifier of the selected provider credential. This is an identifier, never its secret. |
provider_key_name | The display name of that same provider credential. Renaming a credential starts a new series when this variable is selected. |
api_key_id | The identifier of the authenticating gateway API key, never its plaintext value. |
api_key_name | The display name of that same API key; unknown when it has none. Renaming the key starts a new series when this variable is selected. Not supported by 1.5.0 or earlier. |
team_id | The team associated with the authenticating API key. |
team_name | The display name of that same team, read from its team name document; unknown when no document names it. Renaming the team starts a new series when this variable is selected. Not supported by 1.5.0 or earlier. |
user_id | The member associated with the authenticating API key. |
user_name | The display name of that same member, read from its member name document, or from the API key's own user_name when no such document exists; unknown when neither names the member. On 1.5.0 or earlier, always the name carried by the API key. |
stream | Whether the request asked for streaming, encoded as true or false. Used by the detailed request metrics. |
streaming | Whether the observed request is streaming, encoded as true or false. Used by the request latency histograms. |
is_fallback | Whether request attribution identifies a fallback attempt, encoded as true or false. |
status | The HTTP status code on request and usage-event metrics; the HTTP status class on A2A request metrics. |
status_class | The HTTP status class: 2xx, 3xx, 4xx, 5xx, or other. |
side | Which interval a latency observation measures. On the two TTFT metrics: upstream, from the start of the attempt that served the request to the first upstream frame, or downstream, from the gateway receiving the request to the first frame written to the client. On aisix_request_e2e_latency_seconds: upstream, the upstream duration of the attempt that produced the response, or downstream, the whole request as the client experienced it. Always emitted on the metrics that carry it and cannot be removed by a label selection. Gateways running 1.4.0 or earlier emit no side label on the TTFT metrics, and gateways running 1.4.0 or earlier emit none on aisix_request_e2e_latency_seconds. |
status_code | The HTTP status class on usage-event metrics: 2xx, 3xx, 4xx, 5xx, or other. The status variable retains the exact code. |
outcome | The outcome defined by the metric, such as the request result, cache decision, or request-body-limit result. |
client_type | The classified client name from built-in or configured client_type_rules; never the raw User-Agent. |
token_type | The token count category: input, output, or total. Total already includes the input and output categories. |
fallback_model | The configured fallback model attributed to a routing fallback. |
scope | The scope of the rate-limit decision. |
layer | The layer of the rate-limit decision. |
policy_id | The rate-limit policy identifier, or the metric's missing-policy value. |
policy_name | The name of that same rate-limit policy, or the same missing-policy value as policy_id. Not supported by 1.5.0 or earlier. |
reason | The bounded reason code defined by the emitting authentication, guardrail, configuration, or exporter metric. |
method | The authentication method used for the credential decision. |
result | The authentication, guardrail execution, or heap-profile dump result, as defined by the metric. |
guardrail | The configured guardrail name. |
kind | The guardrail kind on guardrail metrics, or the resource kind on configuration metrics. |
phase | The guardrail execution phase. |
error_type | The bounded guardrail failure category, or none when no error occurred. |
handler | The usage event's handler family, such as chat, messages, embeddings, or mcp. |
policy | The configured cache policy name. |
cause | The bounded semantic-cache embedding failure cause. |
op | The semantic-cache storage operation. |
operation | The Redis operation or A2A operation, as defined by the metric. |
exporter | The configured observability exporter name. Empty on component metrics other than the exporter queue. |
agent | The registered A2A agent name. |
state | The A2A task state reported by the upstream agent. |
hash | The hash of the applied gateway resource configuration. |
stat | The allocator statistic: allocated, active, resident, mapped, retained, or metadata. |
runtime | The async runtime: control, or tpc-N for thread-per-core proxy worker N. |
component | The in-process store being measured, from a fixed set such as response_cache or log_queue. |
Variables Available per Metric
env_id is available on every metric. All default labels are selectable variables. The additional column lists variables that can be added beyond those defaults; a dash means none. Required identities must be included in a replacement list.
| Metric family | Default labels | Additional variables | Required identities |
|---|---|---|---|
aisix_requests_total | provider, model, status, outcome | env_id | — |
aisix_request_duration_seconds | provider, model, status | env_id | — |
aisix_ratelimit_rejections_total | scope, layer, policy_id | policy_name, env_id | — |
aisix_tokens_consumed_total | provider, model | env_id | — |
aisix_llm_spend_micro_usd_total | endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name | api_key_name, team_name, env_id | — |
aisix_llm_input_tokens_total | endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name | api_key_name, team_name, env_id | — |
aisix_llm_output_tokens_total | endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name | api_key_name, team_name, env_id | — |
aisix_llm_total_tokens_total | endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name | api_key_name, team_name, env_id | — |
aisix_llm_cached_input_tokens_total | endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name | api_key_name, team_name, env_id | — |
aisix_llm_cache_read_input_tokens_total | endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name | api_key_name, team_name, env_id | — |
aisix_llm_cache_creation_input_tokens_total | endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name | api_key_name, team_name, env_id | — |
aisix_llm_requests_total | endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name, stream, is_fallback, status, outcome | api_key_name, team_name, env_id | — |
aisix_llm_request_duration_seconds | endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name, stream, status, outcome | is_fallback, api_key_name, team_name, env_id | — |
aisix_llm_time_to_first_token_seconds | side (always emitted), endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name | api_key_name, team_name, env_id | — |
aisix_llm_tokens_by_client_total | client_type, model, token_type | env_id | — |
aisix_proxy_in_flight_requests | endpoint, inbound_protocol | env_id | endpoint |
aisix_proxy_requests_total | endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name, stream, is_fallback, status, outcome | api_key_name, team_name, env_id | — |
aisix_proxy_failed_requests_total | endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name, stream, is_fallback, status, outcome | api_key_name, team_name, env_id | — |
aisix_proxy_request_duration_seconds | endpoint, inbound_protocol, upstream_protocol, provider, model, upstream_model, provider_key_id, provider_key_name, api_key_id, team_id, user_id, user_name, stream, status, outcome | is_fallback, api_key_name, team_name, env_id | — |
aisix_proxy_client_cancelled_requests_total | endpoint, model, provider_key_id, provider_key_name | env_id | — |
aisix_proxy_request_body_limit_rejections_total | endpoint, inbound_protocol, outcome | env_id | — |
aisix_deployment_requests_total | provider, model, upstream_model, provider_key_id | provider_key_name, env_id | — |
aisix_deployment_success_responses_total | provider, model, upstream_model, provider_key_id | provider_key_name, env_id | — |
aisix_deployment_failure_responses_total | provider, model, upstream_model, provider_key_id | provider_key_name, env_id | — |
aisix_deployment_state | provider, model, upstream_model, provider_key_id | env_id | provider, model, upstream_model, provider_key_id |
aisix_deployment_cooled_down_total | provider, model, upstream_model, provider_key_id | provider_key_name, env_id | — |
aisix_routing_successful_fallbacks_total | model, fallback_model | env_id | — |
aisix_routing_failed_fallbacks_total | model, fallback_model | env_id | — |
aisix_ratelimit_remaining_requests | api_key_id, model | api_key_name, env_id | api_key_id, model |
aisix_ratelimit_remaining_tokens | api_key_id, model | api_key_name, env_id | api_key_id, model |
aisix_budget_limit_usd | api_key_id, team_id, user_id, user_name | api_key_name, team_name, env_id | api_key_id, team_id, user_id, user_name |
aisix_budget_spent_usd | api_key_id, team_id, user_id, user_name | api_key_name, team_name, env_id | api_key_id, team_id, user_id, user_name |
aisix_budget_remaining_usd | api_key_id, team_id, user_id, user_name | api_key_name, team_name, env_id | api_key_id, team_id, user_id, user_name |
aisix_budget_reset_seconds | api_key_id, team_id, user_id, user_name | api_key_name, team_name, env_id | api_key_id, team_id, user_id, user_name |
aisix_budget_details_present | api_key_id, team_id, user_id, user_name | api_key_name, team_name, env_id | api_key_id, team_id, user_id, user_name |
aisix_redis_failures_total | operation | env_id | — |
aisix_usage_event_drops_total | reason, model, provider_key_id, provider_key_name, user_id, user_name, upstream_protocol | env_id | — |
aisix_usage_events_rejected_total | — | env_id | — |
aisix_guardrail_blocks_total | — | env_id | — |
aisix_guardrail_bypasses_total | reason | env_id | — |
aisix_auth_decisions_total | method, result, reason | env_id | — |
aisix_guardrail_latency_seconds | env_id, guardrail, kind, phase, result, error_type | — | — |
aisix_usage_events_emitted_total | handler, status_code, status, inbound_protocol, upstream_protocol, model, provider_key_id, provider_key_name, user_id, user_name | env_id | — |
aisix_cache_requests_total | policy, outcome | env_id | — |
aisix_cache_semantic_embedding_seconds | policy | env_id | — |
aisix_cache_semantic_embedding_failures_total | policy, cause | env_id | — |
aisix_cache_semantic_store_failures_total | policy, op | env_id | — |
aisix_otlp_fanout_drops_total | exporter, reason | env_id | — |
aisix_otlp_fanout_failures_total | exporter | env_id | — |
aisix_request_e2e_latency_seconds | side (always emitted), env_id, endpoint, model, provider, status_class, streaming | inbound_protocol, upstream_protocol, upstream_model, provider_key_id, provider_key_name, api_key_id, api_key_name, team_id, team_name, user_id, user_name | — |
aisix_request_ttft_seconds | side (always emitted), env_id, endpoint, model, provider, status_class, streaming | inbound_protocol, upstream_protocol, upstream_model, provider_key_id, provider_key_name, api_key_id, api_key_name, team_id, team_name, user_id, user_name | — |
aisix_a2a_requests_total | agent, operation, status | env_id |