# Metrics and Usage Events An AISIX gateway exposes aggregate metrics and exportable usage events. Together, these signals show service health, traffic trends, and the model, route, and policy outcome behind each request attempt. ## Choose a Telemetry Source[​](#choose-a-telemetry-source "Direct link to Choose a Telemetry Source") Start with the source that matches the operational question, then correlate signals by request, model, or provider when an issue needs deeper investigation. | Operational Question | Start With | What It Provides | | ---------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | | Is traffic healthy across gateway instances? | Prometheus metrics | Request rates, latency distributions, token counters, budget gauges, policy outcomes, routing health, cache behavior, and exporter delivery health. | | What happened to one request? | [Access logs](https://docs.api7.ai/ai-gateway/observability/access-logs-and-request-correlation.md#collect-access-logs) | Structured request fields, including status, latency, model, provider, request ID, and routing outcome when available. | | What can the calling application observe? | [Response headers](https://docs.api7.ai/ai-gateway/reference/headers-and-error-codes.md#proxy-response-headers) | Request correlation, cache outcome, retry timing, and selected-target hints on supported routes. | | Where can request records be stored, analyzed, or used for accounting? | Usage events | Per-attempt outcome and consumption records delivered through an observability exporter. | ## Scrape Prometheus Metrics[​](#scrape-prometheus-metrics "Direct link to Scrape Prometheus Metrics") AISIX serves Prometheus metrics on the dedicated metrics listener at `/metrics` by default. Change the path or disable the endpoint through the startup observability settings. This endpoint is unauthenticated by design. Keep the dedicated metrics listener private. Configure Prometheus exposure in the startup configuration: config.yaml ``` observability: metrics: prometheus: enabled: true path: "/metrics" ``` The dedicated listener binds to `0.0.0.0:9090` by default. Set a different listener address when Prometheus should scrape another interface or port. Scrape the default metrics endpoint: ``` curl -sS "http://127.0.0.1:9090/metrics" ``` Traffic metrics appear after activity AISIX publishes configuration status on every scrape. Other metric families are registered on first observation, so traffic metrics might not appear immediately after startup. Send one model request, then check again for series such as `aisix_requests_total` and `aisix_tokens_consumed_total`. AISIX emits native metric names with the `aisix_` prefix. Use the histogram series for latency percentiles across gateway instances and the request counters for success-rate and routing analysis. For exact metric names, label scope, and PromQL examples, see [Metrics Reference](https://docs.api7.ai/ai-gateway/reference/metrics.md). Select the labels of individual metric families with `observability.metrics.labels`. For configuration examples, defaults, and the complete supported-variable list, see [Metric Labels and Variables](https://docs.api7.ai/ai-gateway/reference/metric-labels.md). ## Count Requests and Attempts Separately[​](#count-requests-and-attempts-separately "Direct link to Count Requests and Attempts Separately") A request is one caller interaction. A retry or failover can produce several usage-event attempts for one request. Some attempts stop at a target-level rate limit or during request assembly before calling the provider, so AISIX counts attempts differently depending on the signal: | Unit | Where it is counted | What one sample or row means | | ---------------- | ------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Request | `aisix_proxy_requests_total`, `aisix_llm_requests_total` | One client request, with the status it ended on. For a streamed response that is the status the stream ended with, which can differ from the `200` the caller received when the stream started. | | Attempt | `aisix_deployment_requests_total` and the deployment families | One upstream call to one target model. | | Emission attempt | `aisix_usage_events_emitted_total` | One attempt to enqueue a usage event, counted before the delivery queue accepts or rejects it. | | Attempt | Usage-event records, and the usage log built from them | One recorded processing attempt, including one stopped before an upstream call. Sibling attempts share `request_id`, are ordered by `attempt_index`, and identify the request's terminal row with `is_final_attempt: true`. | If one target returns `502` and a fallback succeeds, the request counter records only the `200` returned to the caller. Deployment counters and usage events retain the failed attempt. A usage log can therefore contain many more `5xx` attempts than the request metrics show; the two sources are measuring different units. Use these queries for the questions they actually answer: ``` # Requests that ended in a server error — what callers experienced. sum(increase(aisix_proxy_requests_total{status=~"5.."}[24h])) # Requests that ended in a server error even after failing over. sum(increase(aisix_proxy_requests_total{status=~"5..", is_fallback="true"}[24h])) # Upstream attempts that failed, by target — including the ones a fallback # rescued. This is the attempt-level view of a 5xx usage-log row count, # restricted to the routes that call a configured model. sum(increase(aisix_deployment_failure_responses_total[24h])) by (model) # Fallbacks that rescued a request, by group and by the target reached. sum(increase(aisix_routing_successful_fallbacks_total[24h])) by (model, fallback_model) # Usage-event emission attempts, and those the handoff queue rejected. sum(increase(aisix_usage_events_emitted_total{status_code="5xx"}[24h])) sum(increase(aisix_usage_event_drops_total[24h])) # One member's rate-limit rejections. `status` carries the raw HTTP code, # so a single failure mode is addressable without scanning a whole family. sum(increase(aisix_usage_events_emitted_total{user_id="", status="429"}[24h])) # Whose queue handoffs were rejected. The queue-side reasons carry the same # model and provider-key labels as the emission counter, so the difference # holds per model, not only in total. sum(increase(aisix_usage_event_drops_total{reason=~"sink_.*"}[24h])) by (model, provider_key_name) # Usage the gateway recorded but could not deliver to the control plane. These # samples carry the member labels only, so group them by member, not by model. sum(increase(aisix_usage_event_drops_total{reason=~"send_failed|retry_budget_exhausted"}[24h])) by (reason, user_name) ``` ### Compare the Same Population[​](#compare-the-same-population "Direct link to Compare the Same Population") Before treating two totals as inconsistent, check their coverage: | Difference | Why it happens | What to check | | ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Scrape coverage | Request counters have no environment label. A per-environment usage total is comparable only when Prometheus scrapes every gateway serving that environment. | Group the raw counter by `job` and `instance`, then compare the contributors with the running gateway instances. | | Time coverage | `increase(...[24h])` uses only samples present in the range. Restarts or shorter Prometheus retention can leave gaps that are absent from a usage-log query. | Graph the raw counter over the same period. | | Delivery coverage | `aisix_usage_event_drops_total` counts both events that could not enter the delivery queue (`sink_disabled`, `sink_full`, `sink_closed`) and events the worker accepted but could not deliver to the control plane (`send_failed`, `retry_budget_exhausted`). Exporter losses are not counted here. | Group queue drops by `model`, and delivery drops by `user_name` — they carry no model or provider-key attribution. Monitor `telemetry batch failed (events dropped)` for control-plane delivery and `sink delivery dropped` for exporter delivery. Events the control plane rejected inside an accepted batch are counted by `aisix_usage_events_rejected_total` instead. | | No upstream call | Target-level rate limits and request-assembly failures still produce usage events but no deployment counter sample. Assembly failures include unusable credentials, a missing `model_name`, and a missing or malformed `api_base`. | A misconfigured target produces failed requests with no deployment failures because the provider was never called. | Deployment metric families cover the upstream calls to configured models, while usage events also cover MCP tool calls, A2A calls, and passthrough routes. Isolate one difference at a time rather than interpreting the entire gap as gateway-side rejection. ## Export Usage Events[​](#export-usage-events "Direct link to Export Usage Events") Usage events are per-attempt records emitted by supported proxy paths. A request that retries or fails over emits multiple events with the same `request_id`, ordered by `attempt_index`. Counting these records therefore counts attempts, not requests — see [Count Requests and Attempts Separately](#count-requests-and-attempts-separately) before comparing a record count against a request metric. Every request has exactly one event with `is_final_attempt: true`. Filter on that field to count requests by their final outcome. It can identify a failed last attempt that has the same status and attempt shape as an earlier failure. Every request also has exactly one event with `attempt_index` `0`, which is its first attempt or its only event when no upstream attempt was made. The control plane uses that first event for its aggregate request count, but its status can describe a failure that a later retry or fallback recovered from. A gateway at 1.8.0 or earlier does not emit `is_final_attempt`. The last row is the only row of a plain request, cache hit, refusal, MCP, A2A, or Realtime call. Otherwise it is one of these: * The fallback that answered, with the failed attempts before it `false`. * The last failure, when every attempt failed. * The refusal that follows an attempt a guardrail rejected after it was billed. * The row a stream writes when it ends, fails part-way, or is abandoned. * The judge or failure row of an ensemble, with its panel rows `false`. Each row of a batch attribution has its own `request_id` and is `true`. If the gateway drops the last event of a request, for example because the delivery queue is full, no row of that request is `true`. The field is written on every row, `false` included, under `metadata_only` as well as `full`: | Destination | Field | Value | | -------------------------------------- | ------------------------ | ----------------------------- | | Control plane, object storage, Datadog | `is_final_attempt` | JSON boolean | | Alibaba Cloud SLS | `is_final_attempt` | String, `"true"` or `"false"` | | OTLP | `aisix.is_final_attempt` | Boolean attribute | The control plane uses it for [Show the Last Attempt of Each Request](https://docs.api7.ai/ai-gateway/cloud/logging-and-auditing.md#show-the-last-attempt-of-each-request). `/v1/messages/count_tokens` and the single-shot endpoints listed in [Supported Endpoints](https://docs.api7.ai/ai-gateway/routing/routing-and-failover.md#supported-endpoints) follow the same rule: each failed attempt that led to a retry or failover emits its own zero-token event. Every event carries `occurred_at`, the time the gateway recorded it, as an RFC 3339 UTC timestamp with exactly three fractional digits, such as `2026-09-24T12:42:25.123Z`. Object storage, Alibaba Cloud SLS, and Datadog receive this string as is, and the OTLP exporter carries the same time in its nanosecond timestamp. The attempts of one request, and events that finish within the same second, can therefore be ordered to the millisecond. RFC 3339 parsers accept the fractional form; a parser that accepts only whole seconds (`…:SSZ`) needs updating. Usage events are not read from a local endpoint. Configure an exporter in [Observability Exporters](https://docs.api7.ai/ai-gateway/observability/exporters.md) to deliver them to OTLP/HTTP, object storage, Alibaba Cloud SLS, or Datadog. Each event includes its outcome, consumption details, the model alias the caller requested, and the resolved model that served the attempt when the gateway can observe those values. On `/v1/chat/completions`, every event also carries `cache_status` (`disabled`, `miss`, `hit`, or `bypass`); a response served from the response cache records the matching layer in `cache_hit_layer` (`exact` or `semantic`) and, for semantic hits, the matched similarity in `cache_similarity`. Streaming responses are never cached, so a streamed request's event reports `cache_status="disabled"` even where the environment has a cache policy enabled. ### Responses Finish Reason[​](#responses-finish-reason "Direct link to Responses Finish Reason") A Responses API result names no finish reason, so AISIX derives the event's `finish_reason` on `/v1/responses` from the result the caller received, served natively or translated, streamed or not, using the chat completions values: | Result `status` | Recorded `finish_reason` | | ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | `completed` | `tool_calls` when the output holds a `function_call` item, a `custom_tool_call` item, or a client-executed `tool_search_call` item, otherwise `stop`. | | `incomplete` | `content_filter` when `incomplete_details.reason` is `content_filter`, otherwise `length`. | | `failed`, `cancelled`, or any other status | None. The field stays empty. | The value reaches the control plane and every exporter like any other finish reason, including OTLP `gen_ai.response.finish_reasons`. A gateway at 1.6.0 or earlier records no finish reason on `/v1/responses`. ### Read the Served Service Tier[​](#read-the-served-service-tier "Direct link to Read the Served Service Tier") `service_tier` is the service tier the upstream response reports it actually served the attempt at, verbatim and in the provider's own vocabulary. It is never the tier the request asked for, and the gateway never fills in a default. Tier names differ across providers and change over time, so the field holds any string: | Upstream | Source | Values the provider reports, for example | | --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | OpenAI and OpenAI-compatible chat completions | The response's top-level `service_tier` | `default`, `flex`, `scale`, `priority`, or `fast`. When OpenAI serves a request with priority processing, GPT-6 models report `fast` and earlier models report `priority`. A downgrade to Standard reports `default`. | | Responses API | The Response object's `service_tier`. On a stream, only the terminal event's Response is read, because an earlier one can still name the requested tier, such as `auto`. | The same values as OpenAI chat completions. | | Anthropic | `usage.speed` when it is not `standard`, otherwise `usage.service_tier` | `fast` in fast mode, otherwise a value such as `standard` or `priority`. | | Amazon Bedrock Converse | `serviceTier.type` | `default`, `flex`, `priority`, or `reserved`. | | Gemini on the Vertex AI adapter | `usageMetadata.serviceTier`, otherwise the `x-gemini-service-tier` response header. A value ending in `unspecified` reports nothing. | `standard`, `flex`, or `priority`. | Passthrough routes that carry a recognized OpenAI, Anthropic, or Responses envelope read the tier the same way, buffered or streamed. The field is omitted when there is no served tier to report: * The upstream reported none. * The response came from the [response cache](https://docs.api7.ai/ai-gateway/traffic-controls/caching.md), which no upstream served. * The attempt received no response, such as a refusal before dispatch or a failed attempt. * A `/v1/responses` stream from a Responses API upstream ended without a terminal event. * A batch job's usage row combines lines that do not all report the same tier. [Model pricing](https://docs.api7.ai/ai-gateway/cloud/model-pricing.md#price-by-service-tier) matches service-tier rates against this field. The control plane shows it as **Service tier** in the expanded [Request Logs](https://docs.api7.ai/ai-gateway/cloud/logging-and-auditing.md#request-logs) row and includes it in the request export. Object storage and Alibaba Cloud SLS carry it as `service_tier`, and Datadog and OTLP as `aisix.service_tier`. The OTLP attribute `openai.request.service_tier` is a different value: the tier the request asked for. A gateway at 1.6.x or earlier does not record the field. ### Read the Latency Fields[​](#read-the-latency-fields "Direct link to Read the Latency Fields") Latency fields distinguish provider time from caller-visible time: | Field | Scope | | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `upstream_latency_ms` | Time spent on one upstream attempt. It excludes request parsing, guardrails, routing, retry delays, and earlier attempts. | | `upstream_ttft_ms` | Time from the start of an upstream attempt until its first streamed frame of any type — metadata openers such as `response.created` or a role-only chat delta included, matching what caller-side proxies measure. It is omitted or zero for non-streaming requests, errors, and cache hits. | | `downstream_latency_ms` | Time from request receipt to downstream delivery: the complete response for non-streaming, the first token or relayed frame for streaming, and the entire stream for A2A and `/v1/audio/speech`. It includes gateway processing, retries and retry delays, and output holdback, and appears only on the terminal attempt. | | `gateway_overhead_ms` | The part of `downstream_latency_ms` that AISIX itself spent: the same window minus the time spent waiting on upstreams inside it. Every failed, retried, or fallback attempt counts as upstream time for its whole duration, and the winning attempt counts up to its first streamed frame when it streams, or for its full upstream latency otherwise. Retry backoff, guardrail calls, including calls to external guardrail services, and output holdback count as gateway time. It appears only on the event that carries `downstream_latency_ms`. An [ensemble](https://docs.api7.ai/ai-gateway/routing/ensemble-models.md) request reports none, because its members run concurrently outside any attempt. On `/v1/realtime`, it is the time before the gateway connected to the upstream. | | `upstream_tpot_ms` | Mean time per output token of one upstream attempt's stream, in milliseconds, as a decimal number: (`upstream_latency_ms` − `upstream_ttft_ms`) / (output tokens − 1). Output tokens include reasoning tokens, as `gen_ai.usage.output_tokens` counts them. It appears only on a successful streamed attempt that produced at least two output tokens. | Object storage and Alibaba Cloud SLS keep these field names. Datadog and OTLP prefix them with `aisix.`, such as `aisix.gateway_overhead_ms` and `aisix.upstream_tpot_ms`. A gateway at 1.6.0 or earlier records neither `gateway_overhead_ms` nor `upstream_tpot_ms`. ### When the Caller Disconnects[​](#when-the-caller-disconnects "Direct link to When the Caller Disconnects") When a caller disconnects, AISIX stops working on its request. It drops the in-flight upstream request or stream, and it starts no further retry or fallback attempt. Dropping the upstream request does not undo work the provider has already started: the provider can still complete, and bill, a request it received before the disconnect. A caller that abandons a request therefore still produces a usage event. Those events carry status `499` and `error_class="client_disconnected"` rather than `200`, so they are filterable as a class. A request abandoned before the response head records zero tokens and zero cost. A mid-stream disconnect records the tokens that arrived before it; when the upstream had not yet reported usage, AISIX estimates the counts where it can and sets `usage_estimated`. Neither figure is a statement of what the provider charged. Count successes as any `2xx` `status_code`, as the dashboard does, to keep abandoned requests out of the total. An [ensemble](https://docs.api7.ai/ai-gateway/routing/ensemble-models.md) request whose caller leaves part-way still records every panel member and judge call that had already returned. Their events come first, numbered consecutively from the request's next `attempt_index`, and the `499` event follows them. Their tokens are charged to the token rate limits the completed request would have used, so a key whose callers abandon ensemble requests still uses up its token budget for the upstream work those requests caused. ### What the Usage Log Records[​](#what-the-usage-log-records "Direct link to What the Usage Log Records") A usage event is the record of a request, not a billing line. AISIX files one for every request it could attribute to a caller, including the ones that never reached a provider and the ones with nothing billable, which carry zero tokens. Use this table to tell an absent record from a recorded rejection: | Outcome | Recorded | What the record carries | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Success | Yes | Status `200`, the resolved target, and the reported consumption. | | Rerank success with no token count | Yes | Status `200` and zero tokens. Cohere rerank models, for example, report search units rather than tokens, and AISIX does not price search units. | | Provider does not support the endpoint | Yes | Status `501`, `error_class="config"`, an `error_message` with the text the caller received, zero tokens, and no upstream call. AISIX answers this itself on text completions, embeddings, image generation, and video submission when the resolved provider lacks the capability. | | Cache hit | Yes | Status `200` with the stored response's token counts replayed, zero cost, `cache_status="hit"`, `cache_hit_layer` (`exact` or `semantic`), and `cache_similarity` on a semantic hit. It carries no `provider_request_id` and no routing target, because no upstream was contacted, but `provider_model_version` does name the model that produced the stored response. Only non-streaming requests are cached, so a cache hit and a streamed response never coincide. | | Model not found | Yes | Status `404`, `error_class="model_not_found"`, `error_message` such as `model "gpt-x" not found`, and an empty `model_id`. | | Rate-limit rejection | Yes | Status `429`, `error_class="rate_limit_exceeded"`, and an `error_message` naming the limit that was hit, such as `request limit exceeded (...)`; a rate-limit-policy rejection adds `(policy '')`. Caller-key, model, and rate-limit-policy rejections all report this, and all three are checked before dispatch. | | Budget exhausted | Yes | Status `429`, `error_class="billing_error"`, and the budget message returned to the caller as `error_message`. | | Guardrail block | Yes | Status `422`, `error_class="content_filter"`, and `guardrail_blocked=true`, on both input and output blocks. `error_message` names only the guardrail, or why it could not evaluate the content, and never the matched content: for example `request blocked by content policy (guardrail 'pii-check')` or `response rejected: guardrail 'pii-check' could not evaluate it ()`. | | Guardrail refuses a streamed answer after the `200` head | Yes | Status `200`, `guardrail_blocked=true`, `error_class="content_filter"`, and the text of the error frame the stream ended on as `error_message`. Exclude these by `guardrail_blocked` or `error_class` when you count `2xx` statuses as successes. | | Upstream error or timeout | Yes | One event per failed attempt, carrying that attempt's `attempt_index`, `attempt_kind`, `attempt_model`, `error_class`, and `error_message`, and its `provider_request_id` when the provider's response carried a request-ID header. When a later attempt succeeds, its success event is the terminal one; when every attempt fails, the last failed attempt's own event is the terminal one rather than an extra row. | | Caller disconnects before the response head | Yes | Status `499`, `error_class="client_disconnected"`, `error_message="client closed the request before the response head was written"`, zero tokens, and zero cost. | | Response body dropped before the stream was read | Yes | The same status and class, with `error_message="client closed the request before the response body was streamed"`. The upstream had answered, so the event names the target that served it. | | Caller disconnects mid-stream | Yes | The same status and class, with `error_message="client closed the request while the response was streaming"`, carrying the tokens delivered before the disconnect. | | Upstream fails after a stream's `200` headers | Yes | The status the same failure gets before the headers, such as `502` or `504`, with `error_class` and `error_message` taken from the failure. See [Streams That Fail After the Response Head](#streams-that-fail-after-the-response-head). | | Authentication failure | No, except on a passthrough route | An access-log line and `aisix_auth_decisions_total`. The line carries the status and `request_id` but no API key ID, because no key was resolved. A passthrough route also files a usage event, with the route's name and no API key ID. | | Request-body parse error or size-limit rejection | No, except on a passthrough route | Access log and metrics only. A passthrough route reads its body after authenticating the caller and files a usage event for a size-limit rejection. | | Request to a route that needs no credential, or to a path no route serves | No | `/livez` and `/readyz` record nothing. The OAuth discovery routes and an unmatched path's `404` write an access-log line only. | | `/livez`, `/readyz`, `/v1/models`, `/v1/videos/{id}` and `/v1/videos/{id}/content`, the OAuth protected-resource discovery routes, and `/a2a/{agent}/.well-known/agent-card.json` | No | These routes emit no usage event at any outcome. A video job's work is metered by its submission. | Every failed usage event carries a bounded `error_message`, including terminal rows and failures on `/v1/realtime`, `/a2a`, and `/mcp`. A gateway refusal uses the same reason as the access log and the error returned to the caller. An upstream failure records the provider's error detail when AISIX can read it, and a request-validation failure can quote the rejected value. An A2A agent's error body is not read outside a full-content export, so under `metadata_only` the message is `upstream A2A request failed: upstream returned HTTP `. When nothing more specific reaches the record, the message is `request failed with status `, so it is never empty on a failure row. See [Read an Upstream Error Message](https://docs.api7.ai/ai-gateway/deployment/troubleshooting.md#read-an-upstream-error-message) for each form. Guardrail refusals remain an exception: their message uses fixed policy wording and the guardrail name, never the matched content. Exporters treat `error_message` as metadata, so it is present under `content_mode: metadata_only` as well as `full`. An exporter destination can therefore receive text originating in a request or upstream error even when full content capture is off. A gateway at 1.8.0 or earlier leaves the terminal event's message empty for failures that could quote the request or upstream, including `invalid_request_error`, an unknown video ID, and upstream errors; an upstream error's per-attempt event already carries its detail. It also writes no `request failed with status ` text, so any failure without a named reason has an empty message. A cache hit answers from a stored response, so its event describes the entry the caller addressed and never a target: * `provider_model_version` names the model that **produced** the stored response, read from the cache entry rather than from this request, and matches the value the original call's own event carried. For a Model Group it is the only field on the row that names the producer at all. It is empty only when the stored response carried no model name. * `provider_request_id` stays empty. It is the identifier a customer reconciles against the provider's console, and this request reached no provider. * On a hit of a **routing** or **semantic** entry, the fields derived from the provider key — `provider_kind`, `provider_featured`, `branded_provider`, `pk_label`, and `byo_label` — are empty, and `provider` reports `unknown`. A gateway at 1.2.0 or earlier filled them from whichever target the request's routing strategy ranked first, which named a target that never ran and could differ between two hits of the same stored entry. One consequence: filtering the control plane's [Request Logs](https://docs.api7.ai/ai-gateway/cloud/logging-and-auditing.md#request-logs) by a provider-key label no longer returns a group's cache hits, and a per-provider request panel counts them under `provider="unknown"` beside the `cache_status="hit"` that explains why. * A hit of a **direct** model keeps that model's own provider, provider key, and upstream model, which are static properties of the model rather than evidence of a dispatch. The request's access-log line follows the same rules; see [Read the Cache Verdict](https://docs.api7.ai/ai-gateway/observability/access-logs-and-request-correlation.md#read-the-cache-verdict). A cancelled request reports `requested_model` as the entry the caller addressed — the group name for a routing request — and `model_id` as the target it had committed to. Before any attempt settled, `model_id` follows [Which Model a Record Names](#which-model-a-record-names): a direct model records its own identifier, and a routing group records none. `requested_model_id` carries the addressed entry's identifier in both cases. A cancelled request also carries no guardrail attribution and no token counts from an attempt that had already answered, because both live in response processing the gateway never reached. Routes where the caller names no model — `/mcp`, `/a2a/{agent}`, the passthrough namespace, `/v1/realtime` before its upgrade, and the files, batches, and fine-tuning surfaces — file the `499` record too. Their model fields are empty and the family's own attribution stands in their place: `passthrough_route_name`, `mcp_server_name` with `mcp_tool_name`, or `a2a_agent_name` with `a2a_method` and `a2a_operation`. The record still carries `api_key_id`, the caller identity, `auth_type`, `operation`, and `inbound_protocol`. ### Which Model a Record Names[​](#which-model-a-record-names "Direct link to Which Model a Record Names") `requested_model` is the name the caller sent, and `requested_model_id` is the ID of the model it resolved to. `model_id` is narrower: it only names a model that calls an upstream itself, never the routing model, semantic router, or ensemble model the caller addressed. Its value depends on the kind of model the caller requested: | Requested model | `model_id` | | ---------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Direct or embedding model, or a wildcard model | The model's own ID on every record, including failures before dispatch and response-cache hits. For a wildcard match it is the wildcard model's ID. | | Routing model or semantic router | On each attempt's record, the target model that attempt was dispatched to. On text completions, rerank, image, audio, video, and other single-request endpoints, a failure after the winning attempt carries the winner's ID. Empty on every record written before a target is selected: a guardrail block, an exhausted budget, a rate-limit rejection before dispatch, a denied model or IP restriction, a semantic routing failure, a request cancelled before the first attempt, and a response-cache hit. | | Ensemble model | Empty when the request is refused before the panel runs. Each panel member's and the judge's record carries that model's ID. The ensemble's aggregate record and failures after the panel carry the ensemble model's own ID. | | A name that does not resolve to a model, and MCP, A2A, and passthrough calls | Empty. | So filtering by a routing model's or semantic router's ID in `model_id` does not find its records that never selected a target, such as its guardrail blocks, rate-limit rejections, and cache hits. Filter by `requested_model_id` or `requested_model` to find every record of a request addressed to a group. A gateway at 1.6.0 or earlier does not apply this rule on every endpoint: depending on the endpoint, a group's failure before dispatch can carry the group's own ID, and a direct model's can carry an empty `model_id`. ### Streams That Fail After the Response Head[​](#streams-that-fail-after-the-response-head "Direct link to Streams That Fail After the Response Head") A streamed response sends its `200` status line before the upstream has produced the answer, so a failure that arrives later cannot change what the client receives. The usage record is written when the stream ends, and it reports how the stream ended rather than the `200` on the client's response line: | How the stream ended | Recorded status | | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | The upstream dropped the connection, or sent something AISIX could not decode | `502` | | A read from the upstream timed out mid-stream | `504` | | A Redis-backed passthrough concurrency lease disappeared while the stream was open | `429`, with `error_class="rate_limit_lease_lost"` and `error_message="distributed concurrency lease was lost"`. The access log records the same status and `error_class`, with the message as `error`. | | The upstream sent an [error inside the stream](#errors-inside-a-successful-response) carrying a `4xx` status | That status. An Anthropic in-band error maps to the status Anthropic documents for its type, so `rate_limit_error` is recorded as `429`. | | The upstream sent any other error event inside the stream, including a `5xx` status or a Responses API `error` or `response.failed` event, which carries no HTTP status | `502` | | A bridged `/v1/responses` upstream stream carried no content, reasoning, tool call, or finish reason | `502`, with `error_class="stream_aborted"` and a message saying the upstream returned an empty stream | | The caller left before the stream ended | `499` and `error_class="client_disconnected"` | | The stream finished | `200` | `error_class` and `error_message` describe the failure itself, even though the client received a `200`. When an upstream failure and a disconnect both occur, the upstream failure is recorded, because a relay that passes a transport error on aborts the connection, which looks the same as the caller leaving. This applies to `/v1/chat/completions`, including the judge of a streaming ensemble (the panel members' rows stay `200`), to `/v1/responses` and `/v1/messages` on both the native and translated paths, to streaming `/v1/audio/transcriptions`, to `/v1/audio/speech`, and to [passthrough routes](https://docs.api7.ai/ai-gateway/endpoints/provider-passthrough.md). A passthrough route that carries a recognized envelope (OpenAI Chat Completions, Completions, or Responses, and Anthropic Messages) reads in-band error events as the typed endpoints do; a raw route has no error envelope to recognize and records only transport failures and timeouts. A non-streamed request whose `2xx` body reports an error is described in [Errors Inside a Successful Response](#errors-inside-a-successful-response). A `/v1/realtime` session the upstream refuses after the `101` upgrade is recorded as `upstream_in_band` too, under its own status rule; see [Upstream Refusals After the Upgrade](https://docs.api7.ai/ai-gateway/endpoints/realtime.md#upstream-refusals-after-the-upgrade). Some outcomes are unchanged: a guardrail block keeps its own status, a stream that produced content and then ended without a finish reason is still a `200`, and a stream that finishes is a `200`. A guardrail that refuses the answer after the head also keeps the `200`, recorded with `guardrail_blocked=true` and `error_class="content_filter"`. That refusal outranks an upstream failure recorded on the same stream. Because these streams are now recorded as failures, the dashboard's success rate falls by the number of them, and they leave the latency percentiles, which count only `2xx` requests. The `status` and `status_code` labels on `aisix_usage_events_emitted_total` move the same way, and the request's access-log status follows its usage event. Cost, budgets, and rate limits do not depend on the status and are unaffected. A gateway at 1.4.0 or earlier records most of these streams as `200` with empty `error_class` and `error_message`. On its translated `/v1/messages` path an upstream failure is recorded as `499`, as is an upstream that drops a streamed transcription. The request metrics follow the same ending. `aisix_requests_total`, `aisix_proxy_requests_total`, `aisix_proxy_failed_requests_total`, and `aisix_llm_requests_total` count a streamed request once, when the stream ends, under the status in the table above, so a stream that fails or is abandoned after its head is a failure there as well. Their duration companions, `aisix_request_duration_seconds`, `aisix_proxy_request_duration_seconds`, and `aisix_llm_request_duration_seconds`, measure the whole stream rather than the time to response start, so streaming percentiles read from them include generation time. `aisix_request_e2e_latency_seconds` and `aisix_request_ttft_seconds` carry the `status_class` of the status the stream ended on. A `/v1/responses` stream that an output guardrail held back and relayed as one buffered body is recorded the same way: when the upstream failed it in-band, the request metrics carry the failure status its usage event records. These rules cover streamed `/v1/chat/completions` (direct models, routing groups, and ensembles), `/v1/messages`, `/v1/responses`, the streamed `/v1/audio/transcriptions` and `/v1/audio/translations` relays, `/v1/audio/speech`, `/a2a`, and passthrough routes. A caller that leaves before the response head is written is still absent from these request families; only [`aisix_proxy_client_cancelled_requests_total`](https://docs.api7.ai/ai-gateway/reference/metrics.md#request-metrics) counts it. Because a streamed request is counted when it ends rather than when it starts, a `rate()` over these counters lags by roughly the length of one stream while a burst of long streams is in progress. Steady-state rates are unchanged. Keep this in mind when you [scale on request load](https://docs.api7.ai/ai-gateway/cloud/kubernetes.md#scale-on-request-load-with-keda). A gateway at 1.5.0 or earlier records these request families when the response head is sent, except on passthrough routes: a streamed request counts as `200`, its duration stops at response start, and the two latency histograms can report `2xx` for a chat, messages, or responses stream that was abandoned or failed. ### Errors Inside a Successful Response[​](#errors-inside-a-successful-response "Direct link to Errors Inside a Successful Response") Some upstreams answer a failed call with HTTP `200` and report the failure in the payload instead: a top-level `error` in a stream event, or a non-streamed body that is only an `error` object. AISIX records both as an upstream failure with `error_class="upstream_in_band"` and an `error_message` of `upstream reported an in-band error: `. An empty message reads `upstream reported an in-band error with no message: `. A gateway at 1.8.0 or earlier writes `in-band stream error` in these two texts. The closing error frame a client receives on a stream still says `in-band stream error`, on every version. See [Read an Upstream Error Message](https://docs.api7.ai/ai-gateway/deployment/troubleshooting.md#read-an-upstream-error-message). One rule decides whether a top-level `error` reports a failure, on stream events and whole bodies alike: it must carry something, meaning a non-empty string, or an object with a non-empty `message` or a non-null `code`. The rule holds whatever else the payload carries, so a stream chunk with `"choices": []` beside such an `error` is a failure. An empty `error`, such as `null`, `{}`, or `""`, reports nothing, because some OpenAI-compatible servers send one beside a normal answer. The status follows the stream table above: an embedded `4xx` code is passed through, an Anthropic error maps to the status Anthropic documents for its type, and anything else is `502`. On a non-streamed request the caller receives that status as an error instead of the body. These non-streamed calls read the body for an error: * `/v1/chat/completions`, `/v1/completions`, `/v1/embeddings`, and `/v1/images/generations` on an OpenAI-compatible target, and `/v1/images/edits`. * `/v1/chat/completions` on Azure OpenAI, on Bedrock's OpenAI-compatible API, on Vertex AI's OpenAI-compatible and Claude models, and on an Anthropic target. * `/v1/messages` and `/v1/responses` on a native target, and `/v1/responses` translated to chat. On the native path, a body that is only an `error` object is a failure, but a response object is the answer even when it carries `"status": "failed"` and its own `error`: AISIX relays it as a `200` and counts its usage. * `/v1/messages/count_tokens`, which fails instead of returning the error body as a count. Gemini and Bedrock targets reached through their native or Converse APIs report failures with an HTTP error status, so AISIX does not read their `2xx` bodies for one. Neither does it on rerank, audio, video, batch, file, or decision requests. A [passthrough route](https://docs.api7.ai/ai-gateway/endpoints/provider-passthrough.md) that carries a recognized envelope applies the same rule to a buffered `2xx` body. The route still relays the upstream's status and bytes as sent, so the caller receives the `200`, but the usage event and access log record the status the failure maps to, with `error_class="upstream_in_band"`. An output guardrail on the route does not block such a body as an answer it could not scan, though a body that carries answer text beside its error is still scanned. A raw route is not probed, and neither is a body that the upstream compressed because no output guardrail covers the route and the caller sent `Accept-Encoding`. A gateway at 1.8.0 or earlier reads only stream events for these errors, and treats a chunk that carries `choices` as an answer. It records a non-streamed `2xx` error body as `upstream_decode`, or relays it as a successful `200` on `/v1/completions`, `/v1/images/generations`, and native `/v1/responses`, and it returns such a body from `/v1/messages/count_tokens` as a count. It records a passthrough route's buffered error body as a `200`, and a passthrough stream event whose `error` is empty as a failure. ### Access Log, Usage Event, and the Logs Page[​](#access-log-usage-event-and-the-logs-page "Direct link to Access Log, Usage Event, and the Logs Page") The three signals count different things, so a request that retried produces a different number of each: | Signal | How many per request | Where to read it | | --------------- | ----------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | | Access-log line | Exactly one, whatever the outcome | The gateway's standard error stream | | Usage event | One per attempt, plus the terminal one for the request; `is_final_attempt` marks the last | An [observability exporter](https://docs.api7.ai/ai-gateway/observability/exporters.md), and the control plane | | Logs page row | One row per usage event | The control plane's [Request Logs](https://docs.api7.ai/ai-gateway/cloud/logging-and-auditing.md#request-logs) | Correlate them by `request_id`, which the caller also receives as `x-aisix-request-id`. Within one request, order the attempts by `attempt_index` and read `attempt_kind` to tell a retry from a failover. A terminal event that speaks for no attempt of its own, such as the `499` event of a caller that left between attempts or the terminal event of a failed ensemble, takes the index following the last one the request emitted and has an empty `attempt_kind`. An empty kind means `initial` only on an `attempt_index` `0` event, and `initial` never labels a later one. Because the counts differ, comparing a row count against a request metric compares two different units — see [Count Requests and Attempts Separately](#count-requests-and-attempts-separately) first. On a gateway newer than 1.2.0, a streamed request's access-log line is written at the end of the stream together with the terminal usage event. Both records report the same status, and the line's `error_class` and `error` carry the event's `error_class` and `error_message`. On a gateway at 1.5.0 or earlier, a passthrough stream's access log leaves the error class and `error` empty. ### Read the Error Class[​](#read-the-error-class "Direct link to Read the Error Class") The access log and the usage event both name a failure in `error_class`, with one vocabulary, on streamed and non-streamed requests and on per-attempt and terminal events alike. A failure that comes from the upstream call, or from preparing it, is named by its upstream class: | Class | Cause | | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `upstream_status` | The upstream answered with a non-`2xx` HTTP status. On `/v1/realtime`, it answered the WebSocket handshake with anything but the `101` upgrade, including a `200`, which is recorded as `502`. | | `upstream_in_band` | The upstream answered with a `2xx` status and reported an error in a stream event or in the response body, or a `/v1/realtime` upstream completed the `101` upgrade and sent an `error` event before any other frame. See [Errors Inside a Successful Response](#errors-inside-a-successful-response). It also covers a JSON-RPC `error` from an upstream MCP server on `tools/call`, or from an A2A agent under HTTP `200`. See [Failed Tool Calls](https://docs.api7.ai/ai-gateway/mcp-gateway/observability.md#failed-tool-calls) and [JSON-RPC Errors From an Agent](https://docs.api7.ai/ai-gateway/agent-gateway/observability.md#json-rpc-errors-from-an-agent). | | `upstream_decode` | AISIX could not parse the upstream response. | | `stream_aborted` | The upstream ended the response mid-stream, or a bridged `/v1/responses` stream ended empty. | | `timeout` | An upstream call exceeded a time budget, including the connect timeout. | | `transport` | The connection to the upstream failed. | | `config` | The provider does not support the requested operation, or AISIX could not build the upstream request. | | `invalid_config` | The target is configured in a way that cannot work, such as a provider key without an `api_base` or a model without a `model_name`. | | `invalid_credentials` | The provider key's credential is missing, empty, or malformed. | | `tool_error` | An MCP tool returned a result flagged `isError`. | | `credential_error` | AISIX could not obtain the credential for an upstream MCP server. | | `tool_not_found` | An OpenAPI-backed MCP server does not have the requested tool. | An MCP `tools/call` keeps status `200` with these classes and with `upstream_status`, `upstream_in_band`, `upstream_decode`, `timeout`, `transport`, and `permission_denied`. Exclude them by `error_class` when you count `2xx` statuses as successes. Every other failure carries the name that an OpenAI-style error envelope reports as `error.type`, such as `model_not_found`, `rate_limit_exceeded`, `billing_error`, `content_filter`, `invalid_api_key`, or `permission_denied`. Two classes describe how a request ended rather than an error the caller was sent: `client_disconnected` and `rate_limit_lease_lost`. One failure is named differently from its `error.type`: a semantic router that could not make its routing decision under a `fail` policy is recorded as `semantic_decision_failed`, while the caller receives `error.type` `api_error` with `error.code` `semantic_routing_failed`. The usage event's `error_message` carries the access log's `error`, `semantic router '' could not pick a route ()`. See [Semantic Routing Failures](https://docs.api7.ai/ai-gateway/reference/headers-and-error-codes.md#semantic-routing-failures). A route's own `501` for a provider lacking the capability is recorded as `config`, while the caller receives `error.type` `not_implemented`. A `tools/call` that the MCP protocol handler refuses with a `4xx` over its headers, session, or parameters is recorded as `invalid_request_error`, and `error_message` holds what the response body says. A gateway at 1.8.0 or earlier records neither class on these failures. For upstream failures the class is not the client-facing `error.type`. The error envelope keeps its own spelling, so a caller can receive `error.type` `upstream_error` or `transport_error` for a failure that the access log and usage event record as `upstream_status` or `transport`. See [OpenAI-Style Proxy Errors](https://docs.api7.ai/ai-gateway/reference/headers-and-error-codes.md#openai-style-proxy-errors). A gateway at 1.5.0 or earlier uses the client-facing spelling on the access-log line of a non-streamed request, and on a terminal usage event that carries no attempt record, such as the error event of an embeddings, rerank, or image request. Those records report `upstream_error`, `transport_error`, `upstream_decode_error`, `upstream_in_band_error`, `config_error`, `invalid_request_error`, and `authentication_error` where a later gateway reports `upstream_status`, `transport`, `upstream_decode`, `upstream_in_band`, `config`, `invalid_config`, and `invalid_credentials`. Streamed lines and per-attempt events already used the class names. A query or alert that matches the older values needs both spellings while gateways of both versions report. ### How Usage Events Reach the Control Plane[​](#how-usage-events-reach-the-control-plane "Direct link to How Usage Events Reach the Control Plane") This leg exists in managed mode only. The gateway queues usage events in memory and a worker delivers them to the control plane in batches. The queue holds 16,384 events; when it is full the event being emitted is dropped rather than made to wait, because telemetry must not slow the request path. Each such drop increments `aisix_usage_event_drops_total{reason}`, with `sink_full`, `sink_closed`, or `sink_disabled` naming the cause. For `sink_full` and `sink_closed`, the gateway also logs `usage event dropped` at `WARN`, at most once per reason per 10 seconds: the first drop is written at once, and each later line carries `dropped=`, the drops since the previous line for that reason. Drops after the last line wait for the next drop's line, so the log can understate a burst that has ended. Read the metric for the exact count. A gateway at 1.9.0 or earlier writes one line per drop. The worker flushes whenever it has 100 events or every 5 seconds, whichever comes first, which is why a freshly completed request takes a few seconds to appear. One batch is in flight at a time, and batches go out in order. A batch that fails to deliver is re-sent, as long as the control plane has signalled that it recognizes a batch it already stored and will not count it twice. The first re-send waits 1 second and the wait doubles per attempt, up to 30 seconds. A control-plane outage therefore no longer costs the usage recorded during it. The first failure logs `telemetry batch failed; re-sending (the control plane de-duplicates by batch id)`, and a batch that lands afterwards logs `telemetry batch delivered after re-sending`. Re-sending is bounded, because a usage record that arrives too late no longer reaches the billing window it belongs to: * A batch is given up on 30 minutes after its **oldest** event occurred — not 30 minutes after its first attempt. A batch already older than that when its turn comes is given up on without being sent at all. * A batch is also given up on after 8 consecutive failures that the control plane answered. Failures with no answer at all — a connection error or a timeout, which is what an outage produces — do not count towards this limit; the age limit bounds those instead. * An answer that says the batch is permanently unacceptable ends it at once, with no re-send. * A control plane that does not signal de-duplication gets one attempt per batch, exactly as before re-sending existed. * Shutdown makes one final attempt for the batch in hand and does not wait out a backoff. Every event in a batch that is given up on is counted against `aisix_usage_event_drops_total{reason}` under `send_failed` (given up without ever being re-sent) or `retry_budget_exhausted` (the age limit, or the eight answered failures), and logs `telemetry batch failed (events dropped)` with `reason`, `attempts`, and `batch_id`. These samples carry the member labels (`user_id`, `user_name`) only; `model` and the provider-key labels report `unknown`, because the worker holds events rather than the label set the emitting handler had. New events keep accumulating in the queue while a batch is being re-sent, and the queue is the only thing holding them. So usage is still lost in two cases: a queue that fills up, which drops the newly emitted event under `sink_full`, and an outage that outlasts the limits above. Alert on any increase of `aisix_usage_event_drops_total` and break it down by `reason` to tell the two apart. The control plane can also accept a batch while rejecting individual events in it, when an event fails its per-field checks — for example, a malformed ID or a status code outside the HTTP range. Those events are dropped rather than re-sent, because the control plane would refuse them again. A gateway newer than 1.4.0 adds their number to `aisix_usage_events_rejected_total` and logs `control plane rejected usage events in an accepted telemetry batch (events dropped)` with `batch_id`, `count`, and `rejected`. The counter has no labels, because the control plane answers with a count only, and these events are not counted in `aisix_usage_event_drops_total`. A gateway at 1.4.0 or earlier treats any successful answer as full delivery. A missing row is therefore a request that files none — see the table above — an event lost at one of those points, which the drop counter's `reason` distinguishes, or an event the control plane rejected, which `aisix_usage_events_rejected_total` counts. ### Tell Request Kinds Apart[​](#tell-request-kinds-apart "Direct link to Tell Request Kinds Apart") Every event carries `operation`, which identifies the kind of work from the matched endpoint. Neither `inbound_protocol` nor the model name reliably distinguishes a conversation, image, video, or other operation, so use this field when grouping traffic by request type. The values form a fixed set suitable for indexing, grouping, and charting: | Value | Endpoint | | --------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | `chat` | `/v1/chat/completions` | | `messages` | `/v1/messages` | | `count_tokens` | `/v1/messages/count_tokens` | | `responses` | `/v1/responses` | | `completions` | `/v1/completions` | | `embeddings` | `/v1/embeddings` | | `rerank` | `/v1/rerank` | | `systemone` | `/v1/systemone` | | `decisions` | `/v1/decisions` | | `image_generation` | `/v1/images/generations` | | `image_edit` | `/v1/images/edits` | | `transcription` | `/v1/audio/transcriptions` | | `translation` | `/v1/audio/translations` | | `speech` | `/v1/audio/speech` | | `video_generation` | `POST /v1/videos` | | `realtime` | `/v1/realtime` | | `files`, `batches`, `fine_tuning` | The file, batch, and fine-tuning management endpoints | | `batch_completion` | The gateway's own accounting of a finished batch job, recorded when the job completes rather than when it was submitted | | `mcp`, `a2a` | The MCP and A2A gateways | | `passthrough` | A passthrough route | Keep these behaviors in mind when querying the field: * **It describes the request, not the outcome.** A request that failed, or that a guardrail refused, carries the same value a successful one would. It is the only field on such a record that names the endpoint at all. * **It is request-scoped.** A request that retries or fails over emits one event per attempt, and every attempt carries the same value, so counting events per operation counts attempts. See [Count Requests and Attempts Separately](#count-requests-and-attempts-separately). * **Polling a video job is not video generation.** Only the submission (`POST /v1/videos`) produces a usage event; retrieving the job's status or downloading its result does not. A count of `video_generation` is therefore a count of videos asked for, not of requests made about them. Because `operation` is metadata, exporters retain it in `metadata_only` mode even though they receive no prompt content. In Alibaba Cloud SLS, it arrives as a separate column. Enable analytics for the fields used by a query, then group traffic by operation: ``` * | SELECT operation, COUNT(*) AS calls, SUM(prompt_tokens + completion_tokens) AS tokens GROUP BY operation ORDER BY calls DESC ``` To isolate one kind of traffic, filter on the operation directly, for example `operation: video_generation`. Do not infer it from the model: one model can serve several endpoints, and `requested_model` identifies the configured alias or group rather than the request type. ### Attribute Events to a Member[​](#attribute-events-to-a-member "Direct link to Attribute Events to a Member") Each event uses `user_id` to identify the organization member who owned the caller API key when the request ran: | Situation | Recorded behavior | | ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The caller API key has no member | `user_id` is absent. Ownership is assigned explicitly, not inferred from whoever created the key. | | One member uses several credentials | Events can have different `api_key_id` values and the same `user_id`. This includes API key and OIDC calls that resolve to different keys owned by one member. | | A key is reassigned or deleted | Existing events keep the original `user_id`. Reassignment attributes later requests to the new owner; deletion does not erase earlier attribution. | | The event predates gateway support for this field | No member is recorded, so member filters cover traffic only from the upgrade forward. | Filter by `user_id` to include all credentials owned by a member, or by `api_key_id` to isolate one credential. Each event also carries `api_key_name`, the display name of the caller API key when the request ran, so a later rename does not change who an earlier event names. It is omitted when the key has no name. Object storage and Alibaba Cloud SLS keep the name `api_key_name`, and Datadog and OTLP carry it as `aisix.api_key_name`, regardless of `content_mode`. A gateway at 1.6.0 or earlier does not record it. In AISIX Enterprise and AISIX Cloud, the dashboard's **Logs** page offers this as the **Member** filter, alongside a **Status** filter that accepts a family (`4xx`), an exact code (`429`), or a range (`500-599`). Combining the two answers questions like "which of this member's requests were rate-limited in the last 24 hours" in one query. The CSV export carries `user_id` and the member's name. ### Attribute Events to a Team and Tags[​](#attribute-events-to-a-team-and-tags "Direct link to Attribute Events to a Team and Tags") Each event also carries two attribution fields that the organization controls: * `team_id`: the ID of the team the caller API key is bound to. It is omitted when the key has no team binding. * `tags`: the request's [cost attribution tags](https://docs.api7.ai/ai-gateway/cloud/cost-attribution-tags.md), as an object of tag key to tag value. Each tag takes the value a JWT caller's token supplies through the trust provider's `tag_claims`, else the caller API key's own value, else the value set on the key's team, else the value set on the key's member. It is omitted when the request resolves no tags. Both are fixed when the request runs and are the same on every event of the request, including retries, failovers, and the gateway's own rejections. They are never read from request headers or the body. Each exporter carries them under these names, regardless of `content_mode`: | Exporter | Team | Tags | | ----------------------- | --------------- | -------------------------------------------------------------------------------------- | | OTLP | `aisix.team_id` | One `aisix.tags.` string attribute per tag, on the operation span | | Datadog | `aisix.team_id` | A nested `aisix.tags` object, which Datadog facets as `aisix.tags.` | | Alibaba Cloud SLS | `team_id` | One `tags.` field per tag, so each tag is a column a query can filter on directly | | Object storage (NDJSON) | `team_id` | A `tags` object | For example, an SLS query can sum tokens by department with `* | SELECT "tags.department" AS department, SUM(prompt_tokens + completion_tokens) AS tokens GROUP BY department`. Tags and team never become Prometheus metric labels. A gateway at 1.6.x or earlier records neither field. ### Attribute Events with Request Headers[​](#attribute-events-with-request-headers "Direct link to Attribute Events with Request Headers") When several agents or end users share one caller API key, `api_key_id` and `user_id` name the key and its owner but not who each request was for. Callers can send that attribution in request headers of their own, such as a sub-user ID, a department, or a project, and an [observability exporter](https://docs.api7.ai/ai-gateway/observability/exporters.md) can record those header values on every usage event it exports. An external system that receives the events can then attribute cost by them. Recording is all the gateway does with these values. They do not affect routing, rate limits, budgets, or billing, and they are never sent to the control plane, so the dashboard does not display them or offer them as a filter in Request Logs. They reach you only through exporters. Recording a header never changes what the gateway forwards upstream. #### Configure Recorded Headers[​](#configure-recorded-headers "Direct link to Configure Recorded Headers") List the header names in an exporter's `request_headers` field. Every exporter kind accepts it, and each exporter records only the headers in its own list, so two exporters can record different headers. An omitted or empty list records none. The same exporters also accept `upstream_response_headers`; see [Record Upstream Response Headers](#record-upstream-response-headers). In AISIX Enterprise or AISIX Cloud, enter comma-separated names in the exporter form's **Request headers to record** and **Upstream response headers to record** fields in the dashboard. The form converts the names to lowercase and removes duplicates before saving. Through the AISIX Cloud Admin API, set the lists when creating the exporter or patch an existing one, using the variables from [Observability Exporters](https://docs.api7.ai/ai-gateway/observability/exporters.md#prerequisites): ``` curl -sS -X PATCH "$AISIX_CP/environments/$ENV_ID/observability_exporters/$EXPORTER_ID" \ -H "Authorization: Bearer $AISIX_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "request_headers": ["x-sub-user", "x-department"], "upstream_response_headers": ["x-request-id"] }' ``` A patch replaces each list it names as a whole, and an empty list clears it. Reads return both lists. For an open-source gateway, add the lists to the exporter entry in the resources file, then validate and reload it: resources.yaml ``` observability_exporters: - name: request-events-s3 kind: object_store provider: s3 bucket: acme-aisix-events prefix: ai-gateway region: us-east-1 credential_ref: acme_s3 request_headers: ["x-sub-user", "x-department"] upstream_response_headers: ["x-request-id"] ``` The lists take effect when the gateway loads the updated exporter, without a restart. The AISIX Cloud Admin API and `aisix validate` refuse a list that contains: * A name with characters other than lowercase letters, digits, and ``! # $ % & ' * + . ^ _ ` | ~ -``. Write names in lowercase; matching against the headers is case-insensitive. * A header that carries a credential: `api-key`, `authorization`, `cookie`, `proxy-authorization`, `x-amz-content-sha256`, `x-amz-date`, `x-amz-security-token`, `x-api-key`, or `x-goog-api-key`, or the gateway credential header `x-aisix-api-key`. * `set-cookie`, in `upstream_response_headers` only. * The same name twice. The lists have no length limit. A [passthrough route](https://docs.api7.ai/ai-gateway/endpoints/provider-passthrough.md#gateway-authentication-modes) that uses `auth_mode: header_key` reads the caller's gateway key from its `auth_header_name`, and that header is never recorded on the route's requests, even when an exporter lists it. warning Do not list a header that carries a credential or other secret. The checks refuse only the fixed names above. Any other listed header is recorded like any other, and its value is exported in plaintext to that exporter's destination. A gateway at 1.6.0 or earlier does not support the lists. Connected to AISIX Enterprise or AISIX Cloud, it ignores them: the control plane still saves the exporter and returns a `field_ignored` warning for such gateways, which the dashboard shows after saving. An open-source gateway at 1.6.0 or earlier rejects a resources file that sets them. #### Upgrade from the Startup Setting[​](#upgrade-from-the-startup-setting "Direct link to Upgrade from the Startup Setting") A gateway at 1.6.0 selects recorded headers in its startup configuration, under `observability.usage_event.request_headers` (`AISIX_OBSERVABILITY__USAGE_EVENT__REQUEST_HEADERS`), and records them for every exporter. From 1.7.0, that setting is retired. A configuration that sets it still loads, but the setting has no effect. A gateway at 1.7.x logs a warning at startup saying that `observability.usage_event` does nothing; a later gateway treats it as an unknown key and logs that it [was ignored](https://docs.api7.ai/ai-gateway/reference/environment-variables.md#misspelled-and-retired-keys). Before upgrading a gateway that sets it, copy its list into the `request_headers` of each exporter that should keep receiving the headers. Until an exporter lists them, its records carry no request headers. Then delete the setting from the startup configuration, environment, or Helm values. #### Recorded Values[​](#recorded-values "Direct link to Recorded Values") Each usage event an exporter delivers carries a `request_headers` object keyed by the lowercase header name, holding the headers in that exporter's list: ``` { "request_headers": { "x-sub-user": "alice", "x-department": "eng, ops" } } ``` The values follow these rules: * A listed header the request did not send has no key. When the request sent none of the listed headers, the record has no `request_headers` field at all. * A header sent more than once is recorded as one value, its occurrences joined with `", "` in arrival order. * Control characters are removed and the value is cut to 256 characters. A value left empty has no key. * Every usage event of the request carries the same object, whatever the request type: the model endpoints, passthrough routes, MCP, A2A, and the others, including the event of each failed attempt that led to a retry or failover. * Recording does not depend on an exporter's `content_mode`. The headers are exported in the default metadata-only mode too. The values are asserted by the caller: anyone holding the API key can send any value. They are suitable for attribution, not as an authentication or authorization boundary. #### Record Upstream Response Headers[​](#record-upstream-response-headers "Direct link to Record Upstream Response Headers") An exporter's `upstream_response_headers` lists headers to record from the upstream's response, such as a provider's request ID. They are recorded per attempt: each attempt's usage event carries the headers of that attempt's own upstream response, so a request that retried or failed over keeps the response headers of every attempt. An attempt that received no response head carries none. The values follow the same rules as request headers, under an `upstream_response_headers` object. `/mcp` records request headers but no upstream response headers. #### Batch Job Usage[​](#batch-job-usage "Direct link to Batch Job Usage") The usage event that records a batch job's token usage is emitted when a `GET /v1/batches/{id}` request observes the batch completing. It carries the request headers of that request, not of the request that created the batch — the same request its API key and member attribution come from. A background poller that sends no attribution headers therefore leaves that batch's usage without them. #### Headers in Each Exporter[​](#headers-in-each-exporter "Direct link to Headers in Each Exporter") Destinations render the objects in their own format: | Exporter | Field | | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Object storage | Nested `request_headers` and `upstream_response_headers` objects in each NDJSON record. | | Alibaba Cloud SLS | `request_headers` and `upstream_response_headers` fields whose values are the objects as compact JSON strings. | | Datadog | Nested `aisix.request_headers` and `aisix.upstream_response_headers` attributes, addressed per header as `aisix.request_headers.` and `aisix.upstream_response_headers.`. | | OTLP | One attribute per header, following the OpenTelemetry HTTP semantic conventions: `http.request.header.` on the request's SERVER span, and `http.response.header.` on the attempt's GenAI operation span. Each value is a string array with one element. | ### Recorded Token Counts[​](#recorded-token-counts "Direct link to Recorded Token Counts") A usage event records token counts as the upstream reported them. The caller reads a usage block adapted to the protocol it used, while the event keeps the accounting the upstream used, so the two can differ. On `/v1/chat/completions`, the caller's `usage.prompt_tokens_details` and `usage.completion_tokens_details` carry every counter the upstream reported, with its value, a reported `0` included: `cached_tokens`, `reasoning_tokens`, `audio_tokens`, `image_tokens`, `text_tokens`, `accepted_prediction_tokens`, and `rejected_prediction_tokens`. A counter the upstream did not report is omitted, and so is a details object with nothing reported. A response served from the [response cache](https://docs.api7.ai/ai-gateway/traffic-controls/caching.md) reports only non-zero `cached_tokens` and `reasoning_tokens`. A gateway at 1.6.x or earlier drops every counter of `0` and never relays `text_tokens`, `image_tokens`, or the prediction counters. The event carries `total_tokens`, the total the upstream itself reported, verbatim: | Upstream | Source of `total_tokens` | | ---------------------------------- | ------------------------ | | Gemini on the Vertex AI adapter | `totalTokenCount` | | OpenAI-compatible and Azure OpenAI | `usage.total_tokens` | | Responses API | `usage.total_tokens` | | Amazon Bedrock Converse | `totalTokens` | The field is omitted when the upstream reports no total. Anthropic, for one, never reports a total. The gateway never fills it with a sum of its own. When the gateway combines several usage reports into one figure, such as the usage frames of one stream, the event keeps a total only if every report carried one. Every endpoint family that reports token usage records the field: chat completions, completions, `/v1/messages`, `/v1/responses`, the decision endpoints, embeddings, rerank, audio transcription and translation, image generation and editing, Realtime, completed-batch attribution, and passthrough routes metered as one of these. A Realtime session event, and a completed batch's event for one model (`operation` `batch_completion`), record the sum of the totals their upstream responses reported, and omit the field if any response that counted tokens reported none. Amazon Bedrock and Vertex AI embeddings report no total, so their events omit it. Events that count no tokens, such as Count Tokens, video submission, and `/v1/audio/speech`, carry none. How far back the field reaches depends on the gateway version: * A gateway at 1.7.0 or later records it on every family listed above. * A gateway from 1.5.0 through 1.6.x records it only on chat completions, completions, `/v1/messages`, `/v1/responses`, and passthrough routes. Its events for embeddings, rerank, audio, images, Realtime, and batches omit it. * A gateway at 1.4.0 or earlier never records it. The counters are not reconciled with each other, so `reasoning_tokens` can exceed `completion_tokens`, and `cached_prompt_tokens` can exceed `prompt_tokens`. [How Reasoning Tokens Are Billed](https://docs.api7.ai/ai-gateway/cloud/model-pricing.md#how-reasoning-tokens-are-billed) explains how the control plane uses `total_tokens` to tell the two ways of counting reasoning apart. For a Gemini thinking model on the Vertex AI adapter, Gemini reports the thinking tokens (`thoughtsTokenCount`) beside the answer tokens (`candidatesTokenCount`). The event records `completion_tokens` as the answer tokens, `reasoning_tokens` as the thinking tokens, and `total_tokens` as `totalTokenCount`, as long as prompt + completion + reasoning tokens equal that total. When Gemini reports no total, or a total those counts do not add up to, the event keeps `completion_tokens` with the thinking tokens included, as a gateway at 1.4.0 or earlier records it. The caller's response is unchanged either way: an OpenAI-shape response still counts the thinking tokens inside `completion_tokens`, and so do token rate limits and the Prometheus token metrics. See [Google Vertex AI](https://docs.api7.ai/ai-gateway/providers/google-vertex-ai.md#endpoint-and-content-support). On a gateway newer than 1.4.0, `/v1/messages` and `/v1/completions` record `reasoning_tokens` and `total_tokens` exactly as `/v1/chat/completions` does, so one upstream call records the same counts whichever of these endpoints addressed it. A gateway at 1.4.0 or earlier records no reasoning count on those two endpoints. Exporters receive these recorded counts. Object storage and Alibaba Cloud SLS carry the event as is, including `total_tokens`. OTLP and Datadog report `gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens` under the OpenTelemetry convention instead, where input includes cached tokens and output includes reasoning; see [Token Counts in Exported Telemetry](https://docs.api7.ai/ai-gateway/observability/exporters.md#token-counts-in-exported-telemetry). ### Cache Creation by Lifetime[​](#cache-creation-by-lifetime "Direct link to Cache Creation by Lifetime") A one-hour prompt-cache write costs more than a five-minute one, so an event can split `cache_creation_tokens` by cache lifetime: | Field | Meaning | | -------------------------- | --------------------------------------------------------------------- | | `cache_creation_5m_tokens` | The part of `cache_creation_tokens` written with a 5-minute lifetime. | | `cache_creation_1h_tokens` | The part of `cache_creation_tokens` written with a 1-hour lifetime. | The split comes from Anthropic's `usage.cache_creation.ephemeral_5m_input_tokens` and `ephemeral_1h_input_tokens`, on Anthropic's API, on Claude's native Messages API on Amazon Bedrock and Vertex AI, on the chat and Responses translations to Claude, and on passthrough routes metered as Anthropic. It also comes from Bedrock Converse `usage.cacheDetails`, by `ttl`. The two fields appear together, and only when they add up to a non-zero `cache_creation_tokens`. An upstream that reports no split, a split that does not cover the whole write, and the all-zero split of an uncached request all leave both fields off; the gateway never infers a split. `cache_creation_tokens` stays the total either way, and a cache hit carries no split. Object storage, Alibaba Cloud SLS, and the control plane keep these names. Datadog and OTLP report them as `aisix.cache_creation_5m_tokens` and `aisix.cache_creation_1h_tokens`, while `gen_ai.usage.cache_creation.input_tokens` stays the total. AISIX Enterprise and AISIX Cloud store the split and price the one-hour part separately: at the price's one-hour cache write rate when it sets one, otherwise at twice the input rate in effect for a Claude model on Anthropic, Amazon Bedrock, or Google Vertex AI, and at the cache write rate for every other model. See [Model Pricing](https://docs.api7.ai/ai-gateway/cloud/model-pricing.md#add-a-pricing-override). A gateway at 1.6.x or earlier does not record the split, so the control plane bills its whole write at the cache write rate. ### Audio and Image Tokens[​](#audio-and-image-tokens "Direct link to Audio and Image Tokens") An event can carry the audio and image token counts the upstream reported: `prompt_audio_tokens`, `prompt_image_tokens`, `completion_audio_tokens`, and `completion_image_tokens`. Each is a part of `prompt_tokens` or `completion_tokens`, cached tokens included, and is never added to them. A count the upstream did not report is omitted; a reported `0` stays zero. | Upstream Usage | Source | | ----------------------------------------------------------------------------- | ---------------------------------------------------------------- | | OpenAI-format chat completions, including legacy completions and batch output | `prompt_tokens_details` and `completion_tokens_details` | | Responses API, including `/v1/decisions` | `input_tokens_details` and `output_tokens_details` | | Realtime, and audio transcription billed by tokens | `input_token_details` and `output_token_details` | | Image generation and image edits | `input_tokens_details` and `output_tokens_details` | | Gemini on the Vertex AI adapter | `promptTokensDetails` and `candidatesTokensDetails`, by modality | Passthrough routes that carry a recognized envelope read the same fields. Anthropic and Amazon Bedrock Converse do not report these counts. A cache hit carries none, because the counts are not stored with the cached response. A gateway at 1.6.0 or earlier does not record them. Object storage and Alibaba Cloud SLS keep these field names, Datadog prefixes them with `aisix.`, and OTLP reports them as `gen_ai.usage.audio.input_tokens`, `gen_ai.usage.image.input_tokens`, `gen_ai.usage.audio.output_tokens`, and `gen_ai.usage.image.output_tokens`. ### OpenAI Cache-Write Tokens[​](#openai-cache-write-tokens "Direct link to OpenAI Cache-Write Tokens") AISIX recognizes `cache_write_tokens` inside Chat Completions `usage.prompt_tokens_details` or Responses `usage.input_tokens_details`. It preserves that raw value as the optional `cache_write_tokens` field of the usage event, including requests bridged through `/v1/messages` or `/v1/responses` and supported protocol-aware passthrough routes. This requires a gateway newer than 1.1.0. `cache_write_tokens` counts the tokens written to the cache. They are included in `prompt_tokens` and are not included in `cached_prompt_tokens`, which is how OpenAI reports them on both Chat Completions and Responses. Usage event fields ``` { "prompt_tokens": 101, "completion_tokens": 11, "cached_prompt_tokens": 19, "cache_write_tokens": 37 } ``` The field is omitted when the upstream did not report it; an explicit `0` remains zero. The control plane's Request Logs detail view, usage-events API, and JSON/CSV export preserve that distinction. CSV uses an empty cell for an absent value. Logs exported through a backend with prefixed field names use its usual prefix, such as `aisix.cache_write_tokens` in Datadog. This is separate from Anthropic's additive `cache_creation_tokens`, and it does not increase input or output totals: in the example above, input plus output is still 112. AISIX Enterprise and AISIX Cloud bill these tokens at the cache write rate instead of the input rate; see [Model Pricing](https://docs.api7.ai/ai-gateway/cloud/model-pricing.md#openai-cache-writes). #### Cache Writes by Retention[​](#cache-writes-by-retention "Direct link to Cache Writes by Retention") On a 1.10.0 or later gateway, an event can also split `cache_write_tokens` by OpenAI's cache retention, because a 12-hour write is priced above a 30-minute one: | Field | Meaning | | ------------------------ | -------------------------------------------------------------------- | | `cache_write_30m_tokens` | The part of `cache_write_tokens` written with a 30-minute retention. | | `cache_write_12h_tokens` | The part of `cache_write_tokens` written with a 12-hour retention. | For example, this event reports 37 cache-write tokens, of which 12 used the 30-minute retention and 20 used the 12-hour retention. The remaining five tokens have no reported retention: ``` { "cache_write_tokens": 37, "cache_write_30m_tokens": 12, "cache_write_12h_tokens": 20 } ``` AISIX reads the split from `cache_write_tokens_details` beside `cache_write_tokens`. This applies wherever it reads an OpenAI-shape usage object: buffered and streamed Chat Completions and Responses, legacy Completions, the decision endpoints, and bridged or passthrough routes that read the same usage. Each split field appears independently when the upstream reports it; an unreported value is omitted, not written as zero. AISIX omits both fields when their sum exceeds `cache_write_tokens` or when there is no write. Any write the split does not cover remains part of `cache_write_tokens`, as in the example above. OpenAI returns the split on a Responses request that selects a cache retention, through `prompt_cache_options.ttl` or a content block's `prompt_cache_breakpoint.ttl`, and omits it otherwise. Chat Completions supports only the 30-minute retention and returns no split. A request that sets no retention therefore reports the total alone. AISIX forwards both request parameters to the upstream unchanged on the native Chat Completions and Responses paths. Object storage, Alibaba Cloud SLS, and the control plane keep these names. Datadog and OTLP report them as `aisix.cache_write_30m_tokens` and `aisix.cache_write_12h_tokens`, next to `aisix.cache_creation_5m_tokens` and `aisix.cache_creation_1h_tokens`. In AISIX Enterprise and AISIX Cloud, the request detail on the **Logs** page shows them below **Cache write tokens** as **Cache write, 30-minute retention** and **Cache write, 12-hour retention**. ## Next Steps[​](#next-steps "Direct link to Next Steps") Configure [Observability Exporters](https://docs.api7.ai/ai-gateway/observability/exporters.md) to send usage events to an external collector, log destination, object store, or warehouse workflow. Use [Access Logs and Request Correlation](https://docs.api7.ai/ai-gateway/observability/access-logs-and-request-correlation.md) to investigate individual requests, and use the [Metrics Reference](https://docs.api7.ai/ai-gateway/reference/metrics.md) when building Prometheus dashboards or alerts.