Metrics and Logs
AISIX AI Gateway exposes aggregate metrics, per-request logs, response headers, and exportable usage events. Together, these signals show service health and connect a caller-visible response to the model, route, and policy outcome that produced it.
Choose a Telemetry Source
Start with the source that matches the operational question, then correlate signals by request, model, or provider when an issue needs deeper investigation.
| Operational Question | Start With | What It Provides |
|---|---|---|
| Is traffic healthy across gateway instances? | Prometheus metrics | Request rates, latency distributions, token and cost counters, policy outcomes, routing health, cache behavior, and exporter delivery health. |
| What happened to one request? | Access logs | Structured request fields, including status, latency, model, provider, request ID, and routing outcome when available. |
| What can the calling application observe? | Response headers | Request correlation, cache outcome, retry timing, and selected-target hints on supported routes. |
| Where can request records be stored, analyzed, or used for accounting? | Usage events | Per-attempt outcome and consumption records delivered through an observability exporter. |
Scrape Prometheus Metrics
Prometheus metrics are suited for monitoring traffic trends, latency, token counters, cost counters, rate-limit outcomes, cache behavior, and exporter delivery health.
AISIX serves Prometheus metrics on the dedicated metrics listener at /metrics by default. Change the path or disable the endpoint through the startup observability settings.
This endpoint is unauthenticated by design. Keep the dedicated metrics listener private.
Configure Prometheus exposure in the startup configuration:
observability:
metrics:
prometheus:
enabled: true
path: "/metrics"
The dedicated listener binds to 0.0.0.0:9090 by default. Set a different listener address when Prometheus should scrape another interface or port.
Scrape the default metrics endpoint:
curl -sS "http://127.0.0.1:9090/metrics"
AISIX publishes configuration status on every scrape. Other metric families are registered on first observation, so traffic metrics might not appear immediately after startup. Send one model request, then check again for series such as aisix_requests_total and aisix_tokens_consumed_total.
AISIX emits native metric names with the aisix_ prefix. Use the histogram series for latency percentiles across gateway instances and the request counters for success-rate and routing analysis. For exact metric names, label scope, and PromQL examples, see Metrics Reference.
Count Requests and Attempts Separately
A request and an upstream attempt are different units, and mixing them is the most common source of apparently contradictory numbers. One client request can place several upstream calls: a same-target retry, or a failover to the next target in a Model Group. AISIX counts both units, in different places.
| Unit | Where it is counted | What one sample or row means |
|---|---|---|
| Request | aisix_proxy_requests_total, aisix_llm_requests_total | One client request, with the status the caller received. |
| Attempt | aisix_deployment_requests_total and the deployment families | One upstream call to one target model. |
| Emission attempt | aisix_usage_events_emitted_total | One attempt to enqueue a usage event, counted before the delivery queue accepts or rejects it. |
| Attempt | Usage-event records, and the usage log built from them | One attempt, joined to its siblings by request_id and ordered by attempt_index. |
The consequence worth internalizing: a failed attempt that a fallback rescued is invisible in the request counters. The caller received a 200, so the request counter records status="200" and nothing else. The 502 the first target returned lives in the deployment counters and in the usage log, as its own attempt.
So a gateway fronting one broken target in an otherwise healthy Model Group can legitimately show tens of thousands of 5xx rows in the usage log while sum(increase(aisix_proxy_requests_total{status=~"5.."}[24h])) returns a few hundred. The log is counting attempts that failed; the metric is counting callers who saw a failure. Neither is wrong, and they are not expected to converge.
Use these queries for the questions they actually answer:
# Requests that ended in a server error — what callers experienced.
sum(increase(aisix_proxy_requests_total{status=~"5.."}[24h]))
# Requests that ended in a server error even after failing over.
sum(increase(aisix_proxy_requests_total{status=~"5..", is_fallback="true"}[24h]))
# Upstream attempts that failed, by target — including the ones a fallback
# rescued. This is the attempt-level view of a 5xx usage-log row count,
# restricted to the endpoints that dispatch through a Model Group.
sum(increase(aisix_deployment_failure_responses_total[24h])) by (model)
# Fallbacks that rescued a request, by group and by the target reached.
sum(increase(aisix_routing_successful_fallbacks_total[24h])) by (model, fallback_model)
# Usage-event emission attempts, and those the handoff queue rejected.
sum(increase(aisix_usage_events_emitted_total{status_code="5xx"}[24h]))
sum(increase(aisix_usage_event_drops_total[24h]))
# One member's rate-limit rejections. `status` carries the raw HTTP code,
# so a single failure mode is addressable without scanning a whole family.
sum(increase(aisix_usage_events_emitted_total{user_id="<member-id>", status="429"}[24h]))
# Whose queue handoffs were rejected. Both counters carry the same model and
# provider-key labels, so the difference holds per model, not only in total.
sum(increase(aisix_usage_event_drops_total[24h])) by (model, provider_key_name)
Before concluding that two sources disagree, confirm they cover the same population:
- Scrape coverage. Request counters carry no environment label, so a per-environment figure from usage records is not comparable with a gateway-wide PromQL result unless every gateway instance serving that environment is scraped.
sum by (job, instance) (...)shows which instances contributed; compare that against the instances actually running. - Window coverage.
increase(...[24h])measures only the samples present in the range. If instances restarted partway through the window, or Prometheus retention is shorter than the window, the result covers less time than the usage-log filter does. Graphing the raw counter over the same range makes both gaps visible. - Delivery.
aisix_usage_event_drops_totalmeasures events rejected at the producer-to-worker queue handoff. After the queue accepts an event, delivery can still fail without incrementing this counter, so it does not bound the difference between emission attempts and records in final storage. Group queue drops bymodel. For later failures, monitortelemetry batch failed (events dropped)for control-plane delivery andsink delivery dropped after retriesfor exporter delivery. - Attempts that never left the gateway. The deployment families count upstream calls, so an attempt that produced no upstream call is deliberately absent from them: one the target's own rate limits refused, and one rejected while the request was still being assembled — an unusable credential, a missing
model_name, a missing or malformedapi_base. Those attempts still appear in the usage log and inaisix_usage_events_emitted_total. This is therefore one of the reasons a deployment attempt count runs below a usage-log attempt count, not the whole of it: the scope noted above is another (the deployment families cover only endpoints dispatched through a Model Group, while usage events also cover direct models), as are the scrape, window, and delivery gaps in this list. Isolate one contributor at a time rather than reading the whole difference as gateway-side rejections. The exclusion does have one unambiguous signature: a misconfigured target produces failed requests with no deployment failures at all, because the provider was never asked anything.
Collect Access Logs
Access logs describe an individual proxy request. AISIX writes them through the process logger to the standard error stream, so they appear in the container or process logs collected by your runtime.
Configure process logging in the startup configuration:
observability:
log_level: "info"
The RUST_LOG environment variable can override the configured log level.
One access log entry is written per proxy request, whether it succeeded, failed, or stopped early. It includes fields such as method, path, status, latency, provider, model, API key ID, request ID, token counts, and routing outcome when those values are available.
Its timing differs by response type, which determines which fields it can carry. A non-streamed request is written when the request ends, so its entry carries everything the gateway resolved. A streamed one is written when the response is opened, before the stream is consumed — so its entry has no token counts and no provider response ID, because neither exists yet. Usage events, described below, carry the streamed figures.
When the gateway has it by the time the entry is written, the entry also carries provider_request_id: the response object ID the provider returned, such as an OpenAI chat.completion.id, an Anthropic message id, or a Responses API resp_…. This is the ID a provider's own console and support channel index a call by. The field is omitted rather than blank whenever there is none to record, so you can filter on its presence. Besides the streamed case above, that covers a rejection before dispatch such as a guardrail block, a response served from cache, and the endpoints whose provider response carries no ID at all, including embeddings, audio, and image generation.
For the calls whose ID cannot reach the access log entry, the gateway emits a separate provider call completed entry, carrying request_id, attempt_index, attempt_kind, and provider_request_id. One is written per provider call that returned an ID — so a streamed response produces one, and a request that failed over part-way through a stream produces one per provider call it made. Join them to the access log entry by request_id, and tell one call apart from another within a retried or failed-over request by attempt_index.
Two further fields describe the connection a request arrived on rather than the request itself. They are attached to the request as a whole, so they appear on the access log entry, on the provider call completed entry, and on every diagnostic line the request emits in between:
| Field | What it records |
|---|---|
peer | The remote end of the accepted downstream connection, as <ip>:<port>. |
downstream_request_id | The correlation ID that arrived in the x-request-id request header — what reverse proxies, ingress controllers, and service meshes stamp on the requests they forward. |
Both are recorded rather than declared: a request that carried neither logs neither field instead of an empty one, so you can filter on presence.
peer is not the caller's IP address, and substituting one for the other loses the point of both. The gateway resolves the caller's address separately, following proxy.real_ip when it sits behind trusted proxies, and uses it for access control and usage records; that value carries no port. peer is the remote end of the TCP connection the request physically arrived on, and its port is the whole of its value. Behind a layer-4 load balancer, with the gateway on the host network, peer is the only field that joins a gateway log line to the fronting proxy's own record of the same connection.
A streamed response is the one case where that attachment could quietly lapse: its body is written after the point at which a request's log context normally ends. The streaming paths carry that context forward explicitly, so the lines written at the end of a stream — provider call completed among them — carry the same request_id, peer, and downstream_request_id as the rest of the request.
A caller that disconnects before the gateway sends response headers also produces an entry, with status 499 and error_kind="client_disconnected". It carries the model and provider the request had resolved by then, and no token fields — the request never reached a response to count tokens from. A request abandoned earlier still, before the model was resolved, carries neither. Requests recorded this way are also counted by aisix_proxy_client_cancelled_requests_total. A caller that disconnects mid-response is not recorded this way, because that request already has a normal status and a usage event.
A request refused for exceeding proxy.request_body_limit_bytes produces a second entry on the aisix::body_limit target, carrying declared_content_length, configured_limit_bytes, drained_bytes, and drain_outcome. Join it to the access log entry by request_id.
drain_outcome reports how the gateway finished reading the body it refused, which determines what the caller saw. completed means the caller sent everything it declared and could read the 413. cap_reached, timeout, and client_read_error mean the read stopped early, so the caller usually sees a closed connection instead. The first is logged at info; the other three are logged at warn and are rate limited to one entry per outcome per second, so use aisix_proxy_request_body_limit_rejections_total for volume.
The access_log field is currently reserved and has no effect. Proxy handlers still emit structured access logs, and there is no separate access-log format or sink setting. Collect the standard error stream with your runtime log pipeline when logs need to leave the gateway host.
Correlate Responses with Telemetry
Response headers provide caller-visible correlation and routing hints. They can identify the request, cache outcome, retry timing, or selected target on supported paths.
Use the request ID and other supported response headers to join a caller-visible response to access logs or exported records. For the header scope on each proxy route, see Headers and Error Codes.
Up to three IDs travel with a call, and none of them substitutes for another:
| ID | Assigned by | Use it to |
|---|---|---|
request_id | The gateway, one per request — or the caller, when it supplies one and AISIX adopts it (see below). Returned in the x-aisix-request-id response header. | Correlate the response with any gateway access logs and usage events the request produces, including exported records or entries on the dashboard's Logs page. It means nothing to the provider. |
downstream_request_id | Whatever sits in front of the gateway, in the x-request-id request header. Recorded on the request's log lines when one arrived; never returned to the caller. | Find the same request in the logs of your ingress controller, reverse proxy, service mesh, or CDN. |
provider_request_id | The provider, in its response body, when it sends one. Recorded per attempt on the usage event for that attempt, and on the log entries described above. | Locate the same call in the provider's own console or quote it to provider support. |
A caller reporting a problem normally has request_id, which the gateway returns on every response. Look that request up, read provider_request_id from the attempt that served it, and take that to the provider. A caller who kept the whole response body may be able to read the provider's ID out of it directly, but only on the endpoints that return one.
downstream_request_id is recorded, never adopted. Whether a caller-supplied ID becomes the gateway's own request_id is a separate decision, governed by proxy.request_id.accept_headers — which by default accepts only x-aisix-request-id. So a gateway behind an ingress normally logs two IDs that mean different things, and neither can be mistaken for the other. It is screened by the same rule as an adopted ID (see Accepted Values) and omitted when the header is absent or unusable.
Reuse Your Own Request ID
If your service already generates a request ID for the business call, send it and AISIX adopts it instead of generating one. That ID then becomes the request's identity everywhere: the x-aisix-request-id response header, the request_id on the access log and on every usage event the request produces, and the x-aisix-request-id the provider receives.
Every trail you use to investigate a request is then keyed by an ID your own application logs already carry. That covers the gateway access log, exported usage events, and AISIX Cloud Request Logs, with no second mapping to maintain.
Send it in x-aisix-request-id:
# AISIX_PROXY is the gateway origin; omit a trailing slash and endpoint path.
export AISIX_PROXY="YOUR_AISIX_GATEWAY_ORIGIN"
curl "$AISIX_PROXY/v1/chat/completions" \
-H "Authorization: Bearer $AISIX_API_KEY" \
-H "Content-Type: application/json" \
-H "x-aisix-request-id: req_abc123-orders-svc" \
-d '{"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Hello"}]}' -i
The response echoes the same value:
HTTP/1.1 200 OK
x-aisix-request-id: req_abc123-orders-svc
A request that retries or fails over emits one usage event per attempt. All carry your ID, so filtering on it returns the whole chain rather than only the attempt that succeeded.
Accepted Values
An ID is used as sent when it is 1–256 bytes and contains only visible ASCII characters (! through ~): no spaces, no control characters, no non-ASCII. Qualifying shapes include a UUID, a ULID, a prefixed ID such as req_abc123, and the hexadecimal ID nginx puts in $request_id.
A value outside that range is ignored and AISIX generates a UUID instead, which is the behavior when no ID is sent at all. The request itself is never rejected over its correlation ID.
AISIX does not require IDs to be unique. Sending the same ID for two different requests makes them indistinguishable in every trail that keys on it, so generate a fresh one per request.
Choose the Accepted Headers
By default AISIX reads only its own x-aisix-request-id. Set proxy.request_id.accept_headers to change that:
proxy:
request_id:
accept_headers: ["x-aisix-request-id", "x-request-id"]
Headers are consulted in the order listed, and the first acceptable value wins, so the list is also a priority order. Environment-only deployments set the list comma-separated as AISIX_PROXY__REQUEST_ID__ACCEPT_HEADERS.
x-request-id is not accepted by default on purpose. Reverse proxies, ingress controllers, and load balancers stamp that header on every request they forward. Enabling it behind one of those means the correlation ID comes from your infrastructure rather than from the calling service. Add it when AISIX is the first hop, or when the ID your proxy assigns is the one you want to trace by.
Accepting it is not what makes it visible, though. An acceptable x-request-id is recorded as downstream_request_id on the request's log lines either way, so the infrastructure's own ID is available to search on without changing this setting. Listing it here does something further: it makes that value the request's identity everywhere — the response header, the access log's request_id, and every usage event — at which point request_id and downstream_request_id hold the same value, which is the honest report of what happened.
Set accept_headers: [] to ignore caller-supplied IDs entirely and always generate one.
A header name that is not a valid HTTP header name fails startup rather than being silently skipped.
Export Usage Events
Usage events are per-attempt records emitted by supported proxy paths. A request that retries or fails over emits multiple events with the same request_id, ordered by attempt_index. Counting these records therefore counts attempts, not requests — see Count Requests and Attempts Separately before comparing a record count against a request metric. Each event includes its outcome, consumption details, the model alias the caller requested, and the resolved model that served the attempt when the gateway can observe those values. A response served from the response cache records the matching layer in cache_hit_layer (exact or semantic) and, for semantic hits, the matched similarity in cache_similarity.
Latency fields distinguish provider time from caller-visible time:
| Field | Scope |
|---|---|
upstream_latency_ms | Time spent on one upstream attempt. It excludes request parsing, guardrails, routing, retry delays, and earlier attempts. |
upstream_ttft_ms | Time from the start of an upstream attempt until its first streamed frame of any type — metadata openers such as response.created or a role-only chat delta included, matching what caller-side proxies measure. It is omitted or zero for non-streaming requests, errors, and cache hits. |
downstream_latency_ms | Total time the caller waited for the request. It includes gateway processing, retries and retry delays, and any output holdback, and appears only on the terminal attempt. |
A caller that disconnects part-way through a streamed response still produces a usage event, because the upstream did the work and may have charged for it. That event carries status 499 rather than 200, and its token counts cover only what arrived before the disconnect. Count streamed successes with status_code = 200 to keep abandoned responses out of the total.
Tell Request Kinds Apart
Every event carries operation, the kind of work the request asked for. Without it, one field says which protocol addressed the gateway and another says which model answered. Neither says whether the call was a conversation, an image, or a video. Every OpenAI-compatible endpoint reports the same inbound_protocol, so a chat completion, an image generation, and a video submission arrive as one undifferentiated stream.
The value comes from the endpoint the request matched, so it is a small fixed set safe to index, group, and chart:
| Value | Endpoint |
|---|---|
chat | /v1/chat/completions |
messages | /v1/messages |
count_tokens | /v1/messages/count_tokens |
responses | /v1/responses |
completions | /v1/completions |
embeddings | /v1/embeddings |
rerank | /v1/rerank |
image_generation | /v1/images/generations |
image_edit | /v1/images/edits |
transcription | /v1/audio/transcriptions |
translation | /v1/audio/translations |
speech | /v1/audio/speech |
video_generation | POST /v1/videos |
realtime | /v1/realtime |
files, batches, fine_tuning | The file, batch, and fine-tuning management endpoints |
batch_completion | The gateway's own accounting of a finished batch job, recorded when the job completes rather than when it was submitted |
mcp, a2a | The MCP and A2A gateways |
passthrough | A passthrough route |
Three points are worth knowing before you write a query against it:
- It describes the request, not the outcome. A request that failed, or that a guardrail refused, carries the same value a successful one would. It is the only field on such a record that names the endpoint at all.
- It is request-scoped. A request that retries or fails over emits one event per attempt, and every attempt carries the same value, so counting events per operation counts attempts. See Count Requests and Attempts Separately.
- Polling a video job is not video generation. Only the submission (
POST /v1/videos) produces a usage event; retrieving the job's status or downloading its result does not. A count ofvideo_generationis therefore a count of videos asked for, not of requests made about them.
The field is part of the record's metadata, so it is present under both content modes. An exporter configured for metadata_only — which never receives a prompt — can still separate traffic by kind, which content inspection could not do there at all.
In Alibaba Cloud SLS, the field arrives as its own column and needs no parsing. Enable analytics for the fields a query names, since SQL analysis reads indexed fields only:
* | SELECT operation, COUNT(*) AS calls, SUM(prompt_tokens + completion_tokens) AS tokens GROUP BY operation ORDER BY calls DESC
To pull one kind of traffic, filter on it directly — operation: video_generation — instead of matching a model name or searching the prompt. A model name is a poor substitute here. One model can serve several endpoints, and callers address models through aliases and model groups, so requested_model answers "which configured entry," not "what kind of call."
Attribute Events to a Member
Each event carries user_id: the organization member who owned the API key the request authenticated with, assigned on the key itself. It is absent for keys that belong to no member — ownership is set explicitly when a key is created or edited, not inferred from whoever created it.
One member usually holds several credentials, and this field is what makes them one identity. A JWT-authenticated request runs as the API key its identity resolves to, so a member calling through both an API key and an OIDC token produces events under two different api_key_id values and the same user_id. Filtering on the member returns the whole picture; filtering per key returns a slice of it.
The value is a snapshot taken when the request ran, not a lookup performed when you read the event. Reassigning a key to a different member changes who later requests are attributed to and leaves earlier events attributed to the previous owner, which is what makes a historical query answerable. Deleting the key does not erase the attribution of the events it already produced.
Events written before a data plane that records this field carry no member, so a member filter covers traffic from that upgrade forward.
In the dashboard, Logs offers this as the Member filter, alongside a Status filter that accepts a family (4xx), an exact code (429), or a range (500-599). Combining the two answers questions like "which of this member's requests were rate-limited in the last 24 hours" in one query. The CSV export carries user_id and the member's name.
Usage events are consumed through a sink rather than read from a local endpoint. Configure an observability exporter to deliver them to OTLP/HTTP, object storage, Alibaba Cloud SLS, or Datadog.
Next Steps
Configure Observability Exporters to send usage events to an external collector, log destination, object store, or warehouse workflow. Use the Metrics Reference when building Prometheus dashboards or alerts.