Metrics and Logs

AISIX AI Gateway exposes aggregate metrics, per-request logs, response headers, and exportable usage events. Together, these signals show service health and connect a caller-visible response to the model, route, and policy outcome that produced it.

Choose a Telemetry Source

Start with the source that matches the operational question, then correlate signals by request, model, or provider when an issue needs deeper investigation.

Operational Question	Start With	What It Provides
Is traffic healthy across gateway instances?	Prometheus metrics	Request rates, latency distributions, token and cost counters, policy outcomes, routing health, cache behavior, and exporter delivery health.
What happened to one request?	Access logs	Structured request fields, including status, latency, model, provider, request ID, and routing outcome when available.
What can the calling application observe?	Response headers	Request correlation, cache outcome, retry timing, and selected-target hints on supported routes.
Where can request records be stored, analyzed, or used for accounting?	Usage events	Per-attempt outcome and consumption records delivered through an observability exporter.

Scrape Prometheus Metrics

Prometheus metrics are suited for monitoring traffic trends, latency, token counters, cost counters, rate-limit outcomes, cache behavior, and exporter delivery health.

AISIX serves Prometheus metrics on the dedicated metrics listener at /metrics by default. Change the path or disable the endpoint through the startup observability settings.

This endpoint is unauthenticated by design. Keep the dedicated metrics listener private.

Configure Prometheus exposure in the startup configuration:

config.yaml

observability:
  metrics:
    prometheus:
      enabled: true
      path: "/metrics"

The dedicated listener binds to 0.0.0.0:9090 by default. Set a different listener address when Prometheus should scrape another interface or port.

Scrape the default metrics endpoint:

curl -sS "http://127.0.0.1:9090/metrics"

Traffic metrics appear after activity

AISIX publishes configuration status on every scrape. Other metric families are registered on first observation, so traffic metrics might not appear immediately after startup. Send one model request, then check again for series such as aisix_requests_total and aisix_tokens_consumed_total.

AISIX emits native metric names with the aisix_ prefix. Use the histogram series for latency percentiles across gateway instances and the request counters for success-rate and routing analysis. For exact metric names, label scope, and PromQL examples, see Metrics Reference.

Count Requests and Attempts Separately

A request and an upstream attempt are different units, and mixing them is the most common source of apparently contradictory numbers. One client request can place several upstream calls: a same-target retry, or a failover to the next target in a Model Group. AISIX counts both units, in different places.

Unit	Where it is counted	What one sample or row means
Request	aisix_proxy_requests_total, aisix_llm_requests_total	One client request, with the status the caller received.
Attempt	aisix_deployment_requests_total and the deployment families	One upstream call to one target model.
Attempt	aisix_usage_events_emitted_total	One emitted usage event, which AISIX emits per attempt.
Attempt	Usage-event records, and the usage log built from them	One attempt, joined to its siblings by request_id and ordered by attempt_index.

The consequence worth internalizing: a failed attempt that a fallback rescued is invisible in the request counters. The caller received a 200, so the request counter records status="200" and nothing else. The 502 the first target returned lives in the deployment counters and in the usage log, as its own attempt.

So a gateway fronting one broken target in an otherwise healthy Model Group can legitimately show tens of thousands of 5xx rows in the usage log while sum(increase(aisix_proxy_requests_total{status=~"5.."}[24h])) returns a few hundred. The log is counting attempts that failed; the metric is counting callers who saw a failure. Neither is wrong, and they are not expected to converge.

Use these queries for the questions they actually answer:

# Requests that ended in a server error — what callers experienced.
sum(increase(aisix_proxy_requests_total{status=~"5.."}[24h]))

# Requests that ended in a server error even after failing over.
sum(increase(aisix_proxy_requests_total{status=~"5..", is_fallback="true"}[24h]))

# Upstream attempts that failed, by target — including the ones a fallback
# rescued. This is the attempt-level view of a 5xx usage-log row count,
# restricted to the endpoints that dispatch through a Model Group.
sum(increase(aisix_deployment_failure_responses_total[24h])) by (model)

# Fallbacks that rescued a request, by group and by the target reached.
sum(increase(aisix_routing_successful_fallbacks_total[24h])) by (model, fallback_model)

# Usage events AISIX intended to emit, and those a full or closed queue lost.
sum(increase(aisix_usage_events_emitted_total{status_code="5xx"}[24h]))
sum(increase(aisix_usage_event_drops_total[24h]))

Before concluding that two sources disagree, confirm they cover the same population:

• Scrape coverage. Request counters carry no environment label, so a per-environment figure from usage records is not comparable with a gateway-wide PromQL result unless every gateway instance serving that environment is scraped. sum by (job, instance) (...) shows which instances contributed; compare that against the instances actually running.
• Window coverage. increase(...[24h]) measures only the samples present in the range. If instances restarted partway through the window, or Prometheus retention is shorter than the window, the result covers less time than the usage-log filter does. Graphing the raw counter over the same range makes both gaps visible.
• Delivery. Usage records reach storage through an exporter, and an overloaded queue drops events rather than blocking request handling. aisix_usage_event_drops_total bounds that loss: records can fall short of the emitted counter, never exceed it.
• Attempts that never left the gateway. The deployment families count upstream calls, so an attempt that produced no upstream call is deliberately absent from them: one the target's own rate limits refused, and one rejected while the request was still being assembled — an unusable credential, a missing model_name, a missing or malformed api_base. Those attempts still appear in the usage log and in aisix_usage_events_emitted_total. This is therefore one of the reasons a deployment attempt count runs below a usage-log attempt count, not the whole of it: the scope noted above is another (the deployment families cover only endpoints dispatched through a Model Group, while usage events also cover direct models), as are the scrape, window, and delivery gaps in this list. Isolate one contributor at a time rather than reading the whole difference as gateway-side rejections. The exclusion does have one unambiguous signature: a misconfigured target produces failed requests with no deployment failures at all, because the provider was never asked anything.

Collect Access Logs

Access logs describe an individual proxy request. AISIX writes them through the process logger to the standard error stream, so they appear in the container or process logs collected by your runtime.

Configure process logging in the startup configuration:

config.yaml

observability:
  log_level: "info"
  access_log: true

The RUST_LOG environment variable can override the configured log level.

One access log entry is written per proxy request, whether it succeeded, failed, or stopped early. It includes fields such as method, path, status, latency, provider, model, API key ID, request ID, token counts, and routing outcome when those values are available.

Its timing differs by response type, which determines which fields it can carry. A non-streamed request is written when the request ends, so its entry carries everything the gateway resolved. A streamed one is written when the response is opened, before the stream is consumed — so its entry has no token counts and no provider response ID, because neither exists yet. Usage events, described below, carry the streamed figures.

When the gateway has it by the time the entry is written, the entry also carries provider_request_id: the response object ID the provider returned, such as an OpenAI chat.completion.id, an Anthropic message id, or a Responses API resp_…. This is the ID a provider's own console and support channel index a call by. The field is omitted rather than blank whenever there is none to record, so you can filter on its presence. Besides the streamed case above, that covers a rejection before dispatch such as a guardrail block, a response served from cache, and the endpoints whose provider response carries no ID at all, including embeddings, audio, and image generation.

For the calls whose ID cannot reach the access log entry, the gateway emits a separate provider call completed entry, carrying request_id, attempt_index, attempt_kind, and provider_request_id. One is written per provider call that returned an ID — so a streamed response produces one, and a request that failed over part-way through a stream produces one per provider call it made. Join them to the access log entry by request_id, and tell one call apart from another within a retried or failed-over request by attempt_index.

A caller that disconnects before the gateway sends response headers also produces an entry, with status 499, error_kind="client_disconnected", and no model or token fields. The gateway resolves those fields while handling the request, so they are unavailable for a request that ends this early. Requests recorded this way are also counted by aisix_proxy_client_cancelled_requests_total. A caller that disconnects mid-response is not recorded this way, because that request already has a normal status and a usage event.

A request refused for exceeding proxy.request_body_limit_bytes produces a second entry on the aisix::body_limit target, carrying declared_content_length, configured_limit_bytes, drained_bytes, and drain_outcome. Join it to the access log entry by request_id.

drain_outcome reports how the gateway finished reading the body it refused, which determines what the caller saw. completed means the caller sent everything it declared and could read the 413. cap_reached, timeout, and client_read_error mean the read stopped early, so the caller usually sees a closed connection instead. The first is logged at info; the other three are logged at warn and are rate limited to one entry per outcome per second, so use aisix_proxy_request_body_limit_rejections_total for volume.

The access_log field is reserved in the current release. Proxy handlers still emit structured access logs, and there is no separate access-log format or sink setting. Collect the standard error stream with your runtime log pipeline when logs need to leave the gateway host.

Correlate Responses with Telemetry

Response headers provide caller-visible correlation and routing hints. They can identify the request, cache outcome, retry timing, or selected target on supported paths.

Use the request ID and other supported response headers to join a caller-visible response to access logs or exported records. For the header scope on each proxy route, see Headers and Error Codes.

Two IDs travel with a call, and neither substitutes for the other:

ID	Assigned by	Use it to
request_id	The gateway, one per request — or the caller, when it supplies one and AISIX adopts it (see below). Returned in the x-aisix-request-id response header.	Find the request in gateway access logs, usage events, and the dashboard's Logs page. It means nothing to the provider.
provider_request_id	The provider, in its response body, when it sends one. Recorded per attempt on the usage event for that attempt, and on the log entries described above.	Locate the same call in the provider's own console or quote it to provider support.

A caller reporting a problem normally has request_id, which the gateway returns on every response. Look that request up, read provider_request_id from the attempt that served it, and take that to the provider. A caller who kept the whole response body may be able to read the provider's ID out of it directly, but only on the endpoints that return one.

Reuse Your Own Request ID

If your service already generates a request ID for the business call, send it and AISIX adopts it instead of generating one. That ID then becomes the request's identity everywhere: the x-aisix-request-id response header, the request_id on the access log and on every usage event the request produces, and the x-aisix-request-id the provider receives.

Every trail you use to investigate a request is then keyed by an ID your own application logs already carry. That covers the gateway access log, exported usage events, and AISIX Cloud Request Logs, with no second mapping to maintain.

Send it in x-aisix-request-id:

# AISIX_PROXY is the gateway origin; omit a trailing slash and endpoint path.
export AISIX_PROXY="YOUR_AISIX_GATEWAY_ORIGIN"

curl "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer $AISIX_API_KEY" \
  -H "Content-Type: application/json" \
  -H "x-aisix-request-id: req_abc123-orders-svc" \
  -d '{"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Hello"}]}' -i

The response echoes the same value:

HTTP/1.1 200 OK
x-aisix-request-id: req_abc123-orders-svc

A request that retries or fails over emits one usage event per attempt. All carry your ID, so filtering on it returns the whole chain rather than only the attempt that succeeded.

Accepted Values

An ID is used as sent when it is 1–256 bytes and contains only visible ASCII characters (! through ~): no spaces, no control characters, no non-ASCII. Qualifying shapes include a UUID, a ULID, a prefixed ID such as req_abc123, and the hexadecimal ID nginx puts in $request_id.

A value outside that range is ignored and AISIX generates a UUID instead, which is the behavior when no ID is sent at all. The request itself is never rejected over its correlation ID.

AISIX does not require IDs to be unique. Sending the same ID for two different requests makes them indistinguishable in every trail that keys on it, so generate a fresh one per request.

Choose the Accepted Headers

By default AISIX reads only its own x-aisix-request-id. Set proxy.request_id.accept_headers to change that:

config.yaml

proxy:
  request_id:
    accept_headers: ["x-aisix-request-id", "x-request-id"]

Headers are consulted in the order listed, and the first acceptable value wins, so the list is also a priority order. Environment-only deployments set the list comma-separated as AISIX_PROXY__REQUEST_ID__ACCEPT_HEADERS.

x-request-id is not accepted by default on purpose. Reverse proxies, ingress controllers, and load balancers stamp that header on every request they forward. Enabling it behind one of those means the correlation ID comes from your infrastructure rather than from the calling service. Add it when AISIX is the first hop, or when the ID your proxy assigns is the one you want to trace by.

Set accept_headers: [] to ignore caller-supplied IDs entirely and always generate one.

A header name that is not a valid HTTP header name fails startup rather than being silently skipped.

Export Usage Events

Usage events are per-attempt records emitted by supported proxy paths. A request that retries or fails over emits multiple events with the same request_id, ordered by attempt_index. Counting these records therefore counts attempts, not requests — see Count Requests and Attempts Separately before comparing a record count against a request metric. Each event includes its outcome, consumption details, the model alias the caller requested, and the resolved model that served the attempt when the gateway can observe those values. A response served from the response cache records the matching layer in cache_hit_layer (exact or semantic) and, for semantic hits, the matched similarity in cache_similarity.

Latency fields distinguish provider time from caller-visible time:

Field	Scope
upstream_latency_ms	Time spent on one upstream attempt. It excludes request parsing, guardrails, routing, retry delays, and earlier attempts.
upstream_ttft_ms	Time from the start of an upstream attempt until its first streamed frame of any type — metadata openers such as response.created or a role-only chat delta included, matching what caller-side proxies measure. It is omitted or zero for non-streaming requests, errors, and cache hits.
downstream_latency_ms	Total time the caller waited for the request. It includes gateway processing, retries and retry delays, and any output holdback, and appears only on the terminal attempt.

A caller that disconnects part-way through a streamed response still produces a usage event, because the upstream did the work and may have charged for it. That event carries status 499 rather than 200, and its token counts cover only what arrived before the disconnect. Count streamed successes with status_code = 200 to keep abandoned responses out of the total.

Usage events are consumed through a sink rather than read from a local endpoint. Configure an observability exporter to deliver them to OTLP/HTTP, object storage, Alibaba Cloud SLS, or Datadog.

Next Steps

Configure Observability Exporters to send usage events to an external collector, log destination, object store, or warehouse workflow. Use the Metrics Reference when building Prometheus dashboards or alerts.