Usage Reporting
Use the Usage view to compare request volume, token consumption, and spend across managed gateway environments and models. It also summarizes organization-wide semantic-cache savings and helps you investigate unexpected spend or confirm that usage records are reaching the control plane.
How Usage Is Reported
A managed gateway serves AI traffic in your runtime environment and reports usage events to the control plane. It derives the telemetry endpoint from the control-plane URL and sends usage data to the fixed /dp/telemetry path. Operators do not configure a separate destination for control-plane usage reporting.
Each event can include request status, latency, token usage, and cost. It also distinguishes the model alias requested by the caller from the resolved model that served an attempt, which helps explain routed and ensemble traffic. Streaming chat requests can report time to first token.
Token counts come from the provider response whenever it reports them. For streaming requests, the gateway also asks OpenAI-compatible upstreams to include usage in the final stream chunk. Some responses still end without any usage data — for example, an OpenAI-compatible relay that never reports usage, a client that disconnects mid-stream, or an upstream error after a partial response. In those cases the gateway counts the tokens itself with a local tokenizer. It counts the input from the request it sent and the output from the content it actually delivered. The usage event still ships, so the request stays visible to spend, budgets, and rate-limit accounting instead of recording zero. Locally counted events are marked as estimated, and the Request Logs view shows an estimated badge on those rows. Estimated counts closely match OpenAI-family models and are approximations for models with proprietary tokenizers. Provider-reported values always take precedence when present.
Interpret Usage
The Usage view summarizes managed gateway traffic across environments over a rolling 30-day window. All totals and tables use this same window.
| Signal | What It Helps Explain |
|---|---|
| Request count | Traffic volume and changes in demand. |
| Token totals | Input and output consumption reported by providers. |
| Spend | Cost calculated from token counts and matching model pricing. |
| Cache savings | Upstream input and output token consumption avoided by semantic-cache hits. |
| Usage by environment | Which deployment environment generated traffic and spend. |
| Top models | Up to 10 environment and caller-requested model alias combinations, ranked by spend. Older records without a requested-model value fall back to the resolved model. |
| Top API keys | Up to 10 environment and caller API key combinations, ranked by spend. |
Token counts appear when the provider response reports them, and are otherwise estimated locally by the gateway (see How Usage Is Reported). Spend also requires a price that matches the event's (provider, model name) pair. Usage rounds spend to two decimal places, so low-volume traffic can show $0.00 even when a matching price exists.
Use Model Pricing to add or inspect a price, and use Request Logs to inspect event or attempt cost at six decimal places. Pricing changes apply only to events recorded after the change.
Usage and Budget Enforcement
The control plane uses managed gateway usage records to evaluate budgets. Each budget defines its scope, spending limit, period, and enforcement mode.
The rolling 30-day Usage window is independent of a budget's configured period, so their totals can cover different time ranges.
When a hard-stop budget is exceeded, the managed gateway can reject matching requests with HTTP 429. Warn-only budgets surface the over-budget state without blocking traffic.
If a budget rejection is unexpected, check the returned budget scope, the configured limit, and the caller API key bindings that determine which team or member budgets apply. For the complete enforcement path, see Budgets.
Troubleshoot Missing or Unexpected Usage
Check the reporting path in order:
- Confirm that the request used the managed gateway attached to the expected environment. Traffic through another environment in the same organization appears under that environment. Traffic in another organization appears in that organization's Usage view.
- Check that the request completed and appears in Request Logs. A newly completed request can take a few seconds to appear because gateways flush telemetry in batches.
- Confirm that the managed gateway has a recent heartbeat.
- Confirm that the gateway can reach the control-plane telemetry endpoint.
- If token counts appear without spend, check Model Pricing for an exact provider and model-name match.
During a temporary control-plane outage, live traffic can continue with the latest projected configuration. Telemetry batches that fail during the outage are dropped instead of retried, so affected usage records can remain missing after connectivity returns. New records resume after the connection recovers. Exporter health, heartbeat, and fresh budget decisions also require control-plane connectivity. Budget checks can temporarily reuse a cached decision before applying the configured fail mode; see Availability and Caching.
Next Steps
Continue with Logging and Auditing to investigate individual requests or control-plane resource changes. Configure Model Pricing when usage contains tokens but cannot derive spend.