Release Notes

These release notes summarize user-visible changes to the AISIX gateway, control plane, dashboard, and deployment packages. Releases are listed newest first.

Release artifacts include the gateway image at docker.io/api7/aisix, control-plane images under docker.io/api7/aisix-cp-*, the aisix-cp Helm chart, and an offline installation package.

1.0.0

Release date: September 4, 2026

This release makes guardrail behavior visible and configurable at the point where operators need to tune it. Semantic guardrails report the similarity they measured, the console can test a saved rule before it is attached to production traffic, and fail-open decisions are recorded consistently across the gateway. The release also applies one client-header forwarding contract to model routes, MCP servers, passthrough routes, and Realtime, improves reasoning controls across provider protocols, and makes routing and configuration failures easier to diagnose.

Behavior Changes

• Model cooldown is now opt-in. A direct model with no cooldown block, or with a block whose enabled field is omitted, is no longer taken out of rotation after request failures. Set cooldown.enabled: true to retain the previous behavior; the default timeout and trigger settings inside an enabled block are unchanged. This is a breaking behavior change for existing models that relied on cooldown without enabling it explicitly.

• Guardrails no longer scan the Files API as one opaque blob. Upload bodies, upload responses, and file downloads under /v1/files now pass without input or output guardrail evaluation. The former whole-file scan could neither evaluate individual JSONL records nor write masked content back to the multipart body; it also decoded binary files with replacement characters. The request envelopes on /v1/batches and /v1/fine_tuning/jobs remain guarded; per-record file screening is not part of this release.

• A body the gateway cannot scan now follows the guardrail that governs that side. The gateway refuses unreadable content only when at least one guardrail in scope both reads that side of the exchange and fails closed on it. A chain that does not read that side does not cause a refusal, and the request continues with an unscannable_body record when every relevant guardrail is fail-open. The fail_open setting now applies to keyword and pii guardrails as well as remote guardrails. Held-back streamed output remains fail-closed when nothing scannable can be recovered, because the alternative would release buffered bytes that no guardrail evaluated.

• Audio transcription and translation responses that cannot be decoded now honor the output failure policy. With an output guardrail that fails closed, such a plain-text transcript returns 422 content_filter with guardrail_unavailable; with a fail-open guardrail, the original bytes are relayed and the bypass is recorded. Transcript content that can be decoded is still scanned.

• Unsupported streaming requests fail before reaching the provider. A request with stream: true to /v1/completions or /v1/images/generations now returns 400 invalid_request_error. These endpoints do not relay streaming responses; previously they sent the request upstream, could repeat billable work through the retry budget, and eventually returned a decode-related 502.

• Guardrail scope now treats reasoning by who supplied it. Reasoning content replayed by the caller is input and is scanned on Chat Completions, Responses, and Anthropic Messages, while reasoning generated by the model remains outside output scanning. Anthropic signed thinking blocks are inspected by block actions but are not rewritten by mask actions, because clients must replay those signed bytes unchanged.

• Streaming error events now use their endpoint's native envelope. Guardrail errors on /v1/messages streams use Anthropic's invalid_request_error type, and /v1/responses emits flat Responses API error events with top-level code, message, param, and sequence_number fields. HTTP error envelopes are unchanged.

• New release images use immutable full-version tags. Stable releases publish :X.Y.Z, :latest, and :sha-*; release candidates publish their full RC tag and SHA tag. New :X.Y and :X aliases are no longer created. Existing abbreviated tags remain in the registries but are not advanced by this or later releases.

New Features

• Semantic guardrails can be measured before and after deployment. The edit form has a dry-run panel that embeds probe text and shows the deny/allow score, configured threshold, closest example, and resulting verdict. Requests evaluated by a semantic guardrail add guardrail_scores to usage events, including requests that pass. The Logs page shows the score, threshold, direction, closest-example index, and embedding model; the usage API and CSV export carry the same record. Durable usage data contains the example index, never the example or screened text.

• Client-header forwarding now spans Provider Keys, MCP servers, passthrough routes, and Realtime. MCP servers and passthrough routes gain forward_client_headers; the existing Provider Key setting now also reaches Realtime and all translated provider paths. On MCP and standard model endpoints it is an allowlist; on passthrough routes it restores a header the route would otherwise strip. Because the field can pass caller credentials and trace context, credential slots, including the AWS SigV4 request-identity headers, and traceparent / tracestate require an exact name and cannot be selected by a glob. An exactly forwarded credential replaces the credential the gateway or default_headers would otherwise put in that slot, never adds a second value. Amazon Bedrock remains the exception: its signer owns authorization and the SigV4 headers and drops supplied values before signing.

• A direct model can normalize reasoning-effort values for its upstream. The new effort_mapping object performs one exact, case-sensitive string-to-string lookup after the final direct target is selected. Unlisted or absent values pass through unchanged, the mapping never adds a missing effort, and a mapped result is not looked up again. It covers Chat Completions, Responses, Anthropic Messages and token-count requests, native and cross-provider paths, streaming requests, Bedrock invocation modes, and direct targets selected through routing, semantic, or ensemble models. The setting is accepted only on direct models and is editable in the console.

Improvements

• Semantic guardrails require a threshold chosen for their embedding model. A non-empty deny_examples list now requires deny_threshold, and a non-empty allow_examples list requires allow_threshold; a direction with no examples requires no value. Existing rows that omitted a threshold are filled with 0.75, preserving what they already enforced. The Dashboard test panel and the per-request scores provide the measurements needed to tune a replacement value.

• Fail-open outcomes are visible on every guarded proxy surface. Success and error usage events carry guardrail_bypassed_reason, including a bypass that occurs before an individual guardrail executes. The aisix_guardrail_bypasses_total metric now includes reason="unscannable_body"; aisix_guardrail_blocks_total also includes fail-closed refusals raised before a timed guardrail execution. A2A and usage-less rerank or unsupported-capability paths emit zero-token usage events when needed to preserve guardrail attribution.

• Guardrail configuration failures are reported as configuration state. Rows that deserialize but cannot build at runtime now appear in /status/config, in rejection metrics, and in managed heartbeat status, and disappear after the row is repaired. Writes reject JSON null in PII, Presidio, keyword, and semantic configuration where the gateway could otherwise drop the whole guardrail. The upgrade migration repairs only stored nulls and missing values with safe defaults, validates the complete projected row, and leaves other malformed rows unchanged.

• Buffered output guardrails cover complete SSE frames. The Messages and Responses paths now handle CRLF framing, multi-line data: payloads, missing final terminators, and non-SSE JSON replies to streaming requests. Content that cannot be structurally masked is removed or refused instead of being released unread, while maskable multi-line frames retain their complete payload.

• Reasoning controls survive protocol translation. Anthropic output_config.effort and adaptive/disabled thinking map to the corresponding OpenAI reasoning_effort; OpenAI effort values map to Anthropic's current output_config.effort form. Anthropic structured-output declarations are also translated to OpenAI response_format with a strict JSON schema instead of being discarded.

• Routing and metrics expose more of the work the gateway performed. A routing target removed for cooldown or background health now produces a throttled warning naming the target and reason, and failed-attempt warnings are consistent across Chat Completions, Messages, token counting, and Responses. Native and translated Responses streams now report both TTFT metric families, and ensemble requests use one consistent usage estimate for quota, usage events, and token metrics.

• The console preserves resource references instead of asking operators to retype them. Existing-resource fields use searchable selectors, fields that may legitimately contain a new value keep editable suggestions, and a stale reference remains visible and is not silently cleared by an unrelated edit. In Logs, typing a requested-model filter remains a case-insensitive substring search, while selecting a known model performs case-sensitive equality in both the feed and CSV export.

Fixes

• Response model names are consistent on rerank and Realtime. A Jina-shaped /v1/rerank response and session.created / session.updated Realtime events now report the gateway model name the caller addressed rather than the provider's model id. A full Realtime session object sent back in session.update is translated to the upstream id again without changing a separately configured transcription model.

• Unsupported provider capabilities no longer spend the retry budget. Missing completions, embeddings, or image-generation support is classified explicitly and returns the existing 501 response without retrying an adapter capability that cannot change between attempts.

• Wildcard model references are validated before they become inert or widen access. Renaming a referenced model into a wildcard alias is refused when it would corrupt an exact-name reference or silently turn an API key's single-model grant into a namespace grant. A semantic guardrail cannot newly select a wildcard alias as its embedding model. A cache policy may select a concrete model name served by a wildcard alias, but cannot select the wildcard pattern itself. Previously stored references are validated again only when the referenced value changes.

API Changes

The OpenAPI structural comparison reports no breaking shape changes. However, several validation changes below are semantic breaking changes: a request that was previously accepted can now return 400. Callers should review them as well as the additive fields.

Endpoint	Change
GET /environments/{env_id}/models GET /environments/{env_id}/models/{model_id}	Model representations add optional effort_mapping and document cooldown.enabled: false as the default.
POST /environments/{env_id}/models PATCH /environments/{env_id}/models/{model_id}	Direct models accept optional effort_mapping, and successful responses return it when configured; null or {} clears it on update, while non-direct kinds reject it. An omitted cooldown block or enabled value means cooldown is off. PATCH rejects a rename into a wildcard alias while an exact-name reference or API-key grant would be corrupted.
POST /environments/{env_id}/guardrails PATCH /environments/{env_id}/guardrails/{guardrail_id}	A semantic example list requires its matching threshold; semantic embedding models cannot newly name a wildcard alias; JSON null is rejected in the affected semantic, PII, Presidio, and keyword config fields. A full replacement config on PATCH must include every threshold required by its example lists.
POST /environments/{env_id}/cache_policies PATCH /environments/{env_id}/cache_policies/{cache_policy_id}	A model:<name> selector may name a concrete address served by a wildcard alias, but the selector itself cannot be a wildcard pattern. Existing values are validated again only when changed.
POST /provider_keys PATCH /provider_keys/{provider_key_id}	Existing request.forward_client_headers and request.default_headers validation now accepts credential slots, rejects transport-breaking and x-aisix-* names, and applies the exact-match and collision rules described above. Some formerly accepted but inert header entries must be removed before a full request-overrides block can be saved again.
GET /environments/{env_id}/passthrough_routes POST /environments/{env_id}/passthrough_routes GET /environments/{env_id}/passthrough_routes/{passthrough_route_id} PATCH /environments/{env_id}/passthrough_routes/{passthrough_route_id}	Requests and responses add optional forward_client_headers; null clears it on update.
GET /mcp_servers POST /mcp_servers GET /mcp_servers/{mcp_server_id} PATCH /mcp_servers/{mcp_server_id} POST /mcp_server_submissions PATCH /mcp_server_submissions/{mcp_server_id}	Requests and responses add optional forward_client_headers. POST /mcp_servers/{mcp_server_id}/approve and /reject also return the field as part of the MCP server representation.
GET /environments/{env_id}/usage_events	Adds optional boolean query parameter requested_model_exact; response items may include guardrail_scores.
GET /environments/{env_id}/usage_events/export	Adds optional boolean query parameter requested_model_exact; exported items gain guardrail_scores.

Upgrade Notes

• Cooldown opt-in is a breaking behavior change. Before upgrading, set cooldown.enabled: true on every direct model that should retain the pre-1.0 implicit cooldown behavior.
• Treat /v1/files as outside guardrail coverage in this release. Do not rely on a whole-file keyword, PII, or remote-guardrail decision for uploads or downloads.
• Review the hooks and failure policy on every guardrail. A body that AISIX cannot scan and that a relevant fail-closed guardrail governs is still refused, but one that no relevant fail-closed guardrail governs can now continue. In particular, fail_open: true now takes effect for keyword and pii guardrails and can newly forward a request or response that AISIX cannot scan.
• Clients that replay reasoning content can now be blocked by input guardrails. Model-generated reasoning is no longer evaluated by output guardrails, and a mask action does not rewrite Anthropic signed thinking blocks.
• Handle 422 content_filter when transcription or translation response bytes cannot be decoded and an output guardrail fails closed. With fail-open, the original bytes can now be relayed without content evaluation.
• A stream: true request to /v1/completions or /v1/images/generations now returns 400 invalid_request_error without contacting the provider, instead of eventually returning a decode-related 502. Update status-code handling and retry policies.
• Update custom stream parsers for native guardrail error envelopes: Messages streams use Anthropic's invalid_request_error, while Responses streams put code, message, param, and sequence_number at the top level.
• Review stored guardrails containing JSON null before upgrade. A safely repairable row that the previous gateway skipped can begin enforcing after the migration; a semantic row missing only a threshold keeps its existing 0.75 behavior.
• API clients that create or replace semantic guardrail configs must include the threshold corresponding to every non-empty example list. The threshold and null checks are semantic breaking Admin API changes even though the OpenAPI shapes are not structurally breaking.
• Review automation that renames models or writes model references. A rename into a wildcard alias is rejected when it would corrupt an exact reference or widen an API key grant; semantic guardrails cannot newly use a wildcard alias as their embedding model; cache policies may name a concrete model served by an alias but not the wildcard pattern itself.
• Review Provider Key forward_client_headers patterns used with Realtime: 1.0.0 starts honoring the setting on that surface. Credential slots and traceparent / tracestate must be named exactly; broad globs do not select them. Remove rejected transport, x-stainless-*, anthropic-version, or x-aisix-* entries before saving a full overrides block. On Bedrock-compatible endpoints, a forwarded caller value now wins a collision with default_headers in the x-api-key, api-key, x-goog-api-key, proxy-authorization, and cookie slots. Bedrock's signer still owns authorization and the AWS SigV4 headers and discards caller-supplied values.
• Update image references or automation that expected a newly published :X.Y or :X alias to use the full version or :latest instead.
• If an alert is based on aisix_guardrail_bypasses_total, account for the new reason="unscannable_body" series and the additional events now included in the total.

0.13.0

Release date: September 1, 2026

This release is about telling traffic apart and reaching an upstream on its own terms. A Provider Key can now declare which API surfaces its endpoint serves natively and where each one lives, so one credential reaches both the OpenAI-compatible path and the Anthropic-compatible path an upstream serves, without a second key or a second model. Every usage record now names the work that was asked for, so an image generation and a video submission are no longer the same undifferentiated OpenAI stream in an exporter or in Logs. A caller the gateway rate-limits gets a response that says what the cap was and when to come back. And a response now reports the gateway model name the caller addressed on every endpoint that serves one, rather than the provider's own id on some of them.

Behavior Changes

• Existing DeepSeek provider keys reach DeepSeek's own routes after the upgrade, instead of being translated. AISIX now carries a verified declaration of the API surfaces DeepSeek serves — its native /v1/responses and its Anthropic-compatible /anthropic/v1/messages — and applies it once, at upgrade, to provider keys that have no declaration of their own, use the deepseek provider, and still point at https://api.deepseek.com. Those keys report apis_source: catalog. What changes for traffic: a Responses API request is forwarded to DeepSeek's own /v1/responses rather than translated to chat completions, so reasoning output items survive; and an Anthropic-wire request reaches the Anthropic-compatible path, so prompt-cache breakpoints and thinking blocks survive. A key you declared yourself is left untouched.

• A client reading the model field off a response now sees the gateway model name it asked for, on /v1/messages, /v1/responses, /v1/completions and /v1/embeddings — streamed and buffered alike — and on /v1/videos polls. These paths previously returned the provider's own id, so a model aliased gpt4o-mini came back as gpt-4o-mini-2024-07-18, while /v1/chat/completions already returned the alias. Passthrough routes are unchanged and still relay the provider's response as-is.

• A gateway rate-limit rejection now carries response headers. A 429 the gateway itself produces describes the single limit that refused it: x-ratelimit-limit, x-ratelimit-remaining, x-ratelimit-reset, x-ratelimit-scope and Retry-After. x-ratelimit-reset and Retry-After are both a delta in seconds. A concurrency rejection now carries a retry hint where it previously carried none. These headers are added only when the gateway refuses the request; a 429 relayed from an upstream is passed through as the upstream sent it. The per-dimension x-ratelimit-limit-requests / -tokens / -concurrent headers on successful responses are unchanged.

• A Prometheus gauge whose key no longer exists stops reporting its last value. aisix_budget_limit_usd / _spent_usd / _remaining_usd / _reset_seconds / _details_present and aisix_ratelimit_remaining_{requests,tokens} are written only from the request path, so deleting an API key used to freeze its series at whatever it last read — and an alert on budget exhaustion kept firing for a credential that no longer exists. Such a series is now retired to NaN, which Prometheus treats as no value rather than as zero. An alert that compares these gauges no longer matches a deleted key. If you built a dashboard panel that reads a raw value, expect a gap rather than a flat line after a key is deleted.

New Features

• Native API surfaces on a Provider Key. One upstream account often exposes more than one protocol, on different paths of the same host, under one credential — DeepSeek, Zhipu and Kimi each front an OpenAI-compatible path and an Anthropic-compatible one. api_base can name only one of them, so everything else was translated, losing what the target protocol carries and the canonical chat shape does not. A Provider Key now takes an apis block naming each surface and, optionally, the base URL it lives at:

  {
    "apis": {
      "responses": {},
      "messages": { "base": "https://api.deepseek.com/anthropic" }
    }
  }

  The two surfaces resolve on different terms, because the evidence for them differs. messages is additive: a key whose adapter is already anthropic keeps serving /v1/messages natively whatever the map says, and listing it here adds the route to a key whose adapter is something else. responses is authoritative: once the block exists, /v1/responses is served natively only if it is listed — which is how an operator says "this endpoint has no Responses route" and gets the request translated to chat completions instead of 404'd upstream. With no block at all, both fall back to what the gateway inferred before: /v1/messages from the vendor id or the anthropic adapter, /v1/responses from the vendor id alone. Surfaces the block has no key for — embeddings, audio, images, videos, files, batches, fine-tuning, rerank — always use api_base, exactly as before. Set the block to {} to declare that the endpoint serves nothing beyond its adapter's own surface, or to null to clear the declaration. Configurable in the console on the Provider Keys page, and over the Cloud Admin API on POST and PATCH /provider_keys.

• Usage records name the operation that was asked for. Every usage event now carries operation — the kind of work a request asked for, chosen by the route it matched and never by caller text: chat, completions, messages, count_tokens, responses, embeddings, rerank, realtime, image_generation, image_edit, transcription, translation, speech, video_generation, files, batches, batch_completion, fine_tuning, mcp, a2a and passthrough. inbound_protocol reports openai for every OpenAI-shaped route, so nothing could previously separate a text chat from an image generation or a video submission short of a regular expression over a captured prompt — which a metadata_only exporter has no prompt for, and which a zero-token video submission does not answer at all. The field reaches your own exporters (OTLP, Datadog, SLS) and the console: Logs filters on the operation and marks each request with it — the conversational endpoints (chat, messages, responses, completions) are left unmarked as the common case — and GET /usage_events and GET /usage_events/export take an operation query parameter and return the field on each item.

Improvements

• Usage-event and budget metrics name the member, not just their id. aisix_usage_events_emitted_total, aisix_usage_event_drops_total and the five aisix_budget_* gauges now carry user_name beside user_id, matching aisix_proxy_requests_total and the aisix_llm_* families. A usage-event alert or a budget dashboard can name the person it is about without a lookup outside Prometheus.

Fixes

• aisix_budget_remaining_usd is now published. The gateway could only emit this gauge when the control plane stated the remaining amount in its budget decision, and the decision never carried it — so a series the metric reference documents existed in no 0.12.0 or earlier deployment, and a dashboard panel or alert built on it matched nothing. It now appears alongside aisix_budget_limit_usd and aisix_budget_spent_usd, on the same labels and at the same moments. Upgrading the control plane is enough; no gateway change and no configuration.
• The sign-in error for a rejected origin no longer misstates what the server trusts, or points you at the wrong component to restart. It said the server trusts only the one address it is configured to serve on; it also trusts that address's localhost/127.0.0.1 twin and every origin in AISIX_TRUSTED_ORIGINS, and the message now offers that second remedy. It also told the operator to restart cp-api, where the check that rejected them runs in the dashboard.
• The Helm chart's post-install notes no longer name an address the port-forward they tell you to run does not serve, and carried the same overstated claim about trusted origins.

API Changes

No breaking changes. All additions are optional.

Endpoint	Change
POST /provider_keys	New optional request property apis. The 201 response may now carry warnings when a data plane in scope runs a gateway too old to execute a declared surface.
PATCH /provider_keys/{provider_key_id}	New optional request property apis; null clears the declaration, {} declares that the endpoint serves nothing beyond its adapter's surface. Response adds apis, apis_source and warnings.
GET /provider_keys/{provider_key_id}	Response adds apis, apis_source (catalog or operator) and warnings.
GET /environments/{env_id}/usage_events	New optional operation query parameter; each item gains operation.
GET /environments/{env_id}/usage_events/export	New optional operation query parameter; each exported item gains operation.

Note that apis is not echoed by the create response, the same as api_base — read it back with GET /provider_keys/{provider_key_id}, whose apis_source reports catalog when AISIX supplied the declaration rather than you.

Upgrade Notes

• The control plane applies the curated API-surface declaration once, on the first boot after the upgrade, to the provider keys described under Behavior Changes. It is best-effort and idempotent: a boot that cannot complete it retries on the next one, and a key that already carries a declaration is never touched. Set apis yourself on any key whose endpoint you do not want reached natively.
• If you alert on aisix_budget_* or aisix_ratelimit_remaining_*, review the expressions against the gauge-retirement change above before upgrading.

0.12.0

Release date: August 30, 2026

This release makes two contracts say what they mean. The Cloud Admin API now covers everything the console does, ships as an immutable document per release, and validates what it documents. And a guardrail now governs exactly what you attached it to — the old "attached to nothing means everywhere" rule is gone from the gateway, the console, and the upgrade backfill alike, without changing what any existing guardrail enforces. Alongside that, token counts are reported in the protocol the caller asked in, thinking tokens on Gemini models are counted at last, usage records name the person behind the credential, and several ways a delete could leave a live reference behind are closed.

Behavior Changes

• A guardrail's scope is now exactly its attachments; one attached to nothing inspects no traffic. A guardrail with no attachment rows used to be applied to the entire environment at the lowest priority. Keying on the absence of rows is what made that dangerous, because narrow scoping is expressed by deleting the environment attachment: a guardrail scoped to a single model silently widened to every request in the environment the moment that model was deleted. Upgrading preserves what your guardrails enforce today — the control plane converts the old implicit environment scope into an explicit environment attachment as part of the upgrade, before the gateway stops honoring the implicit one. It also returns guardrails that a previous boot had wrongly widened to environment scope back to the narrow scope you configured. What changes from here on is that a new guardrail governs nothing until you attach it. Declaring an unattached guardrail is still not an error: its scope target may simply have been deleted. In the console such a guardrail now reads Not attached — "Attached to nothing, so it inspects no traffic" — where it used to read Global / "Applies to all models", and the gateway logs a warning naming any enabled guardrail that nothing attaches. In the same vein, opening a narrowly scoped guardrail in the console to change something unrelated and pressing Save no longer writes an environment-wide attachment: the editor now represents "attached to nothing" as its own state instead of defaulting to environment scope.
• Twenty-one control-plane routes the dashboard already used are now part of the Admin API contract, and requests to them are validated. They were reachable with an admin token but had no published schema; they now appear in the API reference. Bringing a live route into the contract also turns request validation on for it, so a request body carrying a field the schema does not declare is rejected with 400 instead of being accepted and ignored — POST /teams with a stray field returned 201 before this release and returns 400 additional properties not allowed now. If you have automation calling the usage feed, its export, the usage summary or request-level metrics, teams or team members, model price overrides, invitations, member removal, or the notification delivery log, confirm it sends only documented fields. The routes are listed under API Changes.
• Token counts in a response are now reported in the accounting the caller's own protocol uses. The two protocols disagree on exactly one thing — whether prompt-cache counters live inside the input count or beside it. On the OpenAI protocol prompt_tokens is the full input with cached tokens included; on the Anthropic protocol the input count excludes them and cache_creation_input_tokens / cache_read_input_tokens sit beside it. The gateway used to copy through whichever shape the upstream reported, so a client speaking one protocol in front of an upstream speaking the other was answered in the other protocol's accounting: an OpenAI-protocol client over an Anthropic upstream saw prompt_tokens: 40 alongside total_tokens: 2104, arithmetic that does not add up and a cache hit reported nowhere. The same request now reports 2088 and 2104 with the hit under prompt_tokens_details.cached_tokens. Where that conversion happens, total_tokens is recomputed from the projected input and output counts rather than echoing a total that was computed under the other accounting; where no conversion is needed, the total the upstream reported is passed through unchanged. Streaming, non-streaming and /v1/responses are all covered. The recorded usage event is unchanged, so billed and historical cost are unaffected — only the counts reported back to the caller move.
• OpenAI-protocol responses can now carry a cache-write count. prompt_tokens_details.cache_creation_tokens (and input_tokens_details on /v1/responses) reports prompt tokens the provider wrote into its cache this turn. It is not an OpenAI field — OpenAI has no cache-write concept — but without it an Anthropic or Bedrock cache write is an unexplained rise in prompt_tokens the caller cannot price, and a write is billed above the plain input rate. It is emitted only when non-zero, so a strict client that rejects unknown fields should be checked.
• Thinking tokens on Gemini models are now counted, and billed. Gemini reports thoughtsTokenCount, which the gateway parsed nowhere — so a thinking model's completion_tokens was recorded short by its thinking tokens and reasoning_tokens read 0 on every protocol. Both are now carried, with reasoning_tokens naming the subset of completion_tokens, and inclusiveness decided from the total the upstream itself reports, so nothing is double-counted where Gemini already folds thinking tokens into its own candidate count. Cost charges reasoning_tokens at the reasoning rate, or the completion rate where no reasoning rate is set. If you run thinking Gemini models, recorded completion tokens and cost will rise — to match what Google already bills you for and the gateway was not recording. Unlike the protocol conversion above, this one does change the usage record.
• /v1/messages/count_tokens now emits a usage record. The route emitted none on any outcome, so a request that an input guardrail refused was correctly refused and then impossible to find in the logs — including in the Guardrail blocks view. It still bills nothing: prompt_tokens and completion_tokens are zero on both the answered and the refused path. "Not metered" and "not recorded" were being treated as one statement; they are two.
• A delete that would leave a live reference behind is now refused, repaired, or reported. Deleting a model named as a semantic guardrail's embedding model returns 409, and renaming that model rewrites the guardrail rather than orphaning it. The stakes were high, because guardrails fail closed: before this release that delete succeeded, and every request in the guardrail's scope then answered 422 semantic_embed_unresolved. Deleting an API key that a claim mapping resolves to returns 409 as well, matching how a passthrough route's anonymous key is already protected — a dangling mapping fails closed but is not free, because matching takes the highest-priority rule without falling through, so a broken one masks a lower-priority rule that still works. Deleting a team or removing a member now clears the ownership it leaves on their caller keys and publishes that to the gateway. Both columns were cleared by a database foreign key, which emits no projection event, so the gateway kept reading a document naming a team that no longer exists — and a rate limit scoped to the deleted team went on throttling those keys with nothing in the console to explain the 429s. Deleting an MCP server no longer strips the per-key rate limits that name it: it used to drop those while keeping the tool grants, so re-creating a server under the same name — a supported path — restored the permissions and silently lost the limits. Deleting an OIDC provider still succeeds — both of its carriers hold a name, a rename is a delete plus a re-create, and dangling fails closed and self-heals — but the response now reports how many API keys and claim mappings still name it.

New Features

• The Cloud Admin API is published per release. Every release tag now publishes an immutable OpenAPI document generated from that tag's own source, and the API reference serves a page per released version together with a generated changelog between adjacent versions. You can read the API your deployment exposes and diff it against the version you are upgrading to, rather than against a document that tracks development.
• Usage records and logs are attributed to the org member, not only to the credential. A usage record named the API key a request arrived on but never the person behind it, so "show me this member's errors in the last 24 hours" meant enumerating every key they own and merging the results by hand — and a member calling through both an API key and OIDC was split across two identities no single filter could join. Usage records now carry the member and the Logs page filters by member. The attribution is a snapshot taken when the request is served: re-assigning or deleting a key does not rewrite the history it already produced.
• Logs can be filtered by an exact status code. Filtering previously offered status families only, so there was no way to ask for 429 without also getting every other 4xx. Exact codes and ranges such as 500-599 are now accepted.
• aisix_usage_events_emitted_total and aisix_usage_event_drops_total carry the member and the exact status. Both gain a user_id label and a status label holding the response code itself, so "how many of this member's requests were rate limited" and "did this member lose any usage records" are answerable from metrics. The existing status_code family label is unchanged, so alerts built on it keep working — but a dashboard that aggregates these two counters without naming its labels will see the new dimensions.

Improvements

• The usage page no longer reports zeros while it is still loading. Before the first response arrived, /usage rendered $0.00, 0 requests and 0 tokens under two tables reading "No usage in the last 30 days.", then replaced it all seconds later. On a slow connection an operator was told, definitely, that their gateway had seen no traffic. It now shows — with a "Loading…" caption, matching the overview page.
• The offline installer reports the version it installed and the port it actually published. An offline deployment had no way to confirm on the way out which release it had just installed, and when the port came from the environment rather than the default, the closing banner still printed the default — so an install listening on one port told the operator to open another.
• The reference now spells out how a custom PII rule's replacement text behaves. $1 is not expanded as a capture-group reference, and a pattern that has capture groups replaces only the first group rather than the whole match — so replacement: "ACCT-$1-****" against ACCT-([0-9]{4})-([0-9]{4}) yields ACCT-ACCT-$1-****-5678, which looks redacted while leaving the original in place. The console now warns when a replacement contains $.

Fixes

• Two control-plane replicas booting at the same time no longer crash each other. Schema migration was check-then-act throughout — probe a catalog, then create or alter — which PostgreSQL does not serialize, so a high-availability install could fail a pod on startup across several distinct error codes. Boot-time schema work is now serialized; eight replicas started simultaneously against a fresh database all come up, where roughly seven in eight used to fail.
• The upgrade backfill no longer re-widens a model-scoped guardrail. The backfill that gives pre-attachment guardrails an environment-scoped attachment ran on every control-plane boot and identified its targets as "has no environment attachment" — a predicate written when that could only mean "predates attachments". Once scoping a guardrail to a model began deleting its environment attachment, every restart quietly widened it back to the whole environment.
• Deleting a guardrail now retracts its attachments from the gateway, and upgrading cleans up the ones earlier deletes left behind. The attachment rows were removed by a database cascade, which emits no projection event, so the attachment documents stayed live and every gateway logged an unknown-guardrail warning on each index rebuild, indefinitely. Upgrading retracts those leftovers, and the warning stops on its own — no manual cleanup is needed.

API Changes

Twenty-one routes enter the contract this release. They are newly documented, not newly added — each was already reachable with an admin token; what changes is that they now have a published schema and their requests are validated (see Behavior Changes).

Route	Change
GET /environments/{env_id}/usage_events	Newly documented
GET /environments/{env_id}/usage_events/export	Newly documented
GET /environments/{env_id}/usage_metrics	Newly documented
GET /environments/{env_id}/usage_summary	Newly documented
GET /invitations POST /invitations DELETE /invitations/{invitation_id}	Newly documented
DELETE /members/{user_id}	Newly documented
GET /model_pricing PUT /model_pricing DELETE /model_pricing/{id}	Newly documented
GET /notification_deliveries	Newly documented
GET /teams POST /teams GET /teams/{team_id} PATCH /teams/{team_id} DELETE /teams/{team_id}	Newly documented
GET /teams/{team_id}/members POST /teams/{team_id}/members PATCH /teams/{team_id}/members/{user_id} DELETE /teams/{team_id}/members/{user_id}	Newly documented
DELETE /environments/{env_id}/api_keys/{api_key_id}	Adds 409 — the key is resolved by a claim mapping
DELETE /environments/{env_id}/oidc_providers/{oidc_provider_id}	200 response adds an optional warnings property reporting API keys and claim mappings that still name the provider
DELETE /environments/{env_id}/models/{model_id}	409 was already documented; it now also covers a model named as a semantic guardrail's embedding model

No previously documented route changed its request or response shape in a breaking way. Four semantics that are easy to guess wrong are now stated explicitly, and are worth checking your integration against:

• usage_summary counts distinct requests, not upstream attempts — a request that retried across three attempts counts once.
• ModelPricing.source can be snapshot on an offline deployment, not only the values a connected deployment produces.
• Six of the usage filters are case-insensitive substring matches; provider_label and request_id match exactly.
• Caller API key ownership takes the membership identifier from GET /members, not the member's user identifier — a correction to what the previous reference said. The user identifier is rejected.

Upgrade Notes

• Guardrails need no action. The scope change preserves what your guardrails enforce today: the upgrade writes the implicit environment scope out as an explicit environment attachment, and returns any guardrail a previous boot had wrongly widened to its configured narrow scope. It also retracts attachment records that earlier deletes left behind, so the gateway's unknown-guardrail warnings stop on their own. Guardrails created after the upgrade govern only what you attach them to.
• If you call the control-plane API directly, check the twenty-one newly contracted routes. Requests to them are now validated against a published schema, so an undocumented field that used to be accepted and ignored is rejected with 400.
• Expect recorded cost to rise on thinking Gemini models. Their thinking tokens were not being counted at all; they now appear in completion_tokens and reasoning_tokens and are charged. Budgets and cost alerts calibrated against the previous, understated figures should be reviewed. No other model or provider is affected.
• Check dashboards that aggregate aisix_usage_events_emitted_total or aisix_usage_event_drops_total. Both gained a user_id and a status label, which changes the cardinality of an unlabelled sum by (...). The status_code family label is unchanged.

0.11.0

Release date: August 28, 2026

This release is about guardrails. Screening gains two new kinds — one that decides by meaning rather than by pattern, and one that runs a script you write — and the whole subsystem becomes accountable: a guardrail that refuses now says whether your policy matched or your rule could not run, and every enforced hit reaches the logs and the usage record. Alongside that, a gateway rolls over without dropping traffic, prompt-cache usage is billed and measured correctly on the Anthropic protocol, and several ways an operator could quietly lose data or ship a non-upgrade are closed.

Behavior Changes

• Input guardrails now run on every request that reaches a provider. Previously the chain was consulted only where a request carried text the gateway had already extracted, so a request with no text — an argumentless MCP tool call, a transcription with no prompt, an image edit with no prompt, an embedding of an empty string — reached the upstream without being screened, and /v1/messages/count_tokens and /a2a/:agent were never screened at all. What varies per route is the text offered to the chain, not whether the chain is consulted. If you rely on a guardrail to gate egress, it was not covering these paths before this release. A body the Anthropic parser cannot read is now refused rather than forwarded, when a guardrail is attached.
• A guardrail refusal is now recorded on the usage event that reports it. The flag the Logs Guardrail blocks view filters on was set by only two of the fourteen surfaces that can refuse, so that view could read empty while callers were being refused — which looks like the gateway recording no guardrail activity at all. Every surface now sets it.
• A guardrail that cannot run no longer claims your content was blocked. When a screening rule fails — an unreachable provider, a script that returned something that is not a verdict, a timeout — the refusal is now reported as one. The caller gets 422 with "code": "guardrail_unavailable" and a message naming the guardrail and the cause (request rejected: guardrail 'x' could not evaluate it (...)), instead of the request blocked by content policy a real match returns. The gateway logs a warning, and aisix_guardrail_latency_seconds carries error_type set to the cause. The HTTP status and error.type are unchanged, so an alert on those keeps counting both. This applies to every guardrail kind on every endpoint. aisix_guardrail_blocks_total counts both, as it always has — it carries no labels, so it cannot separate them, and it is recorded on the chat-completions path only. To count policy blocks, use aisix_guardrail_latency_seconds_count{result="blocked", error_type="none"}; the same series with error_type != "none" gives the rule failures. Do not select on error_type alone — that histogram records every guardrail execution, allowed requests included.
• Guardrails fail closed by default. fail_open now defaults to false on both planes: a rule that cannot reach its provider refuses the request rather than passing it through unscreened. Set fail_open: true per guardrail to keep the old behavior. The same default governs a script fault.
• Content larger than a provider's limit is split, not clipped. An over-limit body used to be truncated and screened partially, which is a silent hole in a control that exists to see everything. It is now split into chunks and screened whole — every chunk is submitted, with no cap on how many, because a chunk budget would be unscanned content through the back door.
• The mandatory flag is gone. It duplicated what fail_open already expresses. Remove it from stored configurations; a guardrail that must not be bypassed is one with fail_open: false.
• The removed /passthrough/* tunnel now answers a plain 404. 0.10.0 replaced the implicit tunnel with explicit passthrough routes and left a 410 Gone migration pointer for one release. That release has shipped, so unclaimed /passthrough/* paths take the router's ordinary miss path and the namespace is entirely yours to claim with explicit routes. The accompanying warning log and the provider="unresolved" metric series are gone with it.
• /v1/messages reports a prompt-cache hit from an OpenAI upstream. When an OpenAI-compatible upstream serves an Anthropic-protocol request, the cached prefix is no longer folded into input_tokens: the client receives input_tokens = non-cached input and cache_read_input_tokens = the hit. The usage record does not convert: it keeps the OpenAI shape, where prompt_tokens is still the whole prompt and cached_prompt_tokens is the cached subset of it rather than a separate amount to add — and that is what lets Logs, the Usage API, CSV export and billing price the cached portion at the cache-read rate. A streamed response reports it in the closing message_delta. The counter is emitted only when non-zero. Previously the whole prompt billed at the uncached rate.
• Twelve metric families gained an upstream_protocol label. It names the protocol that actually served the request, which a cross-protocol conversion made invisible before. Aggregating queries are unaffected; dashboards, alerts and recording rules that match labels exactly will need updating.
• Stopped gateways read as offline. Node liveness is now decided by the control plane rather than in the browser, and DpNode carries a status field (healthy / warning / offline) derived from heartbeat age. A stopped instance no longer renders as healthy until the page is reloaded, and the Data planes view updates on its own.
• GET /environments/{env_id}/mcp_policy returns 200 with a null policy when an environment has none, instead of 404. A 404 from that endpoint now means only that the environment does not exist.
• A notification channel's URL can no longer be destroyed by writing it back. Read responses mask the URL, because the path of a webhook is its credential. That masked value used to be accepted as a write, so any read-modify-write silently replaced the real webhook and returned 200. An echoed mask is now a no-op, a mask naming a different host is rejected, and creating a channel with a masked URL is refused. Delivery errors no longer echo the remote response body, which could contain the URL.

New Features

• Semantic guardrails. A screening rule can now decide by meaning instead of by pattern: give it example phrases to allow and to deny, and it scores each message against them with an embedding model you nominate, with separate thresholds per side. It screens each message individually and can be pointed at user messages or the whole conversation.
• Custom script guardrails. For a policy the built-in kinds cannot express, write the check yourself: a script the gateway runs in a sandbox, with a timeout you set and secrets you register by name rather than inline. The hook returns {action: "none"} (or "allow") to pass, {action: "block", reason, reason_code} to refuse, or {action: "mask", segments} to rewrite. Those four are the whole vocabulary — anything else is a script fault, reported as one.
• PII patterns can scope to a capture group and choose their replacement. A custom pattern can redact just the part of a match that is sensitive, and name the string that replaces it, instead of masking the whole match with a fixed token.
• Enforced guardrail hits reach the logs and the usage record. Every LLM handler now records which guardrail acted on a request, so a blocked or masked call is attributable after the fact rather than only visible as a refusal. The Logs view distinguishes a policy block from a fail-closed outage.
• validate reports guardrails that load but cannot run. A row whose configuration parses but does not build — an invalid regular expression, an unknown detector or action value, a kind: custom script that does not compile — used to be dropped from the chain with only a warning in the log, leaving the screening it describes silently absent. It is now reported at validation time instead of at the first request. The check builds the configuration in process and makes no network calls, so it reports nothing about whether a provider is reachable, and it skips kind: semantic rows entirely — the embedding model such a row names is not checked.
• A gateway retires HTTP/2 connections when it starts draining. On SIGTERM the gateway sends GOAWAY to HTTP/2 downstreams as the drain window opens, so a client stops sending on that connection and reconnects elsewhere instead of racing the shutdown. HTTP/1.1 already received Connection: close. A draining gateway also stays live to its health probe, so an orchestrator does not kill it mid-drain.
• POST /v1/images/edits. Multipart image editing is now a typed endpoint with the same authentication, guardrails and usage accounting as the rest of the proxy.
• Prompt-cache token counters and upstream_protocol on the metrics. Cache reads and cache writes are now their own Prometheus counters, so the cached share of input is answerable from metrics instead of only from usage exports.
• The gateway's trace id is stored with each request and can be interpolated into a trace-viewer link via {trace_id} in trace_ui_url_template, so a log line leads to the trace it belongs to.

Improvements

• Renaming an environment, and editing an embedding model, are now possible from the Dashboard.
• Every list footer accepts a page number, instead of only step-by-step paging.
• An invitation is now something a new user can actually accept: the link previews the organization before joining, joining is an explicit action rather than a side effect of opening the page, and the invitation is bound to the invited address.
• A read-modify-write of a guardrail's configuration round-trips. The fields describing a stored secret are accepted unchanged and rejected when altered, so scripting and infrastructure-as-code no longer have to strip them by hand. Editing a Bedrock guardrail no longer demands the AWS secret the API never returns.
• The offline package refuses to start over an existing data volume when .env is missing, naming what would be lost, rather than regenerating secrets and crash-looping.
• AISIX_CLOUD_NOTIFY_ALLOW_PRIVATE_URLS, AISIX_CLOUD_MCP_SPEC_ALLOW_PRIVATE_URLS, AISIX_CLOUD_ALLOW_FRESH_BOOTSTRAP and AISIX_CLOUD_PRICESYNC_URL can now be set in the offline package's .env; previously they were documented but unreachable.
• Usage is bucketed by UTC day whatever timezone the session is in, request totals are exact rather than sampled, and estimated usage is disclosed as estimated.
• Ordinary Dashboard browsing no longer exhausts its own authentication budget. A throttled token refresh now reads as "retry in a moment", not as being signed out — previously a signed-in user was told to sign in, and several other flows reported a failed session read as a lost session.
• Overview, Logs, Budgets and Observability no longer render partial or misleading figures while data is still loading.
• The documented on-premises upgrade now actually moves the version. AISIX_VERSION is owned by the package and reconciled on every start; previously it stayed pinned in .env, so following the documented steps left the old images running and reported success. Pin a version with the image variables instead. A downgrade is refused unless AISIX_ALLOW_DOWNGRADE=1.
• A control plane that fails to start now names the cause — wrong database password, wrong master key, mismatched key id, missing CA root — reading both the api and dpm logs, instead of leaving only "container is unhealthy".

Fixes

• /v1/realtime works against OpenAI again. The gateway sent a beta opt-in header on every realtime connection; OpenAI's now-GA endpoint rejects it and closes the session before the client sends anything. The header is now forwarded only when the caller asks for it, so a legacy client still gets the beta shape and everyone else reaches the GA API.
• kind: custom guardrails can be created from the Dashboard. The gateway did not advertise the kind in its capability list, so the Dashboard offered it as unsupported even though both planes could run it.
• An OpenAI provider key with no api_base falls back to the vendor default on every route, not just some.
• A no-op update to a provider key — sending back what is already stored — succeeds instead of failing with an internal error.
• Non-UTF-8 multipart prompt fields are rejected on every surface that accepts them.
• Per-item PII action overrides survive the Dashboard's edit form.
• API key rows written before the layered MCP rule keep loading. The retired mcp_access.mode selector is ignored as an ordinary unknown field, so the row does not fail and the rest of the key's mcp_access block is read normally — a rejected API key row would stop that key authenticating all of its traffic, not just its MCP traffic.
• Static OTLP exporter blocks that were never wired no longer log a false success.

Upgrade Notes

• Upgrade the control plane first, then your gateways. This is the supported order. During the window, a gateway still on 0.10.0 will not run the new guardrail kinds (semantic, custom), the PII replacement field, or the new text_source values: it cannot parse the row and drops it whole, so the rule screens nothing there. The control plane still saves such a configuration — it does not refuse it — and returns a warnings array on the write naming the field, the version that first reads it, and how many of your gateways are affected. Read those warnings: until every gateway is upgraded, the rule is not in force on the ones that are behind.
• Re-enter any notification channel webhook URL that was destroyed before this upgrade. If you ever updated a channel through the API by reading it and writing the object back, the real webhook URL was replaced by its masked form and cannot be recovered — it exists only at the provider. After upgrading, a channel in that state reports it in its delivery error; re-enter the URL from Slack (or your provider) to restore delivery. Channels only edited through the Dashboard are unaffected.
• Re-check what your guardrails were actually covering. Two fixes in this release change the answer: the input chain now runs on paths it previously skipped, and a refusal is now recorded on every surface that can refuse. If a guardrail looked quiet before, that may have been the gap rather than the traffic.
• Update metric queries that match labels exactly. Twelve families gained upstream_protocol.
• A query that reads aisix_guardrail_blocks_total as policy volume is counting rule failures too. That counter has no labels and never separated the two, and it is recorded on the chat-completions path only — as is aisix_guardrail_bypasses_total. With fail_open now defaulting to false, a guardrail whose provider is unreachable refuses the request rather than passing it, so failures that used to land in the bypasses counter land in the blocks counter instead. For policy-block volume across the gateway, use aisix_guardrail_latency_seconds_count{result="blocked", error_type="none"}.

0.10.0

Release date: August 20, 2026

This release is about agent traffic. Forward-proxy traffic — an IDE or coding agent pointed at the gateway — is now carried by an explicit route resource you can audit instead of an implicit tunnel that guessed which credential to borrow. MCP gains the 2026-07-28 revision of the protocol, anonymous access for clients that carry no credential, OAuth 2.1 discovery, and a single coherent rule for which tools a caller may reach. Model groups gain consistent hashing and priority tiers. The gateway keeps serving through a drain window instead of refusing connections a load balancer has not yet stopped sending, and traces exported over OTLP form a real hierarchy.

Behavior Changes

• The implicit passthrough tunnel is replaced by explicit routes. /passthrough/<provider>/<rest> no longer resolves by provider name; an unclaimed /passthrough/* path answers 410 Gone with endpoint_removed and logs a warning naming the caller, so un-migrated clients are findable. To migrate, create a passthrough route with path_prefix: /passthrough/<provider>, target_url set to the old API base, and provider_key_id set to the key it used to borrow — client URLs then keep working byte for byte. Two behaviors differ from the old tunnel: a route is a single-attempt relay, so upstream 5xx and transport retries are gone, and because a route resolves no model, model cooldown is no longer cross-marked from this traffic. Access is an explicit grant on the caller key (allowed_routes); a key without one reaches no route.
• MCP tool access is now one rule across three layers. The environment policy, the caller's team policy, and the key's own mcp_access block each carry allow and deny lists. Allow lists intersect and deny lists combine, so no layer can widen what another restricts; a layer that is not set imposes no constraint, and a caller with no layer at all has no MCP access. mode is gone from both policies and keys, allowed_tools is gone from the key — the key's layer is mcp_access.allow — and the legacy-key migration flow is removed. If you have configured MCP tool policies, re-check them after upgrading: the resulting access is recomputed under these rules and can differ from what the old settings produced. Two changes matter most: a team policy now narrows the environment grant instead of replacing it, so it can no longer grant a tool the environment does not; and a key with no mcp_access block now follows the policy layers instead of standing outside them. allow is required on every layer, so a layer that only means to subtract must spell its allow side ["*"].
• A guardrail that blocks an MCP tool call now answers as a tool error. The call returns success at the protocol level with isError: true on the result and a message naming the guardrail — never the matched content — instead of a JSON-RPC -32600. A calling agent sees the refusal as tool output it can adapt to, rather than a broken transport. A client matching on error.code == -32600 must read result.isError instead.
• The weighted routing strategy and the sticky flag are gone. Weight is now a per-target attribute available under every strategy: round_robin is smooth weighted round robin, so equal or absent weights reproduce the previous declaration-order cycle exactly and unequal weights hold exact proportions. Session affinity is now its own strategy, consistent_hash. Deployments managed through AISIX Cloud are migrated automatically. Declarative configurations must be updated: weighted becomes round_robin with weights kept, and weighted with sticky becomes consistent_hash. A stored configuration still naming weighted fails to load rather than changing meaning silently. Cohort assignments reshuffle once when moving to consistent hashing, since it builds its ring differently.
• An unsupported MCP protocol version is now refused inside the JSON-RPC envelope. A request whose MCP-Protocol-Version header names a revision the endpoint does not serve gets 400 with a JSON-RPC error carrying the supported list, instead of a bare text/plain response outside every gateway envelope. The endpoint serves 2025-03-26, 2025-06-18, 2025-11-25 and 2026-07-28; it no longer advertises 2024-11-05, which only existed for the HTTP+SSE transport.
• Traces exported over OTLP now form a hierarchy. A model request that reaches an upstream produces an HTTP SERVER span, a logical client span covering every retry and failover, and one child span per upstream attempt, replacing one flat span per usage event. An exported request trace with no upstream dispatch has only the SERVER span; protocols without per-attempt tracking have the SERVER span and one client span. Anything keyed on "one span per request" — span-count alerts, name-based counts — will see structural spans appear. The attempt span carries the full attribute set and is identifiable by aisix.attempt_index; structural spans carry a smaller correlation attribute set that includes aisix.request_id, and the SERVER span is span kind 2. Separately, the caller's traceparent and tracestate are no longer forwarded to model providers; an operator's explicit default_headers entry for a trusted upstream still works.
• Text-to-speech latency is now measured to the first byte. /v1/audio/speech streams the synthesized audio as it arrives instead of buffering it, so a player can start on the first bytes and the reported latency is time to first byte rather than time to last byte. This is the convention every other streaming surface here already follows.

New Features

• Passthrough routes. A route binds a gateway entry — a path prefix, an inbound Host allowlist, or both — to one upstream target, so an IDE or coding agent can be pointed at the gateway and have its traffic audited without changing the client. Host matching runs before routing, so forward-proxy traffic delivered with its original Host can claim a path the gateway also uses; path matching runs as the router's fallback, so a route can never shadow the gateway's own API. Callers authenticate with a gateway key, with the key in a header you name so Authorization stays free for the upstream credential, or anonymously as a bound principal gated by a source-CIDR allowlist. The upstream credential is either injected from a provider key or forwarded from the caller after the route removes its gateway credential and fixed header set. An optional device-injected identity header is recorded as the request's end-user identity for per-employee attribution. The request envelope is detected per request — chat, Responses, completions, or opaque — and drives guardrail text extraction, audit capture and token accounting; traffic that is not an LLM call records no phantom tokens. SSE responses relay incrementally. See Passthrough Routes and Forward Proxy for IDE AI Traffic.
• Consistent hashing and priority tiers for model groups. strategy: consistent_hash pins each session to one target by hashing a request key over a ring of targets: the same key keeps landing on the same target, weights scale each target's share, and when a target fails only its own sessions move to their ring successors — every other key keeps its mapping. The key comes from a source chain you configure (hash_on: a header, a cookie, the caller's API key, or the client IP), defaulting to the routing-key header falling back to the API key. Independently, every target takes a priority: targets partition into tiers, higher is preferred, and a lower tier only receives traffic once every target above it has failed or been taken out by health tracking — so a backup pool can sit idle behind an active one and a request that discovers a fully dead tier still succeeds by spilling within itself.
• MCP 2026-07-28. /mcp and /mcp/{server} serve the final revision of the specification, including the handshake-free discovery lifecycle and the stateless transport. A registered MCP server takes an optional protocol_version to pin the revision the gateway uses to open its upstream session, which is the only way to reach a server that no longer answers the older handshake; omitted, the gateway keeps negotiating as before, so existing configurations are unchanged. Selection is explicit in both directions — the gateway never probes or silently falls back, so a mismatch fails visibly instead of quietly negotiating something you ruled out.
• Anonymous MCP access. An environment can let MCP clients that present no credential reach chosen entries, running as an API key you nominate — that key's tool grant, rate limits, budget and usage attribution all apply. It is gated by a mandatory source-CIDR allowlist, names the entries it opens, and treats the aggregate /mcp entry as opt-in. A client that does present a credential is still authenticated normally; an invalid one is rejected rather than served anonymously.
• OAuth 2.1 resource-server discovery for /mcp. With a canonical resource URL configured and an identity provider enabled, the gateway publishes protected-resource metadata and answers unauthenticated MCP requests with a WWW-Authenticate challenge pointing at it, so a standard MCP client can discover where to get a token. Bearer tokens must carry that URL in their audience.
• Guardrails can be scoped to one MCP server, so a rule can guard a single registered server instead of all MCP traffic in the environment. Guardrails now also scan a tool's structured output, not only its text blocks.
• A drain window on shutdown. On SIGTERM the gateway reports itself not-ready immediately, then keeps accepting new connections for at least shutdown.min_drain_secs (default 30s) before closing its listener, so connections a load balancer routes inside its own detection window are still served. HTTP/1.1 responses carry Connection: close during the window so pooling clients retire connections as they use them. The window is a minimum, not a deadline: the listener closes only once it has elapsed and nothing is in flight.
• W3C trace context is now honored. A valid inbound traceparent makes the gateway's spans children of the caller's trace; a malformed or duplicated header is ignored in favor of a local root rather than failing the request. Span and trace ids are minted once per request, so a delivery retry re-sends byte-identical ids instead of fresh ones. Usage events carry the trace id for correlation.
• Streamed audio transcription is relayed live. /v1/audio/transcriptions with stream=true now forwards frames as they arrive instead of answering from a fully read body, while still recording the request's token usage. A blocking or masking output guardrail keeps the buffered path, since it must see the whole transcript before any of it reaches the caller.
• Role management in the Cloud Admin API. Custom role CRUD, organization-role assignment, and environment-scoped role bindings are now part of the published API contract, with generated bindings and request validation.

Improvements

• Release images are now built with profile-guided optimization, and the allocator runs a background purge thread that returns freed memory to the operating system after load subsides.
• Strict validation now names the offending setting when it rejects configuration a model kind never reads, instead of reporting only that the document failed.
• Passthrough traffic records every usage dimension its detected envelope carries.

Fixes

• Streamed audio transcription was billed as free. The gateway's SSE decoder ended an event only on a bare line-feed pair, while providers frame transcription streams with carriage returns, so the terminal event carrying the token counts was never decoded and every streamed transcription recorded zero tokens. Events now end on any pair of line terminators. The decoder is shared by every streaming bridge, so any upstream framing with carriage returns was affected, not only audio.
• Caller API keys whose MCP access was written before the layered rule keep loading rather than being dropped.
• The deployment and fallback counters are now emitted; pre-dispatch failures no longer count toward them, and failed requests are attributed to the caller that was resolved.
• Passthrough traffic is attributed correctly in traces and on rejected-request events, and a 401 from header-based authentication names the header it expected.
• Forward-proxy routes matched on host may claim path prefixes the gateway reserves, and mirror the whole request path.
• Audio metadata parsing no longer logs warnings about ordinary uploads.
• In the Dashboard: the create-model picker no longer reads as "unsupported," the Dimensions field is no longer mislabelled optional, route transitions show a loading boundary, and the MCP authentication settings live on the MCP Access page with the rest of the MCP configuration.

Upgrade Notes

• Upgrade the control plane first, then your gateways. This is the supported order, and the mixed-version window can be as long as you need. While it lasts, a gateway still on 0.9.x will stop serving a model group that used weighted with sticky (its migrated configuration names a strategy that release does not know) and will not apply MCP tool policies (their stored shape changed). Caller API keys keep authenticating throughout — their MCP access is simply off on the older gateway until it is upgraded. Both are visible in the Dashboard's data-plane compatibility view.
• Deployments managed through AISIX Cloud migrate stored routing configurations automatically on the first control-plane start after the upgrade. Declarative configurations are not migrated and must be updated by hand, as described under Behavior Changes.

0.9.0

Release date: August 13, 2026

This release makes the gateway substantially faster and makes agent traffic legible. The proxy now serves from thread-per-core workers, roughly doubling throughput and halving p99 latency. Caching learned to match requests that mean the same thing rather than only those spelled the same way. A2A calls now record which task they advanced and what they cost, and a verified JWT can stand in for a caller API key, so an identity provider's claims decide which caller a request runs as.

It also completes the Admin API deprecation announced in 0.4.0: the gateway's own Admin API is now read-only, and resources are managed declaratively or through AISIX Cloud.

Behavior Changes

• The gateway's Admin API no longer writes resources. The admin listener keeps its whole read surface — lists and gets for every resource kind, model status, health, the OpenAPI reference, and the playground — but POST, PUT, and DELETE on /admin/v1/<kind> now return 405 with Allow: GET, and both spellings of the API key rotate route return 404. This completes the deprecation announced in 0.4.0. Manage resources through a resources file (validate with aisix validate --resources <file>, reload with SIGHUP) or by writing to etcd; gateways connected to AISIX Cloud never exposed this listener and are unaffected. To rotate a caller key declaratively, write the same resource id with a new key_hash — the old secret stops authenticating as soon as the write propagates. The published Admin API reference no longer documents write operations. See Resources File.
• Cached responses are isolated per caller API key by default. A cache policy's new scope field defaults to api_key, so one caller's answers are never served to another. Deployments that relied on environment-wide sharing must set scope: env explicitly. Either way the cache key changes shape, so expect a one-time full miss after upgrading. See Caching.
• Time to first token is now measured at the first streamed frame of any kind. It previously waited for a frame carrying generated output, so a model that thinks silently before answering recorded the end of its thinking phase — a figure that could exceed the request's own reported latency and could not be compared with what a fronting proxy measures. Time spent thinking remains visible in upstream latency. Dashboards and alerts calibrated against the old figure will shift. See Metrics and Logs.
• A request id supplied by the caller is now adopted as the gateway's own. The gateway accepts x-aisix-request-id by default: the value is echoed back, recorded in the access log and usage events, and forwarded upstream. Ids must be 1–256 visible-ASCII characters; anything else is ignored and the gateway generates its own id, so a malformed header never fails a request. Add x-request-id to proxy.request_id.accept_headers to honor that convention too, or set the list to empty to restore the previous behavior. Because the id is now caller-controlled it is neither unique nor trusted, and it is never used as a metric label. See Metrics and Logs.
• Configuration a model kind never reads is now rejected instead of stored and ignored. retries, auto_prompt_caching, and cost on a model group, the generic call settings on an ensemble, and auto_prompt_caching and cost on a semantic router are refused at write time with 400. A resources file carrying one fails to load and names the offending entry. A model already stored with such a setting keeps working: the gateway drops the field and reports it as partially compatible rather than taking the model out of service. In the other direction, semantic routers and embedding models now accept timeout, stream_timeout, and retries, which they previously refused.
• Several configurations that were silently accepted are now refused. Creating a model with a configuration block belonging to a different kind, referencing a wildcard alias as an ensemble member or judge, as a semantic router target or default, or as a cache policy's target, renaming a referenced model into a wildcard, and pointing a cache policy at a model that does not exist all return 400. Each was previously accepted and then quietly ignored.
• Traffic through a wildcard model is reported under the wildcard's own name. Metrics, rate-limit buckets, and health tracking now key on the configured wildcard row rather than the alias each caller happened to send, so one model is one identity instead of a new series per spelling. A dashboard filtered on a caller-minted alias will show its series end at this release.
• Upgrade note. The control plane widens the request_id column on its usage table from uuid to text so a caller's own id can be stored. On a large table this rewrites it — about six minutes per 10 million rows — and the control plane does not serve traffic until it finishes. The Helm chart's startup probe budget has been raised to 30 minutes to accommodate this; if you run your own manifests, raise yours before upgrading.

New Features

• The proxy serves from thread-per-core workers. Each worker has its own runtime, listener, and upstream connection pool, so a request is accepted, dispatched, and answered on a single thread instead of being handed between threads twice. On four cores this raises throughput by 54–88% depending on concurrency and roughly halves p99 latency, cutting system calls per request from 11.9 to 5.0. It is on by default on Linux. Two settings apply at startup: proxy.thread_per_core and proxy.workers, which defaults to the parallelism available to the process and follows a cgroup CPU limit. Below roughly four client connections per worker the kernel spreads connections unevenly and this mode is slower than the shared runtime; set proxy.thread_per_core: false for very low concurrency.
• Semantic caching. A cache policy can now match on meaning: a request that misses exact matching is embedded and served from the nearest stored entry at or above a cosine similarity threshold you set. Only fully textual requests take that path — anything carrying images, audio, or tool calls always matches exactly. Entries can live in each gateway's memory or, with backend: redis, be shared across replicas using Redis vector search. Sharing requires Redis 8 or later, or the search module; the gateway checks at startup and, when it is missing, keeps serving exact matches and says so in the log rather than failing traffic. A policy can also be emptied without deleting it. See Semantic Caching.
• JWT claim mappings. A new resource resolves verified OIDC claims to an existing caller API key, so an identity provider can decide which caller a request runs as without provisioning a key per user. Rules are evaluated in priority order and the first whose claim conditions all hold selects the key; the request then inherits that key's model and tool access, rate limits, and budget unchanged. Conditions support exact string matches and array membership over nested claim paths. A token that matches no rule is denied. Usage events record the subject, the provider, and the mapping that matched — note that the subject is an end-user identifier and reaches any configured observability exporter. See Claim Mappings.
• A2A calls are now observable at the protocol level. Every call records its operation, task id, context id, and final task state, with both wire vocabularies normalized so one figure is not split in two. Streamed calls additionally record time to first event, how many events arrived, and whether the caller was still listening at the end, so a caller that hangs up mid-task is no longer counted as a success. Because the protocol carries no usage block, the gateway estimates tokens from the message text that passed through and marks them as estimated; cost is left at zero, since what an agent charges is not something the gateway can know. Metrics are sliced by agent and operation. See Agent Gateway.
• The provider's own response id is now recorded. It appears in the access log and in a dedicated log line emitted once per upstream attempt, which covers streamed responses and retried or failed-over attempts that a single access-log line structurally cannot. The Dashboard shows it on the log detail panel, so a provider-side problem can be traced to the exact upstream call without joining records by hand.
• Rate-limit policies accept a day window, counted against the UTC day. See Rate Limit Policies.
• The Dashboard's request log is split by gateway surface. LLM, MCP, and A2A traffic each get their own tab with the filters that can actually narrow it, rather than one feed with filters that apply to only part of it.
• The Dashboard's dropdowns are searchable. Every selector that can grow long — models, provider keys, API keys — now filters as you type instead of requiring a scroll through the full list.
• The Dashboard reports gateway configuration compatibility. When a setting you save is not understood by every gateway in the environment, the Dashboard says so at save time and on the data-planes view, naming the field and the version that introduced it, instead of leaving the setting silently inert on older gateways.

Improvements

• Beyond thread-per-core, the per-request path was reworked throughout: a jemalloc allocator and link-time optimization, fast paths that skip machinery a zero-configuration deployment does not need, a per-worker metrics cache, TCP_NODELAY on downstream connections, cached upstream endpoint URLs, and a single configuration snapshot load per request.
• A rate-limit condition on a model now matches the entry the caller addressed as well as the one the request was dispatched to, so a condition naming a model group applies to every request addressed to that group. Previously only the dispatched target was compared, and such a condition could never match.
• A semantic router now honors its members' own access rules when it selects a route, falling through to another target instead of dispatching to a member the caller is not permitted to use.
• Per-model rate limits now apply to every model kind, and the Dashboard offers the rate-limit form on every one.
• A cache policy's target is now validated when the policy is created, and renaming a model that a policy references updates the reference.

Fixes

• An invitation that has expired no longer keeps its email address reserved, so the same person can be invited again. Pending invitations still block a duplicate.
• Creating a resource with a boolean field explicitly set to false now stores false. A column default was substituting true for the zero value, so an OIDC provider created as disabled was stored as enabled and the API reported it as enabled.
• Post-authentication refusals on the realtime endpoint are now attributed to the caller that was resolved, instead of being recorded without an identity.
• Gateways that have stopped sending heartbeats no longer count toward the Dashboard's configuration-compatibility banner, so a decommissioned instance cannot make a healthy fleet look partially incompatible.
• A gateway built from source without version stamping reports a placeholder version; that placeholder is now treated as unknown rather than as a real old version, so such a fleet no longer warns forever about features it may well support.

0.8.2

Release date: August 11, 2026

This is a maintenance release for the A2A gateway. The gateway now announces the wire version each agent is pinned to, finds an agent card published under a path prefix, keeps every address on the card pointed at itself, and relays message/stream as it arrives instead of buffering it. It also applies the agent and MCP server naming and credential rules to every configuration path, not only the gateway's own Admin API.

Behavior Changes

• A2A agent and MCP server definitions supplied through a resources file are now checked against the same rules the Admin API enforces: the name shape, the credential each auth_type requires, and the fields an OpenAPI-backed MCP server needs. A file that breaks one of them no longer loads, and the gateway reports the offending entry and the missing property by name rather than starting with a definition that cannot work — an agent named with a / splits its own /a2a/<name> route in two, and auth_type: bearer without a secret authenticates upstream with nothing. Deployments configured through the control plane are unaffected, since those rules already applied when the resource was created; a gateway managed by the control plane rejects only the offending entry and keeps serving the rest. Check a file before rolling it out with aisix validate --resources <file>. See Agent Gateway.

Fixes

• The gateway now announces an agent's pinned protocol version in A2A-Version on every call it makes to that agent, including the agent-card fetch. Nothing sent the header before, and the A2A specification tells an agent to read its absence as version 0.3 — so an agent pinned to 1.0 answered VersionNotSupportedError and the configured protocol_version had no effect. A version supplied by the caller does not override the registered pin.
• An A2A agent whose card is published under a path prefix now resolves. The well-known card URI was built at the origin, discarding the path the agent was registered with, so any agent behind an ingress path or on a platform that multiplexes tenants under a prefix was asked for a card it does not serve there — and the platform's catch-all 405 came back as the agent's own answer.
• The agent card the gateway serves now points every advertised address back at the gateway. Only the top-level url was rewritten before, leaving the upstream address in supportedInterfaces — and a 1.0 caller reads its endpoint from there, so it could bypass the gateway, and with it the caller authentication and per-agent access control, and reach the agent directly.
• message/stream and tasks/resubscribe now relay the agent's event stream as it arrives. Both were served through the single-response path, which buffered the whole body and parsed it as one JSON document; an event stream is not valid JSON, so the two methods whose purpose is watching a long-running task both failed with 502 and could not be used at all. Events are reassembled across chunk boundaries, and a malformed event ends the stream with an error rather than being skipped, so a truncated task cannot read as a completed one.

0.8.1

Release date: August 7, 2026

This is a maintenance release. It corrects how the gateway decides that an upstream speaks the Anthropic protocol, so a model pointed at a self-managed Anthropic endpoint is served on the whole /v1/messages family with its request body passed through unchanged. It also gives authentication denials enough context to investigate, carries tool grants through an MCP server rename, and fixes several control-plane admission paths that had no workable request shape or no actionable error.

Behavior Changes

• Enabling auto_prompt_caching now requires a model whose provider key speaks the Anthropic protocol, either provider: anthropic or a BYO key whose adapter is Anthropic. The setting is implemented by injecting Anthropic cache_control markers, so on any other upstream it was stored but never took effect, and nothing told the operator that their prompt caching was off. Enabling it elsewhere now returns 400. Disabling remains valid on every provider, so a model configured before this release can still be cleaned up, and a key whose adapter cannot be determined is allowed rather than rejected on a guess. See Prompt Caching.
• A model configured as provider: byo with the Anthropic adapter is now served on /v1/messages by passing its request body through unchanged, instead of re-encoding it through the cross-provider bridge. A cache_control marker set by the client reaches the upstream as written; it was previously dropped, or rewritten to the model's configured TTL when auto_prompt_caching was enabled. This also changes which upstream cache-write price applies, since a one-hour write costs twice the base input price and a five-minute write costs 1.25 times. Authentication is unchanged.

Improvements

• Authentication denials now carry the request context needed to act on them: the caller address resolved through the trusted-proxy configuration, the HTTP method and path, and the request id. Denials for a disabled or expired key also carry the key id. A 401 is refused before any handler runs and therefore never reaches the access log, which left the denial counter as the only record and it cannot say who, when, or against which route. Log levels are unchanged, so scanner traffic stays at debug and the default-level output is exactly as before. See Metrics and Logs.
• The AISIX Cloud Admin API reference now describes provider request headers and conditional rate-limit counter semantics correctly. It previously said that changing a policy's conditions resets the current windows; counter keys are built from the policy id and the selected group_by values, so changing only conditions or limits preserves existing keys and their counts, while changing the grouping dimensions moves requests to different keys.

Fixes

• /v1/messages/count_tokens now serves a model whose provider key speaks Anthropic through an adapter rather than by vendor id. A provider: byo model pointed at an Anthropic-protocol endpoint was rejected with 400 on this endpoint while the sibling /v1/messages served it normally.
• Renaming an MCP server now carries the tool grants with it. Tool names are <server>__<tool>, so every grant references the server by name; the per-key rate limit entries already followed a rename but the access control did not, which silently revoked access to the whole server and made its tools disappear from tools/list with no explanation. The caller key's allowed_tools and mcp_access, and the environment- and team-level access policies, are all rewritten. Wildcards that match a shape rather than one server, such as a bare *, are left alone. See MCP Access Control.
• Provider keys for Amazon Bedrock and Google Vertex can now be created by supplying config alone. Their credentials do not fit in a single api_key string, but api_key was required by the schema and a non-empty value was rejected by the handler, so the only accepted request was an empty string that no documentation mentioned. A request missing its credential now names the field that provider actually accepts. See Provider Keys.
• PATCH with "tls": null now clears a provider key's TLS settings, matching how rate_limit and the other optional blocks behave. It previously returned 400 and the setting could only be cleared with an empty object.
• The usage feed and its CSV export now include audio_duration_seconds. A transcription priced by duration showed a cost whose basis could not be read back from the API. The field is omitted for requests that carry no audio. See Audio.
• The offline package no longer lets two installations become one. The Compose project name came from the directory the package was unpacked into, so unpacking a second copy elsewhere and running run.sh took over the running installation, rebuilding its containers against the second copy's configuration while keeping the original data volume. The project name is now fixed to the package, and starting it while another directory holds the same stack is refused with both paths and the ways forward. Upgrading in place is unaffected, and a second independent installation can still be started with an explicit COMPOSE_PROJECT_NAME. See On-Premises Quickstart.
• The Dashboard container in the offline package now has a writable Next.js cache directory, matching the Helm chart.
• Trusted-proxy CIDRs and the User-Agent client-type rules can now be set from the environment. proxy.real_ip.trusted_proxies and observability.metrics.client_type_rules could only be supplied in a configuration file; setting either through its AISIX_* variable failed to start the gateway. Deployments configured purely through environment variables — the Helm chart and the Dashboard's docker run snippet — therefore could not declare their load balancer as a trusted proxy, so every request appeared to come from it. See Metrics and Logs.
• Two admission errors now say what to change. A weight on a target in a group that is not weighted names the field and the strategy instead of reporting a generic invalid-fields error, and an unrecognized provider names the rejected value and explains that the catalog route needs a Dashboard session rather than an admin token.
• The Dashboard now lets one API key allow several model groups. Checking one group greyed out the rest, so a key could only ever reference one, although the API always accepted a longer list and the edit dialog never enforced the limit. The two dialogs now share one picker, which also fixes model groups, ensembles, embedding models, and semantic routers rendering with an empty arrow in the edit dialog.

0.8.0

Release date: August 6, 2026

This release makes rate limiting expressive enough to describe a real quota policy. A single policy now decides which traffic it applies to through a condition tree, splits its counters along any dimension of that traffic, caps seven different quantities, and can suspend itself on a recurring schedule such as weekday off-peak hours, whole weekends, or named holidays. MCP servers gain an endpoint that serves one server and keeps its original tool names, so a client written against that server works unchanged. Token, spend, and request metrics now cover every endpoint that reports usage rather than only chat and messages, and audio transcription is priced by the duration of the audio.

Behavior Changes

• The gateway now treats api_base as the upstream root on every OpenAI-family endpoint. Previously /v1/chat/completions appended the endpoint to the configured base verbatim, while /v1/responses, /v1/rerank, /v1/audio/*, /v1/realtime, /v1/files, /v1/batches, and /v1/fine_tuning/jobs inserted a /v1 segment unless the base already ended in one, so a key whose upstream root was not /v1 served chat correctly and returned 404 everywhere else. Both paths now read the base the same way. A base carrying a path but no version segment builds https://proxy.corp/openai-shim/responses where it previously built .../openai-shim/v1/responses; move the /v1 into api_base if you relied on the old behavior. A bare host still has /v1 added, a base ending in /v1 is unchanged, and Anthropic is unchanged in every base shape. See OpenAI-Compatible Vendors.
• Histogram bucket boundaries changed for two metrics. aisix_request_ttft_seconds drops its two edges below 50 ms, because it measures the upstream time to first token and never observes values there, and aisix_request_e2e_latency_seconds gains edges at 420 and 600 seconds so histogram_quantile() interpolates past 300 seconds instead of pinning to it. Dashboards and recording rules that depend on the previous edges need updating. Both sets are now configurable per metric under observability.metrics.buckets. See Metrics Reference.
• Time to first token now counts a reasoning model's first reasoning delta. Only ordinary content stamped the measurement before, so a model that reasons before it answers reported a later first token than the caller observed. Reported values for those models drop accordingly. See Metrics and Logs.
• GET /v1/models now lists model groups alongside direct models, semantic routers, and ensembles. A model group was the only virtual alias filtered out of the listing, so a caller key scoped to a group as its public entry point received an empty list. No authorization changed, since those names were always callable by those keys. Only clients that assert an exact model list are affected. See Model Aliases.

New Features

Rate Limiting

• A rate limit policy can now be written in a conditional form that carries a tree of conditions instead of a single scope. Conditions match on team, member, caller API key, model, model name, and provider, combine through explicit and and or groups up to three levels deep, support negation, and match strings by equality, list membership, or regular expression. One resource can therefore hold different quotas for different traffic instead of one flat limit. See Rate Limit Policies.
• A conditional policy splits its counters with group_by, which buckets the counters per team, per member, per caller key, per model, or any combination of those. A single policy now enforces a per-tenant quota that previously required one policy per tenant.
• A policy can cap seven quantities in any combination: requests per second, minute, hour, and day, tokens per minute and per day, and concurrent requests.
• A policy that matches on a model property reserves its quota where the concrete model is known, which is per target for a routing or ensemble parent. An over-limit target is treated as a failed attempt and fails over to the next one, rather than consuming the parent alias's own budget.
• A rate limit rejection now identifies the policy that produced it. The 429 response body carries the policy's id and name, and aisix_ratelimit_rejections_total counts rejections on every endpoint and is labeled by the enforcing layer and policy. With several policies live, an unattributed rejection could not be traced to its cause.
• A policy can carry recurring suspension windows during which it is not enforced. Each window selects days either by weekday or by explicit dates, takes wall-clock start and end times in its own IANA time zone, and treats an end at or before the start as crossing midnight and belonging to the day it starts on. Multiple windows form a union, enforcement resumes automatically, and the counters are not reset by the transition, so suspending and resuming inside one rate window cannot be used to clear consumed quota. In AISIX Cloud, the Dashboard exposes these as a schedule list on the rate limit policy form. See Rate Limit Policies.

MCP Gateway

• A new per-server endpoint at /mcp/{server} serves a single registered MCP server and lists its tools under their original names, without the <server>__ prefix the aggregate endpoint applies. Both the bare and the prefixed spelling are accepted when calling a tool, and tool access control is always evaluated on the prefixed form. A client written against one server's own tool names therefore works through the gateway unchanged. See MCP Gateway Overview.

Deployment

• A new proxy.url_rewrites setting rewrites the request path at the gateway entry, matched by regular expression with capture-group templates in the replacement. Patterns are validated at startup and can be supplied through an environment variable for deployments that ship no configuration file. This lets a client whose base URL cannot be changed reach a different gateway path. See URL Rewriting.

Pricing

• Audio models can now be priced by the duration of the audio rather than by tokens. A transcription request reports the audio's length as its cost basis, and the model's price is expressed per minute of audio. In AISIX Cloud, the Dashboard accepts these rates on the model pricing form. See Audio and Model Pricing.

Improvements

• The detailed request metrics and the token and spend metrics now cover every endpoint that reports usage, including responses, embeddings, rerank, audio, images, and the realtime surface. They previously covered only chat completions and messages, so usage on the other endpoints was recorded in the usage log but missing from the metrics. See Metrics Reference.
• Configuration reads are now forward compatible. A resource document carrying a field this gateway version does not recognize loads and serves, with the unknown fields ignored and reported through partially_compatible on the configuration status endpoint, a deduplicated warning, and a new gauge. Previously an unknown field rejected the whole document, which turned every additive change on the control plane into a breaking one for a gateway that had not yet been upgraded — a caller API key carrying a new field stopped authenticating. Write paths remain strict and still reject unknown fields. See Configuration Status.
• A rejected configuration update no longer costs the last good value. The gateway keeps serving the value it previously loaded for that key across a resynchronization and a restart, instead of dropping the row. See Configuration Propagation.
• Errors that a provider reports inside an already committed stream are now surfaced with the provider's own status and message, across OpenAI-family, Anthropic, Gemini, and Amazon Bedrock streams. An Anthropic error event arriving mid-stream was previously discarded and the truncated stream closed as though it had completed, so a client could not tell a cut-off response from a finished one. Such a stream now ends with an error frame and no completion marker. See Streaming.
• A request refused for exceeding the body limit now reports how the reading of its body ended, through a dedicated metric, so a refusal whose client disconnected is distinguishable from one whose body was read to completion. See Metrics and Logs.
• The gateway warns once when it reaches a credentialed MCP, OpenAPI, or A2A upstream over cleartext http, deduplicated per server and address, so the mistake is visible without filling the log. See MCP Upstream Authentication.
• A request to /v1/realtime that is not a WebSocket upgrade is now recorded and answered in the gateway's own error format, and path rejections elsewhere in the proxy go through the same path as other early refusals, so they appear in the access log and the request count. See Realtime.
• The AISIX Cloud Admin API reference now documents every operation and schema it exposes, and its base URL is an editable field rather than a fixed relative path. The reference is served from the API7 documentation site, not from the reader's control plane. Readers can therefore set the base URL to their Hybrid Cloud or On-Premises control plane so the examples use the correct endpoint.

Fixes

• A streaming transcription now records the usage it reports. Such a request logged zero tokens and therefore cost nothing, while the same transcription made without streaming was billed normally. See Audio.
• The offline package's quickstart script now prints a Dashboard address that works. It previously printed the container's internal port, so following the printed URL failed on the default installation path. See On-Premises Quickstart.
• The Dashboard's model pricing form now accepts audio rates below one cent per minute, which the audio pricing of most providers requires.
• Budget threshold guidance in the Dashboard no longer suggests that a hard stop can overshoot its limit by an arbitrary amount. See Budgets.

0.7.1

Release date: July 31, 2026

This release makes the gateway deployable on networks it could not previously reach. Outbound connections gain a trust configuration, so a model endpoint, guardrail, or MCP upstream whose certificate is signed by a private or enterprise authority can be dialed without a process-wide workaround, either deployment-wide or per endpoint on the provider key that declares it. A data plane running on a separate host and addressed by an IP address can now complete the mutual TLS handshake and join its control plane. Requests that a caller abandons, and requests the gateway refuses before dispatch, are no longer invisible in the access log and metrics.

Behavior Changes

• A streaming response that the caller abandons mid-delivery is now recorded as 499 rather than 200. The usage event previously reported a completed delivery, so a caller that closed the connection halfway through a stream could not be told apart from one that read the stream to the end. Reports and dashboards that select on status 200 will see these requests move to 499. See Metrics and Logs.

New Features

Outbound TLS

• A new upstream.tls setting configures the trust the gateway uses on every connection it dials out on, through ca_file, client_cert_file, client_key_file, and verify. It applies to provider endpoints, all guardrail providers, MCP and A2A upstreams, passthrough routes, JWKS and OIDC discovery, and OpenTelemetry export. An upstream behind a private certificate authority previously failed with a generic connection error, and the only workaround was the process-wide SSL_CERT_FILE environment variable, which several outbound paths did not honor. See TLS and mTLS.
• A provider key can carry its own tls block, with an inline ca_cert and a verify flag, so a deployment facing more than one private authority declares trust where it declares the endpoint. The certificate is supplied inline rather than as a file path, because whoever configures a provider key has no way to place a file on the gateway host. In AISIX Cloud, the dashboard exposes this as an Endpoint TLS section on the provider key form.
• A rediss:// cache or rate-limit backend takes its own tls block with the same fields, since it usually sits inside your own deployment and is issued by a different authority than the model endpoints. Two limits are documented rather than silently ignored: Amazon Bedrock supports ca_file but neither client certificates nor verify: false, and Redis Sentinel mode supports verify but not ca_file.

Improvements

• Requests refused before dispatch now appear in the access log and in aisix_requests_total. A body over request_body_limit_bytes returned a correct 413 and otherwise left no trace, which made a client reporting a rejection the gateway had no record of indistinguishable from a request that never arrived. The 400 for conflicting Content-Length headers had the same gap. These rejections are refused before authentication, so they carry no usage event and do not appear on the logs page in AISIX Cloud.
• Requests abandoned before the response head are recorded instead of vanishing. Such a request was previously absent from the access log, the usage events, and the metrics at the same time, which hid the case operators most want to see: a caller giving up during a long time to first token. A new counter, aisix_proxy_client_cancelled_requests_total, labeled by endpoint, counts them. See Metrics and Logs.

Fixes

• A data plane on a separate host, pointed at its control plane by an IP address, can complete the mutual TLS handshake and join. The data-plane manager issued a server certificate whose subject alternative names came only from its listen address and from the server name the client sent, and in a container neither yields the control plane's external address: the listener is bound to all interfaces, so there is no host to read, and a client dialing an IP address sends no server name at all. The advertised endpoint is now passed to the data-plane manager, which issues the certificate for that host, so an IP address works as well as a DNS name. Set it through AISIX_CLOUD_DPMGR_BASE_URL, or api.dpmgrBaseURL in the Helm chart. See On-Premises Deployment.

0.7.0

Release date: July 31, 2026

This release lets the AISIX gateway generate MCP tools for supported operations described in a REST API's OpenAPI 3.x document, without requiring a separate MCP server. Available in both the open-source AISIX gateway and AISIX Cloud, this capability calls the API and injects the configured credential. MCP servers also gain a per-caller rate limit scoped to a single server. AISIX Cloud adds a review workflow that can require approval before a server reaches callers. Alongside them, every model now has an upstream deadline even when none is configured, and usage records separate what the upstream took from what the caller actually waited for.

Behavior Changes

• Usage event latency fields are renamed to make their scope explicit, and a new field records the caller-facing wait. latency_ms becomes upstream_latency_ms and ttft_ms becomes upstream_ttft_ms; both remain scoped to a single attempt. upstream_ttft_ms is now measured from the start of that attempt rather than from request entry, so it is directly comparable with the latency beside it. The new downstream_latency_ms is scoped to the whole request. On the OpenTelemetry export the span attribute aisix.ttft_ms becomes aisix.upstream_ttft_ms, and aisix.downstream_latency_ms is added, so any dashboard or alert reading those names needs updating. The control plane accepts both the old and the new names, so a gateway managed by AISIX Cloud needs no change. See Metrics and Logs.
• Models that configure no timeout are no longer unbounded. A new deployment-wide upstream.timeout_ms defaults to 6000000, that is 6000 seconds, and applies to any model that sets neither timeout nor stream_timeout. Previously such a model had no upstream deadline at all, so an upstream that accepted the connection and then went silent held the request open indefinitely. The default is deliberately generous, so it is a backstop rather than a responsiveness target. To restore the previous behavior deployment-wide, set upstream.timeout_ms: 0; a single model opts out with timeout: 0. See Configuration Files.
• The timeout and stream_timeout fields on a model group previously had no effect, because members only ever used their own values. They now apply to members that set neither, resolving from the member, then the group, then the deployment default. A model group that already carried these fields starts applying them after this upgrade.
• proxy.request_body_limit_bytes now defaults to 0, which means no cap, instead of 10 MiB. Providers accept larger requests than any fixed gateway default, so a client that worked directly against the provider could break when routed through the gateway. This applies only when the setting is absent from the configuration file; a deployment whose file carries the value explicitly keeps it. To keep a cap on a configuration that omits the setting, set request_body_limit_bytes: 10485760. The value 0 previously meant "reject every request that has a body" and now means "no cap".

New Features

MCP Gateway

• Both the open-source AISIX gateway and AISIX Cloud support MCP servers backed by plain REST APIs. Setting a server's type to openapi and supplying an OpenAPI 3.x document makes the gateway generate one MCP tool per supported operation and execute each tool call as an HTTP request against the API. The gateway holds the configured credential, injects it into outbound calls, and does not expose it to the calling agent. See Expose a REST API as MCP Tools.
• Generated tools go through the same gateway controls as tools from a real MCP upstream, including tool access policies, rate limits, guardrails, and usage records.
• In AISIX Cloud, operators can paste the OpenAPI document, provide a URL for the control plane to fetch once, or load a local file through the dashboard. Each OpenAPI-backed server has a tools page for inspecting the generated tool names and operations.
• AISIX Cloud can require review before publishing an MCP server. A submitted server stays invisible to callers until an approver accepts it, and a change proposed to a server that is already live waits for review without taking the running server offline. See Review MCP Servers Before They Go Live.
• In both products, an API key can carry per-server rate limits, so an agent looping on one MCP server does not consume the same key's allowance for another server. Limits are expressed per second, minute, hour, or day, plus concurrency, and only tool calls are metered, so a client can still connect and list tools after a server-specific allowance is exhausted. The key's own rate limit still applies on top. See Rate Limits and Budgets.

Timeouts

• Two new gateway settings, upstream.timeout_ms and upstream.stream_timeout_ms, provide deployment-wide defaults for the request deadline and the maximum gap between streaming chunks. Deadlines resolve from the model, then the model group, then these defaults, and apply across every endpoint rather than only the chat path. See Configuration Files.
• Model groups expose timeout and stream_timeout in the control plane and dashboard, and an explicit 0 on a model is now preserved rather than normalized away, so a model can opt out of the deployment default.

Observability

• Usage records now report what the caller waited for, measured where the gateway hands bytes to the client rather than where the upstream chunk arrives. Under an output guardrail that masks the response, the stream is held back until the whole response scans clean, and only the caller-facing figure reflects that wait. The dashboard's latency percentiles use this figure and count only successful requests. See Metrics and Logs.
• Subtracting the upstream time-to-first-token from the caller-facing latency isolates the wait the upstream did not account for, which makes gateway-side overhead visible without correlating two systems. On a request that retried, that difference also contains the earlier attempts and the delay between them.

Audit Log

• The audit log supports free-text search across an event's actor, action, and target, an absolute time range, page-by-page navigation, and export of the current result set. See Logging and Auditing.

Deployment

• The data planes page offers a real Helm installation tab alongside the manifest-based one, so a data plane can be installed from the published aisix chart with the environment's connection settings already filled in.

Improvements

• An idle gateway stays in service. The readiness endpoint previously reported a gateway unready once its configuration watch had gone five minutes without an event, which an environment whose resources are not changing produces routinely. Under Kubernetes every replica crossed that threshold at the same moment and the service lost all its endpoints while perfectly healthy. Readiness now reports only whether the gateway is draining and whether configuration has been applied. Configuration freshness remains observable through the health endpoint, the configuration status endpoint, and the aisix_config_* metrics. See Config Status.
• The aisix data-plane Helm chart now uses that readiness endpoint on the proxy port for its readiness probe.
• Requests that exceed a configured body limit return the same error envelope on every route that accepts a body. Previously several JSON endpoints and both raw-body endpoints returned a plain-text rejection, multipart uploads reported a generic client error, and the MCP and agent endpoints returned a bare 400. Malformed JSON on those endpoints likewise now returns the standard envelope. See Headers and Error Codes.
• Amazon Bedrock models honor their configured deadline and connect timeout. The AWS SDK was building its own HTTP client with its own defaults, so a Bedrock model's timeout was not enforced.
• MCP upstream connections honor the gateway's upstream connection settings, including the connect timeout, TCP keepalive, and connection pool sizing.
• Prometheus metric memory no longer grows without bound, and metric series handles are resolved once and cached instead of being looked up on every request.
• Released gateway binaries keep their symbol table, so a profile taken against a shipped image is readable without a special build.
• The overview page shows the full model name in its top-models list instead of truncating it.

Fixes

• The latency recorded for the attempt that succeeded is scoped to that attempt. A request that failed over previously reported the winning attempt's latency measured from request entry, so it included every earlier attempt and the delay between retries, which made a successful failover look like a slow upstream.
• Renaming an MCP server carries its per-server rate limits along, and deleting the server removes them. Previously the limits stayed behind under the old name and silently stopped applying.
• An approver editing an MCP server publishes it instead of taking it offline.
• Providers that the gateway supports natively but the public model catalog does not list can be configured again. They were being rejected during validation.
• Regional provider base URLs match the catalog's provider identifiers, so a region-specific default resolves instead of falling through.
• Audit records are no longer lost when a client disconnects mid-request. The control plane rolls back cleanly, records the disconnect once, and returns a consistent response body.

0.6.0

Release date: July 29, 2026

This release lets callers authenticate with a JWT issued by their own identity provider instead of a gateway API key. It also replaces per-key MCP tool allowlists with a layered access policy resolved from the environment, the team, and the key. Alongside them, the retry budget moves onto the model, upstream requests gain request-context variables and a client-header allowlist, and video generation reaches two more providers.

Behavior Changes

• Retries are now enabled by default. Previously a model group that did not set routing.retries performed no retries at all. The budget now resolves from the model, then the model group, then a new deployment-wide upstream.retries default of 2, so a deployment that never configured retries starts retrying after this upgrade. Each retry re-sends the full request body and stacks on top of any retry the provider's own edge performs. To keep the previous behavior, set upstream.retries: 0 in the gateway configuration file. See Proxy Errors and Retries.

New Features

Authentication

• Callers can authenticate with a JWT issued by an OIDC provider instead of a gateway API key. A new oidc_providers resource holds per-environment trust rows that pin the issuer, its accepted audiences, and its JWKS location, and the gateway validates every claim before the request proceeds. See JWT Authentication.
• An API key binds to an external identity through a provider and subject pair, so a JWT-authenticated caller carries that key's budgets, rate limits, model access, and usage attribution. A subject is only ever resolved for the trust provider named on the key, so a second trusted issuer cannot assert a subject that belongs to another provider.
• JWT authentication applies at the gateway's single authentication point, so every proxy surface accepts it, including chat completions, messages, responses, embeddings, rerank, audio, images, video generation, files, batches, fine-tuning, the MCP and agent endpoints, realtime WebSocket connections, and passthrough.
• Validation is deny-by-default: only asymmetric signing algorithms are accepted, expiry and audience claims are required, the issuer must match an enabled trust row, and an operator can additionally require scopes or pin arbitrary nested claims.

MCP Access Control

• MCP tool access is now governed by a layered policy instead of an allowlist maintained on each key. An environment default applies to every caller, a team policy replaces it for that team's member keys, and a key narrows the result further. See MCP Access Policies.
• A policy grants no tools, a named selection, or all tools. Granting all tools covers current and future tools and is an explicit choice rather than a default.
• Deny patterns from every layer always subtract, so an environment-level deny survives a team policy and still applies to keys created before this release.
• A key can only narrow what it inherits, never widen it. Keys that still use the previous allowed_tools field keep their existing behavior unchanged, so upgrading cannot silently grant access.

Video Generation

• The video generation endpoint adds two providers: Runway (runwayml, covering the Gen family and Runway-hosted Veo) and OpenAI Sora (openai).
• The content route now delivers finished videos in two modes. For providers that return a signed download URL the content route still answers with a 302 redirect, so the transfer goes straight from provider storage to the client. OpenAI requires its own credential to download the file, so the gateway fetches it with the configured provider key and streams the bytes back without holding the file in memory. The provider credential is never exposed to the caller.
• progress now reports a real completion percentage for OpenAI Sora. Providers that do not expose one continue to report 0 until the task completes and 100 afterward.
• OpenAI is the only video provider with a built-in default base URL. The other four still require api_base on the provider key.

Upstream Request Headers

• Default header values on a provider key can reference the request context, for example "x-tenant-id": "${request.api_key.team_id}", and the gateway renders them per request. An internal model service can attribute traffic to the calling team or key without a separate provider credential per tenant. The variable vocabulary is closed and contains no secrets, and a header whose variables do not all resolve is dropped rather than sent with an empty value. See Upstream Request Headers.
• A provider key can relay named inbound client headers upstream through an allowlist of exact names or single-wildcard patterns. It is empty by default, and authentication, transport, and gateway-internal headers are refused from a client even under a wildcard. This lets a caller pass a provider-specific header or propagate a trace context on the standard endpoints instead of dropping to passthrough.
• Video generation and the files, batches, and fine-tuning endpoints previously applied no default headers at all. They now carry the resolved set on every request.

Retries

• The retry budget is now a model-level setting that applies on every endpoint rather than only the chat path. It resolves from the model, then the model group, then the deployment-wide default, and is configurable per model from the dashboard. See Proxy Errors and Retries.

Connection Management

• The gateway can bound how long an accepted client connection may sit idle between requests, and can send heartbeat comments on a streaming response that has not yet produced output, so a proxy in front does not treat a model that is slow to its first token as an abandoned connection. An in-flight request or stream is never interrupted. See Configuration Files.

Improvements

• The logs page supports free-text search across a request's model, key, error message, and identifiers, exports the current result set as CSV or JSON, and shows the date alongside the time on the timestamp column.
• The models page can be filtered by model name, upstream model, or model ID.
• The model ID field suggests the selected provider's catalog models while remaining free-text, so an unlisted or newly released model can still be entered by hand.
• The API key and provider key pages use compact tables with unified search, type filters, and server-side paging, replacing the previous card layout.
• The dashboard sidebar shows the control-plane build version, so an operator can confirm which release an environment is running without shell access.
• The aisix-cp Helm chart adds startup probes to the control-plane API and data-plane manager, so a slow first boot or a long schema migration on upgrade is no longer cut short by the liveness probe.

Fixes

• Input guardrails now scan the union of a message's text content and its structured content blocks. The two are independent fields on the wire and the provider bridges forward the structured blocks when present, so a request carrying benign text alongside a payload in the blocks could previously pass every input guardrail while the model still received the payload.
• A failed request's access log line now names the failure, carrying both a stable error category to filter or alert on and the underlying reason. Previously an upstream error, a name resolution failure, a reaped connection, and an unanswered connection attempt all produced an identical line. See Metrics and Logs.

0.5.0

Release date: July 24, 2026

This release adds a unified video generation endpoint, budget threshold alerts that notify an external system, and automatic prompt caching for Anthropic models. It also corrects several cases where retries, timeouts, rate limits, and usage accounting did not apply on every request path.

New Features

Video Generation

• A new video generation endpoint accepts prompt-to-video tasks on /v1/videos, polls their status, and returns the finished video, following the same three-stage shape as the OpenAI video API. Providers currently mapped are Alibaba Cloud Model Studio, Zhipu AI CogVideoX, and Volcengine Ark Seedance.
• Video requests now pass through the same gateway controls as chat traffic: model aliases, caller API key access checks, client IP allowlists, model-level rate limits, and input guardrail scanning of the prompt. Previously video traffic could only reach a provider through passthrough, which applies none of these per-model controls.
• Task state is not stored by the gateway. The returned video ID carries the routing information, so status and download calls work against any gateway instance.
• Video submissions are recorded in usage logs with zero tokens. Duration-based cost accounting is not yet applied, so video traffic does not consume budgets in this release.

Guardrails

• A new Alibaba Cloud AI Guardrails kind calls the MultiModalGuard service and acts on its suggestion verdicts. When the service returns masked content, the gateway writes the masked text back into the request instead of rejecting it outright.
• The gateway records aisix_guardrail_latency_seconds, a per-execution latency histogram that separates guardrail overhead from upstream model latency. See Metrics.
• Responses from Alibaba guardrail services keep their upstream request ID, and the gateway correlates it with its own request ID for cross-system troubleshooting.

Budgets and Alerts

• Budget threshold alerts notify an external system when spending crosses a configured percentage of a budget. Notification channels support generic webhooks and Slack.
• The dashboard budgets page now manages budgets at every scope from one place, and budgets are part of the AISIX Cloud Admin API contract.

Prompt Caching

• Models can enable automatic prompt caching so the gateway inserts Anthropic cache breakpoints into qualifying requests. Callers get prompt-cache savings without changing their client code, and the option is configurable per model from the dashboard.

Usage Reporting

• When an upstream returns no usage block, the gateway now estimates token counts locally rather than recording zero. Estimated records are flagged as such in usage reporting and on the dashboard logs page, so estimated and reported usage stay distinguishable.
• The aisix_llm_tokens_by_client_total metric gained a model label, and the metric is now recorded for the responses endpoint.
• Built-in client type detection covers more coding agents, and operators can add their own user-agent mapping rules for clients the gateway does not recognize.

Deployment

• A new aisix export command writes a resources.yaml file from a running etcd store, which produces a starting configuration for standalone mode from an existing deployment.
• The gateway can run with the admin listener disabled through the admin.enabled setting, for deployments that configure the gateway declaratively and want no write API exposed.
• The status listener reports per-model runtime health, so upstream reachability can be checked without sending a model request.
• The gateway reports the hash of the configuration it has applied in its heartbeat, and the control plane exposes gateway nodes and rejected resources in its public spec. This makes it possible to confirm which gateways have taken up a configuration change. See Configuration Propagation.

Improvements

• The upstream secret of a provider key can be rotated in place without recreating the key or re-pointing the models that use it.
• The dashboard provider picker is searchable and lists providers under their proper display names.
• The dashboard logs page shows the whole upstream error message instead of truncating it, and its upstream filter no longer offers model groups, which are not upstreams.
• The logs query window follows the organization's usage retention setting instead of a fixed range.
• The control plane reports why a request was rejected instead of returning a generic failure.
• The Admin API write path is deprecated in favor of declarative configuration. It still works in this release. Declare dynamic resources in a resources.yaml file instead, or manage them through the AISIX Cloud Admin API.

Fixes

• The retries setting is now honored on streaming chat requests, which previously ran a separate code path that did not retry.
• On Azure models, stream_timeout applies to the gap between chunks as documented, instead of capping the whole response, so long but healthy streams are no longer cut off.
• Connection failures report the underlying transport cause rather than a generic upstream error, and the connection layer now applies explicit bounds instead of library defaults.
• Client-supplied cache_control markers survive the OpenAI-to-Anthropic bridge, so callers using an OpenAI client against an Anthropic model keep their prompt-cache savings.
• Ensemble panel and judge sub-calls estimate their usage when the backend reports none, so ensemble requests no longer under-report tokens.
• The passthrough tunnel enforces the rate limit of the model named in the request body.
• Group dispatch applies each routing target's own model rate limit and client IP allowlist, instead of only checking the group entry point.
• Per-attempt error messages are no longer clipped at 256 characters, so upstream failures stay readable in logs.
• Minted-token caches for providers that exchange credentials for short-lived tokens are keyed on the whole credential, so a key rotation takes effect immediately instead of after the old token expires.
• The dashboard pod mounts a writable cache directory, which fixes image requests that could hang indefinitely in Helm deployments.
• The playground accepts only dashboard sessions. A personal access token can no longer spend provider quota through it.

0.4.0

Release date: July 16, 2026

This release introduces role-based access control with custom roles and environment-scoped administration, adds status-code-based routing fallback, and expands observability with latency histograms and richer failure diagnostics.

New Features

Access Control

• Organizations can define custom roles that grant fine-grained read and write permissions per resource type, replacing the fixed owner, admin, and member split.
• Environment-scoped access grants a member administrative control over a single environment without granting organization-wide access.
• SCIM group-to-role mappings assign roles automatically from identity-provider group membership.
• The control plane enforces these permissions on every Admin API request.

Routing

• fallback_on_statuses opts selected upstream HTTP status codes into retry and failover, so provider-specific transient codes such as 408 or 409 try the remaining targets instead of returning to the caller.

Observability

• Bucketed time-to-first-token and end-to-end latency histograms support latency quantiles such as p90 and p99 for service-level objectives. See Metrics.
• The gateway captures the request body on failed requests to the chat, messages, and responses endpoints, applying structure-preserving truncation to keep large payloads bounded.
• A new /status/config endpoint reports the loaded observability configuration, and the gateway emits configuration metrics.

Deployment

• Standalone mode can load its resources from a resources.yaml file, so the gateway can run without a control plane or etcd.

Improvements

• The AISIX Cloud Admin API now exposes cache policies, observability exporters, and rate limits.
• The dashboard can set a pricing override for models that are not in the pricing catalog, so their usage is still billed.
• The API-key and rate-limit lists support pagination and search.
• Guardrail configuration errors now log the provider's error-response body to aid debugging.
• The Aliyun content-safety guardrail form exposes the output_fail_open option.

Fixes

• Monitor-mode output guardrails no longer hold back or close streaming responses.
• The aisix_deployment_state metric is now derived from the target's serving state.
• SCIM POST /Users returns the persisted identity instead of echoing the identity provider's payload.

0.3.1

Release date: July 9, 2026

This maintenance release adds SCIM directory sync and improves observability, accounting, on-premises playground access, and control-plane reliability.

New Features

• SCIM 2.0 directory sync can provision and deprovision organization members from any SCIM 2.0 identity provider, including Okta and Microsoft Entra ID, through the new /scim/v2 endpoints.

Improvements

• Gateway builds now report their release version in the Server response header, the output of aisix --version, and the data-plane version shown in the dashboard, rather than reporting a static build version.
• The per-client token metric now includes a cache-inclusive total series. See Metrics for the current metric catalog.
• Deployment cooldown metrics are emitted when health state changes.
• The guardrail management API is now included in the AISIX Cloud Admin API contract.

Fixes

• Every proxied response now includes an x-aisix-request-id header for correlation with logs and usage events.
• Anthropic prompt-cache tokens now count toward token rate limits on the native /v1/messages and /v1/responses endpoints.
• One malformed telemetry event no longer prevents the remaining events in the usage batch from being delivered.
• The on-premises dashboard playground can now reach private or internal LLM endpoints when AISIX_PLAYGROUND_ALLOW_PRIVATE_IPS is enabled.
• Sign-in now accepts the deployment's own origin and corresponding loopback origin, and returns clearer messages for rate-limited or untrusted-origin attempts.
• Member list pagination no longer returns to the first page shortly after the view loads.
• Control-plane restarts no longer produce harmless duplicate-constraint error logs.

0.3.0

Release date: July 9, 2026

This release introduces the MCP Gateway and Agent Gateway, expands the proxy API and guardrail catalog, and adds metric-based routing.

New Features

MCP Gateway

• The new aggregating /mcp endpoint fronts multiple upstream MCP servers behind one AISIX caller API key.
• Upstream MCP servers are first-class resources with registration and full CRUD in the control plane and dashboard, along with enable and disable controls and a configurable upstream timeout.
• Tool access control limits each caller API key to specific MCP tools.
• Upstream authentication supports API keys and OAuth 2.0 client credentials.
• MCP tool calls use the same rate limits, budgets, and input and output guardrails as model traffic. Calls also emit usage events and access logs.

Agent Gateway

• The new Agent Gateway fronts organization-scoped Agent-to-Agent (A2A) agents managed through the control plane and dashboard.
• The allowed_agents field limits each caller API key to specific agents.

APIs

• The /v1/realtime WebSocket relay supports OpenAI-family and Azure providers, browser subprotocol authentication, and per-session usage accounting.
• The new /v1/files, /v1/batches, and /v1/fine_tuning endpoints support OpenAI-family and Azure providers, with cost attribution for batch jobs.
• Embeddings support native Vertex AI, Gemini, and Bedrock Titan and Cohere requests.
• Anthropic Messages supports cross-provider image, tool-use, tool-result, and document content blocks.

Guardrails

• New guardrail integrations support Lakera Guard, Presidio, and OpenAI Moderation.
• PII detection and redaction can mask or block sensitive data in request and response bodies, including streamed responses and non-chat endpoints.
• Monitor-mode guardrail matches appear in usage events and the dashboard Logs view.

Routing

• Multi-target models support least-cost, least-latency, and least-busy target selection.
• Conditional routing can select targets by tags or metadata, and wildcard aliases can route model names such as provider/*.
• Sticky weighted routing supports A/B testing and canary releases.

Traffic Controls and API Keys

• Caller API key lifecycle controls can set an expiration, disable a key, or rotate it in one operation.
• Cluster rate limiting can use shared Redis storage and adds per-second (rps) and per-hour (rph) request limits to the existing per-minute (rpm) and per-day (rpd) limits.

Observability

• Request and response content capture now covers embeddings, rerank, images, and audio.

Dashboard

• MCP servers can be managed from the dashboard, with MCP governance available in the rate-limit, budget, and guardrail views.
• Usage-log retention can be configured by organization.

Improvements

• The container image runs as a non-root user and can bind ports 80 and 443 through the CAP_NET_BIND_SERVICE file capability.
• The dashboard provides a unified, filterable models view and a single model-kind picker when creating a model.
• Routing targets can be reordered by dragging, and least-cost targets display per-target cost badges.
• Member and team lists are paginated.

Fixes

• Cache and rate-limit keys are scoped by environment so shared Redis storage cannot mix state between environments.
• Passthrough endpoints now emit usage events for successful and failed requests, with caller API key attribution.