Skip to main content
API7.ai Logo

Release Notes

These release notes summarize user-visible changes to the AISIX gateway, control plane, dashboard, and deployment packages. Releases are listed newest first.

Release artifacts include the gateway image at docker.io/api7/aisix, control-plane images under docker.io/api7/aisix-cp-*, the aisix-cp Helm chart, and an offline installation package.

0.6.0​

Release date: July 29, 2026

This release lets callers authenticate with a JWT issued by their own identity provider instead of a gateway API key. It also replaces per-key MCP tool allowlists with a layered access policy resolved from the environment, the team, and the key. Alongside them, the retry budget moves onto the model, upstream requests gain request-context variables and a client-header allowlist, and video generation reaches two more providers.

Behavior Changes​

  • Retries are now enabled by default. Previously a model group that did not set routing.retries performed no retries at all. The budget now resolves from the model, then the model group, then a new deployment-wide upstream.retries default of 2, so a deployment that never configured retries starts retrying after this upgrade. Each retry re-sends the full request body and stacks on top of any retry the provider's own edge performs. To keep the previous behavior, set upstream.retries: 0 in the gateway configuration file. See Proxy Errors and Retries.

New Features​

Authentication​

  • Callers can authenticate with a JWT issued by an OIDC provider instead of a gateway API key. A new oidc_providers resource holds per-environment trust rows that pin the issuer, its accepted audiences, and its JWKS location, and the gateway validates every claim before the request proceeds. See JWT Authentication.
  • An API key binds to an external identity through a provider and subject pair, so a JWT-authenticated caller carries that key's budgets, rate limits, model access, and usage attribution. A subject is only ever resolved for the trust provider named on the key, so a second trusted issuer cannot assert a subject that belongs to another provider.
  • JWT authentication applies at the gateway's single authentication point, so every proxy surface accepts it, including chat completions, messages, responses, embeddings, rerank, audio, images, video generation, files, batches, fine-tuning, the MCP and agent endpoints, realtime WebSocket connections, and passthrough.
  • Validation is deny-by-default: only asymmetric signing algorithms are accepted, expiry and audience claims are required, the issuer must match an enabled trust row, and an operator can additionally require scopes or pin arbitrary nested claims.

MCP Access Control​

  • MCP tool access is now governed by a layered policy instead of an allowlist maintained on each key. An environment default applies to every caller, a team policy replaces it for that team's member keys, and a key narrows the result further. See MCP Access Policies.
  • A policy grants no tools, a named selection, or all tools. Granting all tools covers current and future tools and is an explicit choice rather than a default.
  • Deny patterns from every layer always subtract, so an environment-level deny survives a team policy and still applies to keys created before this release.
  • A key can only narrow what it inherits, never widen it. Keys that still use the previous allowed_tools field keep their existing behavior unchanged, so upgrading cannot silently grant access.

Video Generation​

  • The video generation endpoint adds two providers: Runway (runwayml, covering the Gen family and Runway-hosted Veo) and OpenAI Sora (openai).
  • The content route now delivers finished videos in two modes. For providers that return a signed download URL the content route still answers with a 302 redirect, so the transfer goes straight from provider storage to the client. OpenAI requires its own credential to download the file, so the gateway fetches it with the configured provider key and streams the bytes back without holding the file in memory. The provider credential is never exposed to the caller.
  • progress now reports a real completion percentage for OpenAI Sora. Providers that do not expose one continue to report 0 until the task completes and 100 afterward.
  • OpenAI is the only video provider with a built-in default base URL. The other four still require api_base on the provider key.

Upstream Request Headers​

  • Default header values on a provider key can reference the request context, for example "x-tenant-id": "${request.api_key.team_id}", and the gateway renders them per request. An internal model service can attribute traffic to the calling team or key without a separate provider credential per tenant. The variable vocabulary is closed and contains no secrets, and a header whose variables do not all resolve is dropped rather than sent with an empty value. See Upstream Request Headers.
  • A provider key can relay named inbound client headers upstream through an allowlist of exact names or single-wildcard patterns. It is empty by default, and authentication, transport, and gateway-internal headers are refused from a client even under a wildcard. This lets a caller pass a provider-specific header or propagate a trace context on the standard endpoints instead of dropping to passthrough.
  • Video generation and the files, batches, and fine-tuning endpoints previously applied no default headers at all. They now carry the resolved set on every request.

Retries​

  • The retry budget is now a model-level setting that applies on every endpoint rather than only the chat path. It resolves from the model, then the model group, then the deployment-wide default, and is configurable per model from the dashboard. See Proxy Errors and Retries.

Connection Management​

  • The gateway can bound how long an accepted client connection may sit idle between requests, and can send heartbeat comments on a streaming response that has not yet produced output, so a proxy in front does not treat a model that is slow to its first token as an abandoned connection. An in-flight request or stream is never interrupted. See Configuration Files.

Improvements​

  • The logs page supports free-text search across a request's model, key, error message, and identifiers, exports the current result set as CSV or JSON, and shows the date alongside the time on the timestamp column.
  • The models page can be filtered by model name, upstream model, or model ID.
  • The model ID field suggests the selected provider's catalog models while remaining free-text, so an unlisted or newly released model can still be entered by hand.
  • The API key and provider key pages use compact tables with unified search, type filters, and server-side paging, replacing the previous card layout.
  • The dashboard sidebar shows the control-plane build version, so an operator can confirm which release an environment is running without shell access.
  • The aisix-cp Helm chart adds startup probes to the control-plane API and data-plane manager, so a slow first boot or a long schema migration on upgrade is no longer cut short by the liveness probe.

Fixes​

  • Input guardrails now scan the union of a message's text content and its structured content blocks. The two are independent fields on the wire and the provider bridges forward the structured blocks when present, so a request carrying benign text alongside a payload in the blocks could previously pass every input guardrail while the model still received the payload.
  • A failed request's access log line now names the failure, carrying both a stable error category to filter or alert on and the underlying reason. Previously an upstream error, a name resolution failure, a reaped connection, and an unanswered connection attempt all produced an identical line. See Metrics and Logs.

0.5.0​

Release date: July 24, 2026

This release adds a unified video generation endpoint, budget threshold alerts that notify an external system, and automatic prompt caching for Anthropic models. It also corrects several cases where retries, timeouts, rate limits, and usage accounting did not apply on every request path.

New Features​

Video Generation​

  • A new video generation endpoint accepts prompt-to-video tasks on /v1/videos, polls their status, and returns the finished video, following the same three-stage shape as the OpenAI video API. Providers currently mapped are Alibaba Cloud Model Studio, Zhipu AI CogVideoX, and Volcengine Ark Seedance.
  • Video requests now pass through the same gateway controls as chat traffic: model aliases, caller API key access checks, client IP allowlists, model-level rate limits, and input guardrail scanning of the prompt. Previously video traffic could only reach a provider through passthrough, which applies none of these per-model controls.
  • Task state is not stored by the gateway. The returned video ID carries the routing information, so status and download calls work against any gateway instance.
  • Video submissions are recorded in usage logs with zero tokens. Duration-based cost accounting is not yet applied, so video traffic does not consume budgets in this release.

Guardrails​

  • A new Alibaba Cloud AI Guardrails kind calls the MultiModalGuard service and acts on its suggestion verdicts. When the service returns masked content, the gateway writes the masked text back into the request instead of rejecting it outright.
  • The gateway records aisix_guardrail_latency_seconds, a per-execution latency histogram that separates guardrail overhead from upstream model latency. See Metrics.
  • Responses from Alibaba guardrail services keep their upstream request ID, and the gateway correlates it with its own request ID for cross-system troubleshooting.

Budgets and Alerts​

  • Budget threshold alerts notify an external system when spending crosses a configured percentage of a budget. Notification channels support generic webhooks and Slack.
  • The dashboard budgets page now manages budgets at every scope from one place, and budgets are part of the AISIX Cloud Admin API contract.

Prompt Caching​

  • Models can enable automatic prompt caching so the gateway inserts Anthropic cache breakpoints into qualifying requests. Callers get prompt-cache savings without changing their client code, and the option is configurable per model from the dashboard.

Usage Reporting​

  • When an upstream returns no usage block, the gateway now estimates token counts locally rather than recording zero. Estimated records are flagged as such in usage reporting and on the dashboard logs page, so estimated and reported usage stay distinguishable.
  • The aisix_llm_tokens_by_client_total metric gained a model label, and the metric is now recorded for the responses endpoint.
  • Built-in client type detection covers more coding agents, and operators can add their own user-agent mapping rules for clients the gateway does not recognize.

Deployment​

  • A new aisix export command writes a resources.yaml file from a running etcd store, which produces a starting configuration for standalone mode from an existing deployment.
  • The gateway can run with the admin listener disabled through the admin.enabled setting, for deployments that configure the gateway declaratively and want no write API exposed.
  • The status listener reports per-model runtime health, so upstream reachability can be checked without sending a model request.
  • The gateway reports the hash of the configuration it has applied in its heartbeat, and the control plane exposes gateway nodes and rejected resources in its public spec. This makes it possible to confirm which gateways have taken up a configuration change. See Configuration Propagation.

Improvements​

  • The upstream secret of a provider key can be rotated in place without recreating the key or re-pointing the models that use it.
  • The dashboard provider picker is searchable and lists providers under their proper display names.
  • The dashboard logs page shows the whole upstream error message instead of truncating it, and its upstream filter no longer offers model groups, which are not upstreams.
  • The logs query window follows the organization's usage retention setting instead of a fixed range.
  • The control plane reports why a request was rejected instead of returning a generic failure.
  • The Admin API write path is deprecated in favor of declarative configuration. It still works in this release. Declare dynamic resources in a resources.yaml file instead, or manage them through the AISIX Cloud Admin API.

Fixes​

  • The retries setting is now honored on streaming chat requests, which previously ran a separate code path that did not retry.
  • On Azure models, stream_timeout applies to the gap between chunks as documented, instead of capping the whole response, so long but healthy streams are no longer cut off.
  • Connection failures report the underlying transport cause rather than a generic upstream error, and the connection layer now applies explicit bounds instead of library defaults.
  • Client-supplied cache_control markers survive the OpenAI-to-Anthropic bridge, so callers using an OpenAI client against an Anthropic model keep their prompt-cache savings.
  • Ensemble panel and judge sub-calls estimate their usage when the backend reports none, so ensemble requests no longer under-report tokens.
  • The passthrough tunnel enforces the rate limit of the model named in the request body.
  • Group dispatch applies each routing target's own model rate limit and client IP allowlist, instead of only checking the group entry point.
  • Per-attempt error messages are no longer clipped at 256 characters, so upstream failures stay readable in logs.
  • Minted-token caches for providers that exchange credentials for short-lived tokens are keyed on the whole credential, so a key rotation takes effect immediately instead of after the old token expires.
  • The dashboard pod mounts a writable cache directory, which fixes image requests that could hang indefinitely in Helm deployments.
  • The playground accepts only dashboard sessions. A personal access token can no longer spend provider quota through it.

0.4.0​

Release date: July 16, 2026

This release introduces role-based access control with custom roles and environment-scoped administration, adds status-code-based routing fallback, and expands observability with latency histograms and richer failure diagnostics.

New Features​

Access Control​

  • Organizations can define custom roles that grant fine-grained read and write permissions per resource type, replacing the fixed owner, admin, and member split.
  • Environment-scoped access grants a member administrative control over a single environment without granting organization-wide access.
  • SCIM group-to-role mappings assign roles automatically from identity-provider group membership.
  • The control plane enforces these permissions on every Admin API request.

Routing​

  • fallback_on_statuses opts selected upstream HTTP status codes into retry and failover, so provider-specific transient codes such as 408 or 409 try the remaining targets instead of returning to the caller.

Observability​

  • Bucketed time-to-first-token and end-to-end latency histograms support latency quantiles such as p90 and p99 for service-level objectives. See Metrics.
  • The gateway captures the request body on failed requests to the chat, messages, and responses endpoints, applying structure-preserving truncation to keep large payloads bounded.
  • A new /status/config endpoint reports the loaded observability configuration, and the gateway emits configuration metrics.

Deployment​

  • Standalone mode can load its resources from a resources.yaml file, so the gateway can run without a control plane or etcd.

Improvements​

  • The AISIX Cloud Admin API now exposes cache policies, observability exporters, and rate limits.
  • The dashboard can set a pricing override for models that are not in the pricing catalog, so their usage is still billed.
  • The API-key and rate-limit lists support pagination and search.
  • Guardrail configuration errors now log the provider's error-response body to aid debugging.
  • The Aliyun content-safety guardrail form exposes the output_fail_open option.

Fixes​

  • Monitor-mode output guardrails no longer hold back or close streaming responses.
  • The aisix_deployment_state metric is now derived from the target's serving state.
  • SCIM POST /Users returns the persisted identity instead of echoing the identity provider's payload.

0.3.1​

Release date: July 9, 2026

This maintenance release adds SCIM directory sync and improves observability, accounting, on-premises playground access, and control-plane reliability.

New Features​

  • SCIM 2.0 directory sync can provision and deprovision organization members from any SCIM 2.0 identity provider, including Okta and Microsoft Entra ID, through the new /scim/v2 endpoints.

Improvements​

  • Gateway builds now report their release version in the Server response header, the output of aisix --version, and the data-plane version shown in the dashboard, rather than reporting a static build version.
  • The per-client token metric now includes a cache-inclusive total series. See Metrics for the current metric catalog.
  • Deployment cooldown metrics are emitted when health state changes.
  • The guardrail management API is now included in the AISIX Cloud Admin API contract.

Fixes​

  • Every proxied response now includes an x-aisix-request-id header for correlation with logs and usage events.
  • Anthropic prompt-cache tokens now count toward token rate limits on the native /v1/messages and /v1/responses endpoints.
  • One malformed telemetry event no longer prevents the remaining events in the usage batch from being delivered.
  • The on-premises dashboard playground can now reach private or internal LLM endpoints when AISIX_PLAYGROUND_ALLOW_PRIVATE_IPS is enabled.
  • Sign-in now accepts the deployment's own origin and corresponding loopback origin, and returns clearer messages for rate-limited or untrusted-origin attempts.
  • Member list pagination no longer returns to the first page shortly after the view loads.
  • Control-plane restarts no longer produce harmless duplicate-constraint error logs.

0.3.0​

Release date: July 9, 2026

This release introduces the MCP Gateway and Agent Gateway, expands the proxy API and guardrail catalog, and adds metric-based routing.

New Features​

MCP Gateway​

  • The new aggregating /mcp endpoint fronts multiple upstream MCP servers behind one AISIX caller API key.
  • Upstream MCP servers are first-class resources with registration and full CRUD in the control plane and dashboard, along with enable and disable controls and a configurable upstream timeout.
  • Tool access control limits each caller API key to specific MCP tools.
  • Upstream authentication supports API keys and OAuth 2.0 client credentials.
  • MCP tool calls use the same rate limits, budgets, and input and output guardrails as model traffic. Calls also emit usage events and access logs.

Agent Gateway​

  • The new Agent Gateway fronts organization-scoped Agent-to-Agent (A2A) agents managed through the control plane and dashboard.
  • The allowed_agents field limits each caller API key to specific agents.

APIs​

Guardrails​

Routing​

  • Multi-target models support least-cost, least-latency, and least-busy target selection.
  • Conditional routing can select targets by tags or metadata, and wildcard aliases can route model names such as provider/*.
  • Sticky weighted routing supports A/B testing and canary releases.

Traffic Controls and API Keys​

  • Caller API key lifecycle controls can set an expiration, disable a key, or rotate it in one operation.
  • Cluster rate limiting can use shared Redis storage and adds per-second (rps) and per-hour (rph) request limits to the existing per-minute (rpm) and per-day (rpd) limits.

Observability​

  • Request and response content capture now covers embeddings, rerank, images, and audio.

Dashboard​

  • MCP servers can be managed from the dashboard, with MCP governance available in the rate-limit, budget, and guardrail views.
  • Usage-log retention can be configured by organization.

Improvements​

  • The container image runs as a non-root user and can bind ports 80 and 443 through the CAP_NET_BIND_SERVICE file capability.
  • The dashboard provides a unified, filterable models view and a single model-kind picker when creating a model.
  • Routing targets can be reordered by dragging, and least-cost targets display per-target cost badges.
  • Member and team lists are paginated.

Fixes​

  • Cache and rate-limit keys are scoped by environment so shared Redis storage cannot mix state between environments.
  • Passthrough endpoints now emit usage events for successful and failed requests, with caller API key attribution.