Skip to main content
API7.ai Logo

Release Notes

These release notes summarize user-visible changes to the AISIX gateway, control plane, dashboard, and deployment packages. Releases are listed newest first.

Release artifacts include the gateway image at docker.io/api7/aisix, control-plane images under docker.io/api7/aisix-cp-*, the aisix-cp Helm chart, and an offline installation package.

0.9.0​

Release date: August 13, 2026

This release makes the gateway substantially faster and makes agent traffic legible. The proxy now serves from thread-per-core workers, roughly doubling throughput and halving p99 latency. Caching learned to match requests that mean the same thing rather than only those spelled the same way. A2A calls now record which task they advanced and what they cost, and a verified JWT can stand in for a caller API key, so an identity provider's claims decide which caller a request runs as.

It also completes the Admin API deprecation announced in 0.4.0: the gateway's own Admin API is now read-only, and resources are managed declaratively or through AISIX Cloud.

Behavior Changes​

  • The gateway's Admin API no longer writes resources. The admin listener keeps its whole read surface — lists and gets for every resource kind, model status, health, the OpenAPI reference, and the playground — but POST, PUT, and DELETE on /admin/v1/<kind> now return 405 with Allow: GET, and both spellings of the API key rotate route return 404. This completes the deprecation announced in 0.4.0. Manage resources through a resources file (validate with aisix validate --resources <file>, reload with SIGHUP) or by writing to etcd; gateways connected to AISIX Cloud never exposed this listener and are unaffected. To rotate a caller key declaratively, write the same resource id with a new key_hash — the old secret stops authenticating as soon as the write propagates. The published Admin API reference no longer documents write operations. See Resources File.
  • Cached responses are isolated per caller API key by default. A cache policy's new scope field defaults to api_key, so one caller's answers are never served to another. Deployments that relied on environment-wide sharing must set scope: env explicitly. Either way the cache key changes shape, so expect a one-time full miss after upgrading. See Caching.
  • Time to first token is now measured at the first streamed frame of any kind. It previously waited for a frame carrying generated output, so a model that thinks silently before answering recorded the end of its thinking phase — a figure that could exceed the request's own reported latency and could not be compared with what a fronting proxy measures. Time spent thinking remains visible in upstream latency. Dashboards and alerts calibrated against the old figure will shift. See Metrics and Logs.
  • A request id supplied by the caller is now adopted as the gateway's own. The gateway accepts x-aisix-request-id by default: the value is echoed back, recorded in the access log and usage events, and forwarded upstream. Ids must be 1–256 visible-ASCII characters; anything else is ignored and the gateway generates its own id, so a malformed header never fails a request. Add x-request-id to proxy.request_id.accept_headers to honor that convention too, or set the list to empty to restore the previous behavior. Because the id is now caller-controlled it is neither unique nor trusted, and it is never used as a metric label. See Metrics and Logs.
  • Configuration a model kind never reads is now rejected instead of stored and ignored. retries, auto_prompt_caching, and cost on a model group, the generic call settings on an ensemble, and auto_prompt_caching and cost on a semantic router are refused at write time with 400. A resources file carrying one fails to load and names the offending entry. A model already stored with such a setting keeps working: the gateway drops the field and reports it as partially compatible rather than taking the model out of service. In the other direction, semantic routers and embedding models now accept timeout, stream_timeout, and retries, which they previously refused.
  • Several configurations that were silently accepted are now refused. Creating a model with a configuration block belonging to a different kind, referencing a wildcard alias as an ensemble member or judge, as a semantic router target or default, or as a cache policy's target, renaming a referenced model into a wildcard, and pointing a cache policy at a model that does not exist all return 400. Each was previously accepted and then quietly ignored.
  • Traffic through a wildcard model is reported under the wildcard's own name. Metrics, rate-limit buckets, and health tracking now key on the configured wildcard row rather than the alias each caller happened to send, so one model is one identity instead of a new series per spelling. A dashboard filtered on a caller-minted alias will show its series end at this release.
  • Upgrade note. The control plane widens the request_id column on its usage table from uuid to text so a caller's own id can be stored. On a large table this rewrites it — about six minutes per 10 million rows — and the control plane does not serve traffic until it finishes. The Helm chart's startup probe budget has been raised to 30 minutes to accommodate this; if you run your own manifests, raise yours before upgrading.

New Features​

  • The proxy serves from thread-per-core workers. Each worker has its own runtime, listener, and upstream connection pool, so a request is accepted, dispatched, and answered on a single thread instead of being handed between threads twice. On four cores this raises throughput by 54–88% depending on concurrency and roughly halves p99 latency, cutting system calls per request from 11.9 to 5.0. It is on by default on Linux. Two settings apply at startup: proxy.thread_per_core and proxy.workers, which defaults to the parallelism available to the process and follows a cgroup CPU limit. Below roughly four client connections per worker the kernel spreads connections unevenly and this mode is slower than the shared runtime; set proxy.thread_per_core: false for very low concurrency.
  • Semantic caching. A cache policy can now match on meaning: a request that misses exact matching is embedded and served from the nearest stored entry at or above a cosine similarity threshold you set. Only fully textual requests take that path — anything carrying images, audio, or tool calls always matches exactly. Entries can live in each gateway's memory or, with backend: redis, be shared across replicas using Redis vector search. Sharing requires Redis 8 or later, or the search module; the gateway checks at startup and, when it is missing, keeps serving exact matches and says so in the log rather than failing traffic. A policy can also be emptied without deleting it. See Semantic Caching.
  • JWT claim mappings. A new resource resolves verified OIDC claims to an existing caller API key, so an identity provider can decide which caller a request runs as without provisioning a key per user. Rules are evaluated in priority order and the first whose claim conditions all hold selects the key; the request then inherits that key's model and tool access, rate limits, and budget unchanged. Conditions support exact string matches and array membership over nested claim paths. A token that matches no rule is denied. Usage events record the subject, the provider, and the mapping that matched — note that the subject is an end-user identifier and reaches any configured observability exporter. See Claim Mappings.
  • A2A calls are now observable at the protocol level. Every call records its operation, task id, context id, and final task state, with both wire vocabularies normalized so one figure is not split in two. Streamed calls additionally record time to first event, how many events arrived, and whether the caller was still listening at the end, so a caller that hangs up mid-task is no longer counted as a success. Because the protocol carries no usage block, the gateway estimates tokens from the message text that passed through and marks them as estimated; cost is left at zero, since what an agent charges is not something the gateway can know. Metrics are sliced by agent and operation. See Agent Gateway.
  • The provider's own response id is now recorded. It appears in the access log and in a dedicated log line emitted once per upstream attempt, which covers streamed responses and retried or failed-over attempts that a single access-log line structurally cannot. The Dashboard shows it on the log detail panel, so a provider-side problem can be traced to the exact upstream call without joining records by hand.
  • Rate-limit policies accept a day window, counted against the UTC day. See Rate Limit Policies.
  • The Dashboard's request log is split by gateway surface. LLM, MCP, and A2A traffic each get their own tab with the filters that can actually narrow it, rather than one feed with filters that apply to only part of it.
  • The Dashboard's dropdowns are searchable. Every selector that can grow long — models, provider keys, API keys — now filters as you type instead of requiring a scroll through the full list.
  • The Dashboard reports gateway configuration compatibility. When a setting you save is not understood by every gateway in the environment, the Dashboard says so at save time and on the data-planes view, naming the field and the version that introduced it, instead of leaving the setting silently inert on older gateways.

Improvements​

  • Beyond thread-per-core, the per-request path was reworked throughout: a jemalloc allocator and link-time optimization, fast paths that skip machinery a zero-configuration deployment does not need, a per-worker metrics cache, TCP_NODELAY on downstream connections, cached upstream endpoint URLs, and a single configuration snapshot load per request.
  • A rate-limit condition on a model now matches the entry the caller addressed as well as the one the request was dispatched to, so a condition naming a model group applies to every request addressed to that group. Previously only the dispatched target was compared, and such a condition could never match.
  • A semantic router now honors its members' own access rules when it selects a route, falling through to another target instead of dispatching to a member the caller is not permitted to use.
  • Per-model rate limits now apply to every model kind, and the Dashboard offers the rate-limit form on every one.
  • A cache policy's target is now validated when the policy is created, and renaming a model that a policy references updates the reference.

Fixes​

  • An invitation that has expired no longer keeps its email address reserved, so the same person can be invited again. Pending invitations still block a duplicate.
  • Creating a resource with a boolean field explicitly set to false now stores false. A column default was substituting true for the zero value, so an OIDC provider created as disabled was stored as enabled and the API reported it as enabled.
  • Post-authentication refusals on the realtime endpoint are now attributed to the caller that was resolved, instead of being recorded without an identity.
  • Gateways that have stopped sending heartbeats no longer count toward the Dashboard's configuration-compatibility banner, so a decommissioned instance cannot make a healthy fleet look partially incompatible.
  • A gateway built from source without version stamping reports a placeholder version; that placeholder is now treated as unknown rather than as a real old version, so such a fleet no longer warns forever about features it may well support.

0.8.2​

Release date: August 11, 2026

This is a maintenance release for the A2A gateway. The gateway now announces the wire version each agent is pinned to, finds an agent card published under a path prefix, keeps every address on the card pointed at itself, and relays message/stream as it arrives instead of buffering it. It also applies the agent and MCP server naming and credential rules to every configuration path, not only the gateway's own Admin API.

Behavior Changes​

  • A2A agent and MCP server definitions supplied through a resources file are now checked against the same rules the Admin API enforces: the name shape, the credential each auth_type requires, and the fields an OpenAPI-backed MCP server needs. A file that breaks one of them no longer loads, and the gateway reports the offending entry and the missing property by name rather than starting with a definition that cannot work — an agent named with a / splits its own /a2a/<name> route in two, and auth_type: bearer without a secret authenticates upstream with nothing. Deployments configured through the control plane are unaffected, since those rules already applied when the resource was created; a gateway managed by the control plane rejects only the offending entry and keeps serving the rest. Check a file before rolling it out with aisix validate --resources <file>. See Agent Gateway.

Fixes​

  • The gateway now announces an agent's pinned protocol version in A2A-Version on every call it makes to that agent, including the agent-card fetch. Nothing sent the header before, and the A2A specification tells an agent to read its absence as version 0.3 — so an agent pinned to 1.0 answered VersionNotSupportedError and the configured protocol_version had no effect. A version supplied by the caller does not override the registered pin.
  • An A2A agent whose card is published under a path prefix now resolves. The well-known card URI was built at the origin, discarding the path the agent was registered with, so any agent behind an ingress path or on a platform that multiplexes tenants under a prefix was asked for a card it does not serve there — and the platform's catch-all 405 came back as the agent's own answer.
  • The agent card the gateway serves now points every advertised address back at the gateway. Only the top-level url was rewritten before, leaving the upstream address in supportedInterfaces — and a 1.0 caller reads its endpoint from there, so it could bypass the gateway, and with it the caller authentication and per-agent access control, and reach the agent directly.
  • message/stream and tasks/resubscribe now relay the agent's event stream as it arrives. Both were served through the single-response path, which buffered the whole body and parsed it as one JSON document; an event stream is not valid JSON, so the two methods whose purpose is watching a long-running task both failed with 502 and could not be used at all. Events are reassembled across chunk boundaries, and a malformed event ends the stream with an error rather than being skipped, so a truncated task cannot read as a completed one.

0.8.1​

Release date: August 7, 2026

This is a maintenance release. It corrects how the gateway decides that an upstream speaks the Anthropic protocol, so a model pointed at a self-managed Anthropic endpoint is served on the whole /v1/messages family with its request body passed through unchanged. It also gives authentication denials enough context to investigate, carries tool grants through an MCP server rename, and fixes several control-plane admission paths that had no workable request shape or no actionable error.

Behavior Changes​

  • Enabling auto_prompt_caching now requires a model whose provider key speaks the Anthropic protocol, either provider: anthropic or a BYO key whose adapter is Anthropic. The setting is implemented by injecting Anthropic cache_control markers, so on any other upstream it was stored but never took effect, and nothing told the operator that their prompt caching was off. Enabling it elsewhere now returns 400. Disabling remains valid on every provider, so a model configured before this release can still be cleaned up, and a key whose adapter cannot be determined is allowed rather than rejected on a guess. See Prompt Caching.
  • A model configured as provider: byo with the Anthropic adapter is now served on /v1/messages by passing its request body through unchanged, instead of re-encoding it through the cross-provider bridge. A cache_control marker set by the client reaches the upstream as written; it was previously dropped, or rewritten to the model's configured TTL when auto_prompt_caching was enabled. This also changes which upstream cache-write price applies, since a one-hour write costs twice the base input price and a five-minute write costs 1.25 times. Authentication is unchanged.

Improvements​

  • Authentication denials now carry the request context needed to act on them: the caller address resolved through the trusted-proxy configuration, the HTTP method and path, and the request id. Denials for a disabled or expired key also carry the key id. A 401 is refused before any handler runs and therefore never reaches the access log, which left the denial counter as the only record and it cannot say who, when, or against which route. Log levels are unchanged, so scanner traffic stays at debug and the default-level output is exactly as before. See Metrics and Logs.
  • The AISIX Cloud Admin API reference now describes provider request headers and conditional rate-limit counter semantics correctly. It previously said that changing a policy's conditions resets the current windows; counter keys are built from the policy id and the selected group_by values, so changing only conditions or limits preserves existing keys and their counts, while changing the grouping dimensions moves requests to different keys.

Fixes​

  • /v1/messages/count_tokens now serves a model whose provider key speaks Anthropic through an adapter rather than by vendor id. A provider: byo model pointed at an Anthropic-protocol endpoint was rejected with 400 on this endpoint while the sibling /v1/messages served it normally.
  • Renaming an MCP server now carries the tool grants with it. Tool names are <server>__<tool>, so every grant references the server by name; the per-key rate limit entries already followed a rename but the access control did not, which silently revoked access to the whole server and made its tools disappear from tools/list with no explanation. The caller key's allowed_tools and mcp_access, and the environment- and team-level access policies, are all rewritten. Wildcards that match a shape rather than one server, such as a bare *, are left alone. See MCP Access Control.
  • Provider keys for Amazon Bedrock and Google Vertex can now be created by supplying config alone. Their credentials do not fit in a single api_key string, but api_key was required by the schema and a non-empty value was rejected by the handler, so the only accepted request was an empty string that no documentation mentioned. A request missing its credential now names the field that provider actually accepts. See Provider Keys.
  • PATCH with "tls": null now clears a provider key's TLS settings, matching how rate_limit and the other optional blocks behave. It previously returned 400 and the setting could only be cleared with an empty object.
  • The usage feed and its CSV export now include audio_duration_seconds. A transcription priced by duration showed a cost whose basis could not be read back from the API. The field is omitted for requests that carry no audio. See Audio.
  • The offline package no longer lets two installations become one. The Compose project name came from the directory the package was unpacked into, so unpacking a second copy elsewhere and running run.sh took over the running installation, rebuilding its containers against the second copy's configuration while keeping the original data volume. The project name is now fixed to the package, and starting it while another directory holds the same stack is refused with both paths and the ways forward. Upgrading in place is unaffected, and a second independent installation can still be started with an explicit COMPOSE_PROJECT_NAME. See On-Premises Quickstart.
  • The Dashboard container in the offline package now has a writable Next.js cache directory, matching the Helm chart.
  • Trusted-proxy CIDRs and the User-Agent client-type rules can now be set from the environment. proxy.real_ip.trusted_proxies and observability.metrics.client_type_rules could only be supplied in a configuration file; setting either through its AISIX_* variable failed to start the gateway. Deployments configured purely through environment variables — the Helm chart and the Dashboard's docker run snippet — therefore could not declare their load balancer as a trusted proxy, so every request appeared to come from it. See Metrics and Logs.
  • Two admission errors now say what to change. A weight on a target in a group that is not weighted names the field and the strategy instead of reporting a generic invalid-fields error, and an unrecognized provider names the rejected value and explains that the catalog route needs a Dashboard session rather than an admin token.
  • The Dashboard now lets one API key allow several model groups. Checking one group greyed out the rest, so a key could only ever reference one, although the API always accepted a longer list and the edit dialog never enforced the limit. The two dialogs now share one picker, which also fixes model groups, ensembles, embedding models, and semantic routers rendering with an empty arrow in the edit dialog.

0.8.0​

Release date: August 6, 2026

This release makes rate limiting expressive enough to describe a real quota policy. A single policy now decides which traffic it applies to through a condition tree, splits its counters along any dimension of that traffic, caps seven different quantities, and can suspend itself on a recurring schedule such as weekday off-peak hours, whole weekends, or named holidays. MCP servers gain an endpoint that serves one server and keeps its original tool names, so a client written against that server works unchanged. Token, spend, and request metrics now cover every endpoint that reports usage rather than only chat and messages, and audio transcription is priced by the duration of the audio.

Behavior Changes​

  • The gateway now treats api_base as the upstream root on every OpenAI-family endpoint. Previously /v1/chat/completions appended the endpoint to the configured base verbatim, while /v1/responses, /v1/rerank, /v1/audio/*, /v1/realtime, /v1/files, /v1/batches, and /v1/fine_tuning/jobs inserted a /v1 segment unless the base already ended in one, so a key whose upstream root was not /v1 served chat correctly and returned 404 everywhere else. Both paths now read the base the same way. A base carrying a path but no version segment builds https://proxy.corp/openai-shim/responses where it previously built .../openai-shim/v1/responses; move the /v1 into api_base if you relied on the old behavior. A bare host still has /v1 added, a base ending in /v1 is unchanged, and Anthropic is unchanged in every base shape. See OpenAI-Compatible Vendors.
  • Histogram bucket boundaries changed for two metrics. aisix_request_ttft_seconds drops its two edges below 50 ms, because it measures the upstream time to first token and never observes values there, and aisix_request_e2e_latency_seconds gains edges at 420 and 600 seconds so histogram_quantile() interpolates past 300 seconds instead of pinning to it. Dashboards and recording rules that depend on the previous edges need updating. Both sets are now configurable per metric under observability.metrics.buckets. See Metrics Reference.
  • Time to first token now counts a reasoning model's first reasoning delta. Only ordinary content stamped the measurement before, so a model that reasons before it answers reported a later first token than the caller observed. Reported values for those models drop accordingly. See Metrics and Logs.
  • GET /v1/models now lists model groups alongside direct models, semantic routers, and ensembles. A model group was the only virtual alias filtered out of the listing, so a caller key scoped to a group as its public entry point received an empty list. No authorization changed, since those names were always callable by those keys. Only clients that assert an exact model list are affected. See Model Aliases.

New Features​

Rate Limiting​

  • A rate limit policy can now be written in a conditional form that carries a tree of conditions instead of a single scope. Conditions match on team, member, caller API key, model, model name, and provider, combine through explicit and and or groups up to three levels deep, support negation, and match strings by equality, list membership, or regular expression. One resource can therefore hold different quotas for different traffic instead of one flat limit. See Rate Limit Policies.
  • A conditional policy splits its counters with group_by, which buckets the counters per team, per member, per caller key, per model, or any combination of those. A single policy now enforces a per-tenant quota that previously required one policy per tenant.
  • A policy can cap seven quantities in any combination: requests per second, minute, hour, and day, tokens per minute and per day, and concurrent requests.
  • A policy that matches on a model property reserves its quota where the concrete model is known, which is per target for a routing or ensemble parent. An over-limit target is treated as a failed attempt and fails over to the next one, rather than consuming the parent alias's own budget.
  • A rate limit rejection now identifies the policy that produced it. The 429 response body carries the policy's id and name, and aisix_ratelimit_rejections_total counts rejections on every endpoint and is labeled by the enforcing layer and policy. With several policies live, an unattributed rejection could not be traced to its cause.
  • A policy can carry recurring suspension windows during which it is not enforced. Each window selects days either by weekday or by explicit dates, takes wall-clock start and end times in its own IANA time zone, and treats an end at or before the start as crossing midnight and belonging to the day it starts on. Multiple windows form a union, enforcement resumes automatically, and the counters are not reset by the transition, so suspending and resuming inside one rate window cannot be used to clear consumed quota. In AISIX Cloud, the Dashboard exposes these as a schedule list on the rate limit policy form. See Rate Limit Policies.

MCP Gateway​

  • A new per-server endpoint at /mcp/{server} serves a single registered MCP server and lists its tools under their original names, without the <server>__ prefix the aggregate endpoint applies. Both the bare and the prefixed spelling are accepted when calling a tool, and tool access control is always evaluated on the prefixed form. A client written against one server's own tool names therefore works through the gateway unchanged. See MCP Gateway Overview.

Deployment​

  • A new proxy.url_rewrites setting rewrites the request path at the gateway entry, matched by regular expression with capture-group templates in the replacement. Patterns are validated at startup and can be supplied through an environment variable for deployments that ship no configuration file. This lets a client whose base URL cannot be changed reach a different gateway path. See URL Rewriting.

Pricing​

  • Audio models can now be priced by the duration of the audio rather than by tokens. A transcription request reports the audio's length as its cost basis, and the model's price is expressed per minute of audio. In AISIX Cloud, the Dashboard accepts these rates on the model pricing form. See Audio and Model Pricing.

Improvements​

  • The detailed request metrics and the token and spend metrics now cover every endpoint that reports usage, including responses, embeddings, rerank, audio, images, and the realtime surface. They previously covered only chat completions and messages, so usage on the other endpoints was recorded in the usage log but missing from the metrics. See Metrics Reference.
  • Configuration reads are now forward compatible. A resource document carrying a field this gateway version does not recognize loads and serves, with the unknown fields ignored and reported through partially_compatible on the configuration status endpoint, a deduplicated warning, and a new gauge. Previously an unknown field rejected the whole document, which turned every additive change on the control plane into a breaking one for a gateway that had not yet been upgraded — a caller API key carrying a new field stopped authenticating. Write paths remain strict and still reject unknown fields. See Configuration Status.
  • A rejected configuration update no longer costs the last good value. The gateway keeps serving the value it previously loaded for that key across a resynchronization and a restart, instead of dropping the row. See Configuration Propagation.
  • Errors that a provider reports inside an already committed stream are now surfaced with the provider's own status and message, across OpenAI-family, Anthropic, Gemini, and Amazon Bedrock streams. An Anthropic error event arriving mid-stream was previously discarded and the truncated stream closed as though it had completed, so a client could not tell a cut-off response from a finished one. Such a stream now ends with an error frame and no completion marker. See Streaming.
  • A request refused for exceeding the body limit now reports how the reading of its body ended, through a dedicated metric, so a refusal whose client disconnected is distinguishable from one whose body was read to completion. See Metrics and Logs.
  • The gateway warns once when it reaches a credentialed MCP, OpenAPI, or A2A upstream over cleartext http, deduplicated per server and address, so the mistake is visible without filling the log. See MCP Upstream Authentication.
  • A request to /v1/realtime that is not a WebSocket upgrade is now recorded and answered in the gateway's own error format, and path rejections elsewhere in the proxy go through the same path as other early refusals, so they appear in the access log and the request count. See Realtime.
  • The AISIX Cloud Admin API reference now documents every operation and schema it exposes, and its base URL is an editable field rather than a fixed relative path. The reference is served from the API7 documentation site, not from the reader's control plane. Readers can therefore set the base URL to their Hybrid Cloud or On-Premises control plane so the examples use the correct endpoint.

Fixes​

  • A streaming transcription now records the usage it reports. Such a request logged zero tokens and therefore cost nothing, while the same transcription made without streaming was billed normally. See Audio.
  • The offline package's quickstart script now prints a Dashboard address that works. It previously printed the container's internal port, so following the printed URL failed on the default installation path. See On-Premises Quickstart.
  • The Dashboard's model pricing form now accepts audio rates below one cent per minute, which the audio pricing of most providers requires.
  • Budget threshold guidance in the Dashboard no longer suggests that a hard stop can overshoot its limit by an arbitrary amount. See Budgets.

0.7.1​

Release date: July 31, 2026

This release makes the gateway deployable on networks it could not previously reach. Outbound connections gain a trust configuration, so a model endpoint, guardrail, or MCP upstream whose certificate is signed by a private or enterprise authority can be dialed without a process-wide workaround, either deployment-wide or per endpoint on the provider key that declares it. A data plane running on a separate host and addressed by an IP address can now complete the mutual TLS handshake and join its control plane. Requests that a caller abandons, and requests the gateway refuses before dispatch, are no longer invisible in the access log and metrics.

Behavior Changes​

  • A streaming response that the caller abandons mid-delivery is now recorded as 499 rather than 200. The usage event previously reported a completed delivery, so a caller that closed the connection halfway through a stream could not be told apart from one that read the stream to the end. Reports and dashboards that select on status 200 will see these requests move to 499. See Metrics and Logs.

New Features​

Outbound TLS​

  • A new upstream.tls setting configures the trust the gateway uses on every connection it dials out on, through ca_file, client_cert_file, client_key_file, and verify. It applies to provider endpoints, all guardrail providers, MCP and A2A upstreams, passthrough routes, JWKS and OIDC discovery, and OpenTelemetry export. An upstream behind a private certificate authority previously failed with a generic connection error, and the only workaround was the process-wide SSL_CERT_FILE environment variable, which several outbound paths did not honor. See TLS and mTLS.
  • A provider key can carry its own tls block, with an inline ca_cert and a verify flag, so a deployment facing more than one private authority declares trust where it declares the endpoint. The certificate is supplied inline rather than as a file path, because whoever configures a provider key has no way to place a file on the gateway host. In AISIX Cloud, the dashboard exposes this as an Endpoint TLS section on the provider key form.
  • A rediss:// cache or rate-limit backend takes its own tls block with the same fields, since it usually sits inside your own deployment and is issued by a different authority than the model endpoints. Two limits are documented rather than silently ignored: Amazon Bedrock supports ca_file but neither client certificates nor verify: false, and Redis Sentinel mode supports verify but not ca_file.

Improvements​

  • Requests refused before dispatch now appear in the access log and in aisix_requests_total. A body over request_body_limit_bytes returned a correct 413 and otherwise left no trace, which made a client reporting a rejection the gateway had no record of indistinguishable from a request that never arrived. The 400 for conflicting Content-Length headers had the same gap. These rejections are refused before authentication, so they carry no usage event and do not appear on the logs page in AISIX Cloud.
  • Requests abandoned before the response head are recorded instead of vanishing. Such a request was previously absent from the access log, the usage events, and the metrics at the same time, which hid the case operators most want to see: a caller giving up during a long time to first token. A new counter, aisix_proxy_client_cancelled_requests_total, labeled by endpoint, counts them. See Metrics and Logs.

Fixes​

  • A data plane on a separate host, pointed at its control plane by an IP address, can complete the mutual TLS handshake and join. The data-plane manager issued a server certificate whose subject alternative names came only from its listen address and from the server name the client sent, and in a container neither yields the control plane's external address: the listener is bound to all interfaces, so there is no host to read, and a client dialing an IP address sends no server name at all. The advertised endpoint is now passed to the data-plane manager, which issues the certificate for that host, so an IP address works as well as a DNS name. Set it through AISIX_CLOUD_DPMGR_BASE_URL, or api.dpmgrBaseURL in the Helm chart. See On-Premises Deployment.

0.7.0​

Release date: July 31, 2026

This release lets the AISIX gateway generate MCP tools for supported operations described in a REST API's OpenAPI 3.x document, without requiring a separate MCP server. Available in both the open-source AISIX gateway and AISIX Cloud, this capability calls the API and injects the configured credential. MCP servers also gain a per-caller rate limit scoped to a single server. AISIX Cloud adds a review workflow that can require approval before a server reaches callers. Alongside them, every model now has an upstream deadline even when none is configured, and usage records separate what the upstream took from what the caller actually waited for.

Behavior Changes​

  • Usage event latency fields are renamed to make their scope explicit, and a new field records the caller-facing wait. latency_ms becomes upstream_latency_ms and ttft_ms becomes upstream_ttft_ms; both remain scoped to a single attempt. upstream_ttft_ms is now measured from the start of that attempt rather than from request entry, so it is directly comparable with the latency beside it. The new downstream_latency_ms is scoped to the whole request. On the OpenTelemetry export the span attribute aisix.ttft_ms becomes aisix.upstream_ttft_ms, and aisix.downstream_latency_ms is added, so any dashboard or alert reading those names needs updating. The control plane accepts both the old and the new names, so a gateway managed by AISIX Cloud needs no change. See Metrics and Logs.
  • Models that configure no timeout are no longer unbounded. A new deployment-wide upstream.timeout_ms defaults to 6000000, that is 6000 seconds, and applies to any model that sets neither timeout nor stream_timeout. Previously such a model had no upstream deadline at all, so an upstream that accepted the connection and then went silent held the request open indefinitely. The default is deliberately generous, so it is a backstop rather than a responsiveness target. To restore the previous behavior deployment-wide, set upstream.timeout_ms: 0; a single model opts out with timeout: 0. See Configuration Files.
  • The timeout and stream_timeout fields on a model group previously had no effect, because members only ever used their own values. They now apply to members that set neither, resolving from the member, then the group, then the deployment default. A model group that already carried these fields starts applying them after this upgrade.
  • proxy.request_body_limit_bytes now defaults to 0, which means no cap, instead of 10 MiB. Providers accept larger requests than any fixed gateway default, so a client that worked directly against the provider could break when routed through the gateway. This applies only when the setting is absent from the configuration file; a deployment whose file carries the value explicitly keeps it. To keep a cap on a configuration that omits the setting, set request_body_limit_bytes: 10485760. The value 0 previously meant "reject every request that has a body" and now means "no cap".

New Features​

MCP Gateway​

  • Both the open-source AISIX gateway and AISIX Cloud support MCP servers backed by plain REST APIs. Setting a server's type to openapi and supplying an OpenAPI 3.x document makes the gateway generate one MCP tool per supported operation and execute each tool call as an HTTP request against the API. The gateway holds the configured credential, injects it into outbound calls, and does not expose it to the calling agent. See Expose a REST API as MCP Tools.
  • Generated tools go through the same gateway controls as tools from a real MCP upstream, including tool access policies, rate limits, guardrails, and usage records.
  • In AISIX Cloud, operators can paste the OpenAPI document, provide a URL for the control plane to fetch once, or load a local file through the dashboard. Each OpenAPI-backed server has a tools page for inspecting the generated tool names and operations.
  • AISIX Cloud can require review before publishing an MCP server. A submitted server stays invisible to callers until an approver accepts it, and a change proposed to a server that is already live waits for review without taking the running server offline. See Review MCP Servers Before They Go Live.
  • In both products, an API key can carry per-server rate limits, so an agent looping on one MCP server does not consume the same key's allowance for another server. Limits are expressed per second, minute, hour, or day, plus concurrency, and only tool calls are metered, so a client can still connect and list tools after a server-specific allowance is exhausted. The key's own rate limit still applies on top. See Rate Limits and Budgets.

Timeouts​

  • Two new gateway settings, upstream.timeout_ms and upstream.stream_timeout_ms, provide deployment-wide defaults for the request deadline and the maximum gap between streaming chunks. Deadlines resolve from the model, then the model group, then these defaults, and apply across every endpoint rather than only the chat path. See Configuration Files.
  • Model groups expose timeout and stream_timeout in the control plane and dashboard, and an explicit 0 on a model is now preserved rather than normalized away, so a model can opt out of the deployment default.

Observability​

  • Usage records now report what the caller waited for, measured where the gateway hands bytes to the client rather than where the upstream chunk arrives. Under an output guardrail that masks the response, the stream is held back until the whole response scans clean, and only the caller-facing figure reflects that wait. The dashboard's latency percentiles use this figure and count only successful requests. See Metrics and Logs.
  • Subtracting the upstream time-to-first-token from the caller-facing latency isolates the wait the upstream did not account for, which makes gateway-side overhead visible without correlating two systems. On a request that retried, that difference also contains the earlier attempts and the delay between them.

Audit Log​

  • The audit log supports free-text search across an event's actor, action, and target, an absolute time range, page-by-page navigation, and export of the current result set. See Logging and Auditing.

Deployment​

  • The data planes page offers a real Helm installation tab alongside the manifest-based one, so a data plane can be installed from the published aisix chart with the environment's connection settings already filled in.

Improvements​

  • An idle gateway stays in service. The readiness endpoint previously reported a gateway unready once its configuration watch had gone five minutes without an event, which an environment whose resources are not changing produces routinely. Under Kubernetes every replica crossed that threshold at the same moment and the service lost all its endpoints while perfectly healthy. Readiness now reports only whether the gateway is draining and whether configuration has been applied. Configuration freshness remains observable through the health endpoint, the configuration status endpoint, and the aisix_config_* metrics. See Config Status.
  • The aisix data-plane Helm chart now uses that readiness endpoint on the proxy port for its readiness probe.
  • Requests that exceed a configured body limit return the same error envelope on every route that accepts a body. Previously several JSON endpoints and both raw-body endpoints returned a plain-text rejection, multipart uploads reported a generic client error, and the MCP and agent endpoints returned a bare 400. Malformed JSON on those endpoints likewise now returns the standard envelope. See Headers and Error Codes.
  • Amazon Bedrock models honor their configured deadline and connect timeout. The AWS SDK was building its own HTTP client with its own defaults, so a Bedrock model's timeout was not enforced.
  • MCP upstream connections honor the gateway's upstream connection settings, including the connect timeout, TCP keepalive, and connection pool sizing.
  • Prometheus metric memory no longer grows without bound, and metric series handles are resolved once and cached instead of being looked up on every request.
  • Released gateway binaries keep their symbol table, so a profile taken against a shipped image is readable without a special build.
  • The overview page shows the full model name in its top-models list instead of truncating it.

Fixes​

  • The latency recorded for the attempt that succeeded is scoped to that attempt. A request that failed over previously reported the winning attempt's latency measured from request entry, so it included every earlier attempt and the delay between retries, which made a successful failover look like a slow upstream.
  • Renaming an MCP server carries its per-server rate limits along, and deleting the server removes them. Previously the limits stayed behind under the old name and silently stopped applying.
  • An approver editing an MCP server publishes it instead of taking it offline.
  • Providers that the gateway supports natively but the public model catalog does not list can be configured again. They were being rejected during validation.
  • Regional provider base URLs match the catalog's provider identifiers, so a region-specific default resolves instead of falling through.
  • Audit records are no longer lost when a client disconnects mid-request. The control plane rolls back cleanly, records the disconnect once, and returns a consistent response body.

0.6.0​

Release date: July 29, 2026

This release lets callers authenticate with a JWT issued by their own identity provider instead of a gateway API key. It also replaces per-key MCP tool allowlists with a layered access policy resolved from the environment, the team, and the key. Alongside them, the retry budget moves onto the model, upstream requests gain request-context variables and a client-header allowlist, and video generation reaches two more providers.

Behavior Changes​

  • Retries are now enabled by default. Previously a model group that did not set routing.retries performed no retries at all. The budget now resolves from the model, then the model group, then a new deployment-wide upstream.retries default of 2, so a deployment that never configured retries starts retrying after this upgrade. Each retry re-sends the full request body and stacks on top of any retry the provider's own edge performs. To keep the previous behavior, set upstream.retries: 0 in the gateway configuration file. See Proxy Errors and Retries.

New Features​

Authentication​

  • Callers can authenticate with a JWT issued by an OIDC provider instead of a gateway API key. A new oidc_providers resource holds per-environment trust rows that pin the issuer, its accepted audiences, and its JWKS location, and the gateway validates every claim before the request proceeds. See JWT Authentication.
  • An API key binds to an external identity through a provider and subject pair, so a JWT-authenticated caller carries that key's budgets, rate limits, model access, and usage attribution. A subject is only ever resolved for the trust provider named on the key, so a second trusted issuer cannot assert a subject that belongs to another provider.
  • JWT authentication applies at the gateway's single authentication point, so every proxy surface accepts it, including chat completions, messages, responses, embeddings, rerank, audio, images, video generation, files, batches, fine-tuning, the MCP and agent endpoints, realtime WebSocket connections, and passthrough.
  • Validation is deny-by-default: only asymmetric signing algorithms are accepted, expiry and audience claims are required, the issuer must match an enabled trust row, and an operator can additionally require scopes or pin arbitrary nested claims.

MCP Access Control​

  • MCP tool access is now governed by a layered policy instead of an allowlist maintained on each key. An environment default applies to every caller, a team policy replaces it for that team's member keys, and a key narrows the result further. See MCP Access Policies.
  • A policy grants no tools, a named selection, or all tools. Granting all tools covers current and future tools and is an explicit choice rather than a default.
  • Deny patterns from every layer always subtract, so an environment-level deny survives a team policy and still applies to keys created before this release.
  • A key can only narrow what it inherits, never widen it. Keys that still use the previous allowed_tools field keep their existing behavior unchanged, so upgrading cannot silently grant access.

Video Generation​

  • The video generation endpoint adds two providers: Runway (runwayml, covering the Gen family and Runway-hosted Veo) and OpenAI Sora (openai).
  • The content route now delivers finished videos in two modes. For providers that return a signed download URL the content route still answers with a 302 redirect, so the transfer goes straight from provider storage to the client. OpenAI requires its own credential to download the file, so the gateway fetches it with the configured provider key and streams the bytes back without holding the file in memory. The provider credential is never exposed to the caller.
  • progress now reports a real completion percentage for OpenAI Sora. Providers that do not expose one continue to report 0 until the task completes and 100 afterward.
  • OpenAI is the only video provider with a built-in default base URL. The other four still require api_base on the provider key.

Upstream Request Headers​

  • Default header values on a provider key can reference the request context, for example "x-tenant-id": "${request.api_key.team_id}", and the gateway renders them per request. An internal model service can attribute traffic to the calling team or key without a separate provider credential per tenant. The variable vocabulary is closed and contains no secrets, and a header whose variables do not all resolve is dropped rather than sent with an empty value. See Upstream Request Headers.
  • A provider key can relay named inbound client headers upstream through an allowlist of exact names or single-wildcard patterns. It is empty by default, and authentication, transport, and gateway-internal headers are refused from a client even under a wildcard. This lets a caller pass a provider-specific header or propagate a trace context on the standard endpoints instead of dropping to passthrough.
  • Video generation and the files, batches, and fine-tuning endpoints previously applied no default headers at all. They now carry the resolved set on every request.

Retries​

  • The retry budget is now a model-level setting that applies on every endpoint rather than only the chat path. It resolves from the model, then the model group, then the deployment-wide default, and is configurable per model from the dashboard. See Proxy Errors and Retries.

Connection Management​

  • The gateway can bound how long an accepted client connection may sit idle between requests, and can send heartbeat comments on a streaming response that has not yet produced output, so a proxy in front does not treat a model that is slow to its first token as an abandoned connection. An in-flight request or stream is never interrupted. See Configuration Files.

Improvements​

  • The logs page supports free-text search across a request's model, key, error message, and identifiers, exports the current result set as CSV or JSON, and shows the date alongside the time on the timestamp column.
  • The models page can be filtered by model name, upstream model, or model ID.
  • The model ID field suggests the selected provider's catalog models while remaining free-text, so an unlisted or newly released model can still be entered by hand.
  • The API key and provider key pages use compact tables with unified search, type filters, and server-side paging, replacing the previous card layout.
  • The dashboard sidebar shows the control-plane build version, so an operator can confirm which release an environment is running without shell access.
  • The aisix-cp Helm chart adds startup probes to the control-plane API and data-plane manager, so a slow first boot or a long schema migration on upgrade is no longer cut short by the liveness probe.

Fixes​

  • Input guardrails now scan the union of a message's text content and its structured content blocks. The two are independent fields on the wire and the provider bridges forward the structured blocks when present, so a request carrying benign text alongside a payload in the blocks could previously pass every input guardrail while the model still received the payload.
  • A failed request's access log line now names the failure, carrying both a stable error category to filter or alert on and the underlying reason. Previously an upstream error, a name resolution failure, a reaped connection, and an unanswered connection attempt all produced an identical line. See Metrics and Logs.

0.5.0​

Release date: July 24, 2026

This release adds a unified video generation endpoint, budget threshold alerts that notify an external system, and automatic prompt caching for Anthropic models. It also corrects several cases where retries, timeouts, rate limits, and usage accounting did not apply on every request path.

New Features​

Video Generation​

  • A new video generation endpoint accepts prompt-to-video tasks on /v1/videos, polls their status, and returns the finished video, following the same three-stage shape as the OpenAI video API. Providers currently mapped are Alibaba Cloud Model Studio, Zhipu AI CogVideoX, and Volcengine Ark Seedance.
  • Video requests now pass through the same gateway controls as chat traffic: model aliases, caller API key access checks, client IP allowlists, model-level rate limits, and input guardrail scanning of the prompt. Previously video traffic could only reach a provider through passthrough, which applies none of these per-model controls.
  • Task state is not stored by the gateway. The returned video ID carries the routing information, so status and download calls work against any gateway instance.
  • Video submissions are recorded in usage logs with zero tokens. Duration-based cost accounting is not yet applied, so video traffic does not consume budgets in this release.

Guardrails​

  • A new Alibaba Cloud AI Guardrails kind calls the MultiModalGuard service and acts on its suggestion verdicts. When the service returns masked content, the gateway writes the masked text back into the request instead of rejecting it outright.
  • The gateway records aisix_guardrail_latency_seconds, a per-execution latency histogram that separates guardrail overhead from upstream model latency. See Metrics.
  • Responses from Alibaba guardrail services keep their upstream request ID, and the gateway correlates it with its own request ID for cross-system troubleshooting.

Budgets and Alerts​

  • Budget threshold alerts notify an external system when spending crosses a configured percentage of a budget. Notification channels support generic webhooks and Slack.
  • The dashboard budgets page now manages budgets at every scope from one place, and budgets are part of the AISIX Cloud Admin API contract.

Prompt Caching​

  • Models can enable automatic prompt caching so the gateway inserts Anthropic cache breakpoints into qualifying requests. Callers get prompt-cache savings without changing their client code, and the option is configurable per model from the dashboard.

Usage Reporting​

  • When an upstream returns no usage block, the gateway now estimates token counts locally rather than recording zero. Estimated records are flagged as such in usage reporting and on the dashboard logs page, so estimated and reported usage stay distinguishable.
  • The aisix_llm_tokens_by_client_total metric gained a model label, and the metric is now recorded for the responses endpoint.
  • Built-in client type detection covers more coding agents, and operators can add their own user-agent mapping rules for clients the gateway does not recognize.

Deployment​

  • A new aisix export command writes a resources.yaml file from a running etcd store, which produces a starting configuration for standalone mode from an existing deployment.
  • The gateway can run with the admin listener disabled through the admin.enabled setting, for deployments that configure the gateway declaratively and want no write API exposed.
  • The status listener reports per-model runtime health, so upstream reachability can be checked without sending a model request.
  • The gateway reports the hash of the configuration it has applied in its heartbeat, and the control plane exposes gateway nodes and rejected resources in its public spec. This makes it possible to confirm which gateways have taken up a configuration change. See Configuration Propagation.

Improvements​

  • The upstream secret of a provider key can be rotated in place without recreating the key or re-pointing the models that use it.
  • The dashboard provider picker is searchable and lists providers under their proper display names.
  • The dashboard logs page shows the whole upstream error message instead of truncating it, and its upstream filter no longer offers model groups, which are not upstreams.
  • The logs query window follows the organization's usage retention setting instead of a fixed range.
  • The control plane reports why a request was rejected instead of returning a generic failure.
  • The Admin API write path is deprecated in favor of declarative configuration. It still works in this release. Declare dynamic resources in a resources.yaml file instead, or manage them through the AISIX Cloud Admin API.

Fixes​

  • The retries setting is now honored on streaming chat requests, which previously ran a separate code path that did not retry.
  • On Azure models, stream_timeout applies to the gap between chunks as documented, instead of capping the whole response, so long but healthy streams are no longer cut off.
  • Connection failures report the underlying transport cause rather than a generic upstream error, and the connection layer now applies explicit bounds instead of library defaults.
  • Client-supplied cache_control markers survive the OpenAI-to-Anthropic bridge, so callers using an OpenAI client against an Anthropic model keep their prompt-cache savings.
  • Ensemble panel and judge sub-calls estimate their usage when the backend reports none, so ensemble requests no longer under-report tokens.
  • The passthrough tunnel enforces the rate limit of the model named in the request body.
  • Group dispatch applies each routing target's own model rate limit and client IP allowlist, instead of only checking the group entry point.
  • Per-attempt error messages are no longer clipped at 256 characters, so upstream failures stay readable in logs.
  • Minted-token caches for providers that exchange credentials for short-lived tokens are keyed on the whole credential, so a key rotation takes effect immediately instead of after the old token expires.
  • The dashboard pod mounts a writable cache directory, which fixes image requests that could hang indefinitely in Helm deployments.
  • The playground accepts only dashboard sessions. A personal access token can no longer spend provider quota through it.

0.4.0​

Release date: July 16, 2026

This release introduces role-based access control with custom roles and environment-scoped administration, adds status-code-based routing fallback, and expands observability with latency histograms and richer failure diagnostics.

New Features​

Access Control​

  • Organizations can define custom roles that grant fine-grained read and write permissions per resource type, replacing the fixed owner, admin, and member split.
  • Environment-scoped access grants a member administrative control over a single environment without granting organization-wide access.
  • SCIM group-to-role mappings assign roles automatically from identity-provider group membership.
  • The control plane enforces these permissions on every Admin API request.

Routing​

  • fallback_on_statuses opts selected upstream HTTP status codes into retry and failover, so provider-specific transient codes such as 408 or 409 try the remaining targets instead of returning to the caller.

Observability​

  • Bucketed time-to-first-token and end-to-end latency histograms support latency quantiles such as p90 and p99 for service-level objectives. See Metrics.
  • The gateway captures the request body on failed requests to the chat, messages, and responses endpoints, applying structure-preserving truncation to keep large payloads bounded.
  • A new /status/config endpoint reports the loaded observability configuration, and the gateway emits configuration metrics.

Deployment​

  • Standalone mode can load its resources from a resources.yaml file, so the gateway can run without a control plane or etcd.

Improvements​

  • The AISIX Cloud Admin API now exposes cache policies, observability exporters, and rate limits.
  • The dashboard can set a pricing override for models that are not in the pricing catalog, so their usage is still billed.
  • The API-key and rate-limit lists support pagination and search.
  • Guardrail configuration errors now log the provider's error-response body to aid debugging.
  • The Aliyun content-safety guardrail form exposes the output_fail_open option.

Fixes​

  • Monitor-mode output guardrails no longer hold back or close streaming responses.
  • The aisix_deployment_state metric is now derived from the target's serving state.
  • SCIM POST /Users returns the persisted identity instead of echoing the identity provider's payload.

0.3.1​

Release date: July 9, 2026

This maintenance release adds SCIM directory sync and improves observability, accounting, on-premises playground access, and control-plane reliability.

New Features​

  • SCIM 2.0 directory sync can provision and deprovision organization members from any SCIM 2.0 identity provider, including Okta and Microsoft Entra ID, through the new /scim/v2 endpoints.

Improvements​

  • Gateway builds now report their release version in the Server response header, the output of aisix --version, and the data-plane version shown in the dashboard, rather than reporting a static build version.
  • The per-client token metric now includes a cache-inclusive total series. See Metrics for the current metric catalog.
  • Deployment cooldown metrics are emitted when health state changes.
  • The guardrail management API is now included in the AISIX Cloud Admin API contract.

Fixes​

  • Every proxied response now includes an x-aisix-request-id header for correlation with logs and usage events.
  • Anthropic prompt-cache tokens now count toward token rate limits on the native /v1/messages and /v1/responses endpoints.
  • One malformed telemetry event no longer prevents the remaining events in the usage batch from being delivered.
  • The on-premises dashboard playground can now reach private or internal LLM endpoints when AISIX_PLAYGROUND_ALLOW_PRIVATE_IPS is enabled.
  • Sign-in now accepts the deployment's own origin and corresponding loopback origin, and returns clearer messages for rate-limited or untrusted-origin attempts.
  • Member list pagination no longer returns to the first page shortly after the view loads.
  • Control-plane restarts no longer produce harmless duplicate-constraint error logs.

0.3.0​

Release date: July 9, 2026

This release introduces the MCP Gateway and Agent Gateway, expands the proxy API and guardrail catalog, and adds metric-based routing.

New Features​

MCP Gateway​

  • The new aggregating /mcp endpoint fronts multiple upstream MCP servers behind one AISIX caller API key.
  • Upstream MCP servers are first-class resources with registration and full CRUD in the control plane and dashboard, along with enable and disable controls and a configurable upstream timeout.
  • Tool access control limits each caller API key to specific MCP tools.
  • Upstream authentication supports API keys and OAuth 2.0 client credentials.
  • MCP tool calls use the same rate limits, budgets, and input and output guardrails as model traffic. Calls also emit usage events and access logs.

Agent Gateway​

  • The new Agent Gateway fronts organization-scoped Agent-to-Agent (A2A) agents managed through the control plane and dashboard.
  • The allowed_agents field limits each caller API key to specific agents.

APIs​

Guardrails​

Routing​

  • Multi-target models support least-cost, least-latency, and least-busy target selection.
  • Conditional routing can select targets by tags or metadata, and wildcard aliases can route model names such as provider/*.
  • Sticky weighted routing supports A/B testing and canary releases.

Traffic Controls and API Keys​

  • Caller API key lifecycle controls can set an expiration, disable a key, or rotate it in one operation.
  • Cluster rate limiting can use shared Redis storage and adds per-second (rps) and per-hour (rph) request limits to the existing per-minute (rpm) and per-day (rpd) limits.

Observability​

  • Request and response content capture now covers embeddings, rerank, images, and audio.

Dashboard​

  • MCP servers can be managed from the dashboard, with MCP governance available in the rate-limit, budget, and guardrail views.
  • Usage-log retention can be configured by organization.

Improvements​

  • The container image runs as a non-root user and can bind ports 80 and 443 through the CAP_NET_BIND_SERVICE file capability.
  • The dashboard provides a unified, filterable models view and a single model-kind picker when creating a model.
  • Routing targets can be reordered by dragging, and least-cost targets display per-target cost badges.
  • Member and team lists are paginated.

Fixes​

  • Cache and rate-limit keys are scoped by environment so shared Redis storage cannot mix state between environments.
  • Passthrough endpoints now emit usage events for successful and failed requests, with caller API key attribution.