Resources File Reference

This page is the field-by-field reference for the declarative resources.yaml file that a standalone gateway loads when startup configuration sets resources_file. For the workflow around the file — writing it, validating it, loading it, and verifying the result — see the Open-Source AISIX Gateway Quickstart, the CLI Reference, and Configuration Status.

File Format

A resources file is a single YAML document. Its top level is a mapping with a mandatory _format_version and up to nine resource collections:

_format_version: "1"

provider_keys:
  - display_name: openai-prod
    provider: openai
    api_key: ${OPENAI_API_KEY}

models:
  - display_name: gpt-4o
    provider: openai
    model_name: gpt-4o-2024-11-20
    provider_key: openai-prod

api_keys:
  - display_name: ci-bot
    key_env: CI_BOT_KEY
    allowed_models: ["gpt-4o"]

rate_limit_policies:
  - name: cap-gpt4o
    scope: model
    scope_ref: gpt-4o
    window: minute
    max_requests: 300

_format_version must be exactly the string "1", quoted so YAML parses it as a string. A missing or unrecognized version is a load error, and an unquoted 1 gets a dedicated error asking you to quote it.

Each collection, when present, is a sequence of maps. An absent or null collection loads as empty. Unknown top-level keys are load errors that list the known collections.

Collection	Entry identity	Referenced by
provider_keys	display_name	models[].provider_key
models	display_name	api_keys[].allowed_models, model-to-model references inside routing, ensemble, and semantic, and rate_limit_policies[].scope_ref
api_keys	display_name	rate_limit_policies[].scope_ref
guardrails	name	—
mcp_servers	name (or display_name)	—
a2a_agents	name (or display_name)	—
cache_policies	name	—
observability_exporters	name	—
rate_limit_policies	name	—

Identity and Derived IDs

Every entry's identity field must be a non-empty string that is unique within its collection; a duplicate is a load error naming both entries. For mcp_servers and a2a_agents, either name or display_name supplies the identity; an entry that carries both spellings is a load error — use exactly one.

The file accepts no id field on any entry. Entry IDs are derived deterministically from <kind>/<identity> as a UUIDv5, so the same file always produces the same IDs — across reloads and across processes. References and rate-limit counters keyed by ID therefore survive a SIGHUP reload.

Name References

Inside the file, entries reference each other by name, and the loader resolves each reference to the derived ID:

• models[].provider_key names a provider key's display_name. It is mutually exclusive with an explicit provider_key_id, and an unknown name is a load error that lists the defined provider keys.
• api_keys[].allowed_models, routing.targets[].model, ensemble.panel[].model, ensemble.judge.model, and every semantic model reference name a model's display_name. A reference that does not resolve to a defined model is a load error, except that entries containing * are treated as glob patterns and exempt from the existence check.
• rate_limit_policies[].scope_ref names a model or caller API key when scope is model or api_key; for other scopes the value passes through verbatim.

Environment Interpolation

${VAR} references resolve against the gateway process environment, on string scalars only. The value is substituted into the parsed YAML tree, so an environment value can never inject YAML structure, and mapping keys are never interpolated.

• Partial interpolation is supported: api_base: https://${UPSTREAM_HOST}/v1.
• A variable that is unset or empty is a load error naming the variable — never its value.
• $$ produces a literal $; a bare $VAR without braces passes through untouched; an unterminated ${ or empty ${} is an error.
• Non-string scalars — integer, float, boolean, and null values — are never interpolated.

Loading, Errors, and Reload

The load pipeline is identical at boot, on SIGHUP reload, and under aisix validate: read → YAML parse → ${VAR} interpolation → per-entry conversion and name-reference resolution → schema validation (the same validation as every other configuration source) → cross-reference checks.

Errors aggregate across the whole file, scoped to the offending entry and field — for example models[2] ("gpt-4o"). Loading is all-or-nothing: any error rejects the entire file. At boot that is a fail-fast exit. On a SIGHUP reload the gateway keeps serving the last valid snapshot, logs the aggregated report, and GET /status/config reports the rejected load. There is deliberately no file watcher — reloads are explicit.

Provider Keys

A provider key stores an upstream provider credential and the connection shape that goes with it. Models reference a provider key by its display_name. Unknown fields are rejected at every nesting level.

Field	Type	Required	Description
display_name	string	yes	Entry identity, unique within provider_keys. Models reference the key by this name.
api_key	string	yes	The upstream provider's API key. Supply it as ${VAR}. secret is accepted as an alternative spelling; exactly one of the two must be present. For providers whose credential has more than one field — for example AWS Bedrock or Azure OpenAI with Entra ID — the value is a JSON credential document supplied through one environment variable; see the provider guides for each credential shape.
provider	string	no	Upstream provider identifier, such as openai or deepseek. An open string used for provider-specific dispatch. Default: empty.
adapter	enum	no	Upstream protocol family used when no provider-specific dispatch applies: openai, anthropic, bedrock, vertex, or azure-openai. See Adapter Protocol Families.
api_base	string	no	Override base URL for the upstream provider. Required in practice for private OpenAI-compatible endpoints. Supports partial interpolation, such as https://${UPSTREAM_HOST}/v1.
strip_headers	array of strings	no	Inbound headers removed before passthrough forwarding. Default: authorization, cookie, set-cookie, x-api-key. Entries are trimmed, lowercased, and deduplicated. An explicit [] disables stripping — it does not fall back to the default.
telemetry_tags	object	no	Attribution tags emitted with requests routed through this key.
request	object	no	Request-shape overrides applied before dispatch.
response	object	no	Response-shape overrides applied by provider bridges that support them.

telemetry_tags fields, all optional: kind (catalog or byo), featured (boolean, default false), branded_provider (branded slug for catalog entries), pk_label and byo_label (operator-defined labels such as production or a team name).

request fields, all optional:

Field	Type	Description
request.param_renames	map of string to string	Top-level request body keys named on the left are renamed to the key on the right before dispatch.
request.param_constraints.temperature_min, .temperature_max	number	Clamp bounds for temperature in chat completion bodies. Omitted bounds apply no clamp.
request.default_headers	map of string to string	Headers added to the outbound request when the caller did not set them. Reserved authentication headers are dropped as defense in depth.
request.default_body_fields	map of string to JSON value	Top-level body fields added to the outbound request when the caller did not set them.

response fields, all optional:

Field	Type	Description
response.stream_done_marker	enum	Whether the upstream SSE stream is expected to emit data: [DONE]: required, optional, or none. Omitted accepts either.
response.content_list_to_string	boolean	When true, flattens a messages[*].content array of text blocks into a single string before dispatch. Default false.
response.reasoning_field	string	Path used to lift reasoning content from the provider response, such as delta.reasoning_content.
response.error_envelope	string	Stored error-envelope preference kept for compatibility with control-plane configuration; the proxy does not currently apply it.

_format_version: "1"

provider_keys:
  - display_name: openai-prod
    provider: openai
    api_key: ${OPENAI_API_KEY}
  - display_name: internal-vllm
    provider: internal-vllm
    adapter: openai
    api_base: https://${UPSTREAM_HOST}/v1
    api_key: ${INTERNAL_LLM_KEY}

Models

A model entry is a caller-facing model alias: its display_name is what callers put in the request model field. Direct, embedding, ensemble, and semantic aliases also appear in /v1/models; routing aliases and wildcard patterns are excluded from that listing (see Model Aliases). Every entry carries exactly one dispatch shape — the direct triple (provider + model_name + provider_key_id), a routing block, an ensemble block, or a semantic block. Mixing shapes in one entry is a load error, and unknown fields are rejected at every nesting level.

Fields accepted on every shape:

Field	Type	Required	Description
display_name	string	yes	Entry identity, unique within models. The caller-facing alias.
timeout	integer (ms)	no	End-to-end deadline for non-streaming upstream calls. 0 or absent disables it.
stream_timeout	integer (ms)	no	Maximum gap between upstream streaming chunks. 0 or absent falls back to timeout.
rate_limit	object	no	Per-model request, token, and concurrency limits — see below.
allowed_cidrs	array of strings	no	Client IP allowlist in CIDR notation (IPv4 and IPv6). Empty or absent allows all clients. With a restriction configured, a missing or malformed source IP is denied.
cost	object	no	Per-token cost for budget tracking and least_cost ranking: input_per_1k and output_per_1k, both USD per 1,000 tokens and both required when the block is present.

rate_limit sub-fields, all optional and unlimited when absent: rps, rpm, rph, rpd (requests per fixed 1-second, 60-second, 3,600-second, and 86,400-second window), tpm, tpd (tokens per 60-second and 86,400-second window), and concurrency (maximum in-flight requests; a semaphore, not a window).

Direct Models

The direct shape names one upstream model behind one provider key:

Field	Type	Required	Description
provider	string	yes	Vendor identity such as openai or anthropic. Starts with a lowercase letter or digit; ., _, and - are allowed after the first character; 1–64 characters.
model_name	string	yes	Upstream model identifier sent in provider requests, such as gpt-4o-2024-11-20.
provider_key	string	yes*	Name of a provider_keys entry. Mutually exclusive with provider_key_id — in a resources file, use provider_key.
embedding	object	no	Marks the model as embedding-capable: dimensions (required, vector dimensionality) and normalize (default true, whether the endpoint already returns L2-normalized vectors). The model can then serve /v1/embeddings and back a semantic router.
background_model_check	object	no	Background health checks — see Health Checks. Sub-fields enabled, interval_seconds (min 5), timeout_seconds, prompt, max_tokens, and stale_after_seconds are required when the block is present; ignore_statuses (array of status codes) is optional.
cooldown	object	no	Request-path cooldown after retryable upstream failures — see below.

cooldown sub-fields, all optional: enabled (default true), default_seconds (default 30, the TTL when no usable Retry-After header exists), max_seconds (default 600, the cap when Retry-After is used), honor_retry_after (default true), trigger_statuses (default [401, 408, 429, 500, 502, 503, 504]; setting the field replaces the whole list), trigger_on_timeout (default true), and trigger_on_transport (default true).

embedding, background_model_check, and cooldown are permitted only on the direct shape.

Routing Models

A routing entry selects one target per request and dispatches through that target model's upstream configuration:

Field	Type	Required	Description
routing.targets	array	yes	Ordered target set, minimum one entry. targets[].model names a direct model; targets[].weight (default 1) applies to the weighted strategy; targets[].tags scopes the target to requests carrying matching routing tags, with the tag default reserved as the fallback marker.
routing.strategy	enum	no	round_robin, weighted, failover (default), least_cost, least_latency, or least_busy. Positional strategies pick a starting target and walk forward on failure; metric strategies rank all targets best-first.
routing.retries	integer	no	Retry attempts on the current target before failing over. Default 0.
routing.max_fallbacks	integer	no	Maximum later targets attempted after the initial target fails. Default: all later targets. 0 disables failover.
routing.retry_on_429	boolean	no	Whether an upstream 429 participates in retries and failover. Default false.
routing.fallback_on_statuses	array of integers	no	Additional 4xx status codes treated as retryable, for providers that use them for transient conditions — for example [408, 409]. 5xx codes are already retryable.
routing.when_all_unavailable	enum	no	fail (default): return 503 when health and cooldown state rule out every target. try_anyway: attempt every target in declaration order regardless.
routing.sticky	boolean	no	Deterministic weighted selection: hash the x-aisix-routing-key header (else the caller's API key) into the weight distribution, so the same key lands on the same target. Default false.

See Multi-Target Models for behavior and worked examples.

Ensemble Models

An ensemble entry fans each request out to every panel member concurrently, then has a judge model synthesize one answer:

Field	Type	Required	Description
ensemble.panel	array	yes	Panel members, minimum one. panel[].model names a direct model; optional panel[].temperature and panel[].seed override sampling per member; panel[].weight is accepted and currently ignored.
ensemble.judge	object	yes	judge.model names the direct model that performs synthesis; optional judge.synthesis_prompt overrides the built-in synthesis prompt.
ensemble.min_responses	integer	no	Minimum successful panel responses required before synthesis. Default: the smaller of 2 and the panel size. Clamped to the panel size, floored at 1.
ensemble.timeout_ms	integer (ms)	no	Per-call deadline for each panel member and the judge call. 0 or absent disables it.

See Ensemble Models for behavior and response shape.

Semantic Routers

A semantic entry embeds the latest user message, scores it against each route's example embeddings, and dispatches to the best route above threshold — or to default when none clears it:

Field	Type	Required	Description
semantic.embedding_model	string	yes	Name of an embedding-capable direct model (one carrying an embedding block).
semantic.routes	array	yes	Routes, minimum one. Each route requires name (surfaced in the x-aisix-route response header), target (a direct model name), and examples (at least one example utterance; embedded and cached at apply time); optional description (documentation only) and threshold (overrides match.threshold for this route).
semantic.default	string	yes	Direct model that receives requests matching no route.
semantic.match	object	yes	Shared matching parameters: threshold (required, 0.0–1.0, higher is stricter), distance_metric (cosine, the only supported value), and aggregation (max, the only supported value — a route scores by its best-matching example).
semantic.embedding_timeout_ms	integer (ms)	no	Deadline for the embedding call. 0 or absent disables it.
semantic.on_embedding_failure	enum or object	no	What to do when the embedding call fails: default (route to the default model — the default), fail (reject with 503), or { target: "<alias>" } (route to a specific model).

See Semantic Routing for behavior and tuning.

_format_version: "1"

provider_keys:
  - display_name: openai-main
    provider: openai
    api_key: ${OPENAI_API_KEY}

models:
  - display_name: gpt-4o
    provider: openai
    model_name: gpt-4o
    provider_key: openai-main
    timeout: 30000
    rate_limit:
      rpm: 100
      tpm: 100000
    cost:
      input_per_1k: 0.0025
      output_per_1k: 0.01
  - display_name: gpt-4o-mini
    provider: openai
    model_name: gpt-4o-mini
    provider_key: openai-main
  - display_name: text-embed
    provider: openai
    model_name: text-embedding-3-small
    provider_key: openai-main
    embedding:
      dimensions: 1536
  - display_name: balanced
    routing:
      strategy: weighted
      sticky: true
      targets:
        - model: gpt-4o
          weight: 90
        - model: gpt-4o-mini
          weight: 10
      retries: 1
      retry_on_429: true
  - display_name: council
    ensemble:
      panel:
        - model: gpt-4o
        - model: gpt-4o-mini
          temperature: 0.2
      judge:
        model: gpt-4o
      min_responses: 2
      timeout_ms: 45000
  - display_name: smart-router
    semantic:
      embedding_model: text-embed
      routes:
        - name: coding
          target: gpt-4o
          examples:
            - "Write a Python function that parses CSV"
            - "Debug this stack trace"
          threshold: 0.8
      default: gpt-4o-mini
      match:
        threshold: 0.75
      on_embedding_failure: default

Caller API Keys

A caller API key entry authenticates the applications that call the gateway. The file never holds the plaintext key: supply it through key_env, or pre-hash it into key_hash.

Field	Type	Required	Description
display_name	string	yes	Entry identity, unique within api_keys.
key_env	string	yes*	The name of an environment variable holding the plaintext caller key — a bare name such as MY_APP_KEY, not a ${VAR} reference. At load the value is hashed with SHA-256 and dropped; the plaintext never appears in the loaded document, errors, or logs. Mutually exclusive with key_hash; exactly one of the two is required. Do not start the variable name with AISIX_ — that prefix is reserved for startup-configuration overrides.
key_hash	string	yes*	Lowercase-hex SHA-256 hash of the plaintext key, passed through untouched. Two entries resolving to the same credential are a load error that names the entries, never the hashes.
allowed_models	array of strings	yes	Models this key may use. Entries are single-* globs: "*" grants every model, "team-a/*" grants matching names, and an entry without * must match a model display_name defined in the same file. An empty array denies all models.
rate_limit	object	no	Per-key limits with the same sub-fields as a model's rate_limit: rps, rpm, rph, rpd, tpm, tpd, concurrency.
allowed_tools	array of strings	no	MCP tools this key may call, as <server>__<tool> names, matched as single-* globs: "github__*" grants every tool on one server. Omitted, null, or empty means no MCP tool access.
allowed_agents	array of strings	no	A2A agents this key may reach, by registered name, matched as single-* globs. Omitted, null, or empty means no A2A agent access.
expires_at	string	no	RFC 3339 timestamp after which the key stops authenticating with 401. Omitted means the key never expires. A malformed timestamp rejects the entry at load rather than silently never expiring.
disabled	boolean	no	Administratively disable the key: requests are rejected with 401 until it is enabled again. Default false.
team_id	string	no	Team attribution, matched verbatim by rate_limit_policies with scope: team.
user_id	string	no	Owning member attribution, matched verbatim by member-scope policies.
user_name	string	no	Readable owner name for telemetry labels only.

_format_version: "1"

provider_keys:
  - display_name: openai-main
    provider: openai
    api_key: ${OPENAI_API_KEY}

models:
  - display_name: gpt-4o
    provider: openai
    model_name: gpt-4o
    provider_key: openai-main

api_keys:
  # MY_APP_KEY names an environment variable holding the plaintext
  # caller key; it is hashed at load and never stored.
  - display_name: my-app
    key_env: MY_APP_KEY
    allowed_models: ["gpt-4o"]
    rate_limit:
      rpm: 60
      concurrency: 5

Guardrails

A guardrail entry screens request or response content. The resources file has no attachment collection, so every enabled guardrail applies to every request. The kind field selects the provider, and that kind's configuration fields sit directly on the entry — there is no nested config object. Unknown fields for the selected kind are rejected.

Fields on the guardrail entry, kind-independent unless noted:

Field	Type	Required	Description
name	string	yes	Entry identity, unique within guardrails. Surfaces in metric labels and error reasons.
kind	enum	yes	Provider discriminator — see the kinds table below.
enabled	boolean	no	false stages the rule without running it. Default true.
hook_point	enum	no	Where the rule runs: input (request payload, before the upstream call), output (upstream response), or both (default).
enforcement_mode	string	no	block (default) applies the verdict; monitor records what would have happened without blocking or redacting.
fail_open	boolean	no	For remote-API kinds, behavior when the guardrail provider is unreachable on the input hook: true (default) allows the request and records the bypass; false blocks with 422.
output_fail_open	boolean	no	The same policy for the output hook, on the remote-API kinds only — keyword and pii reject it. Default false — a provider outage holds model output rather than releasing unscanned content.
mandatory	boolean	no	true makes guardrail evaluation errors fatal, overriding fail_open on the failure path. Default false.
timeout_ms	integer	no	Per-call timeout for the remote-API kinds only — keyword, pii, and bedrock reject it (Bedrock uses latency_mode instead). Default 5000.
created_at	string	no	RFC 3339 timestamp. When present, guardrails evaluate oldest first; entries without it sort last.

Kind-specific configuration, with required fields per kind:

kind	Required fields	Notable options
keyword	patterns — array of {kind: literal | regex, value} blocklist patterns, evaluated in-process. An empty list loads but matches nothing.	—
pii	—	detectors (built-in detector list: email, china_mobile, china_id_card, bank_card, us_ssn, ip_address, api_key, jwt, private_key, each with optional per-detector action), custom_patterns (operator regexes with name and regex), default_action (mask default, or block). Masked spans become [<DETECTOR>_REDACTED]; matched values never appear in logs or errors.
presidio	analyzer_url, anonymizer_url — base URLs of customer-run Presidio containers.	entities (Presidio entity types with optional per-entity action), default_action, operator (replace default, mask, hash, redact), language (default en), score_threshold.
openai_moderation	api_key	model (default omni-moderation-latest), category_thresholds (per-category score map; empty defers to the provider's flagged decision), endpoint override. Detection-only — never rewrites content.
lakera	api_key	project_id, endpoint override for regional or self-hosted deployments.
azure_content_safety	endpoint, api_key	Azure Prompt Shield on the Cognitive Services endpoint.
azure_content_safety_text_moderation	endpoint, api_key	categories (default all four of Hate, Sexual, SelfHarm, Violence), severity_threshold (default 2), severity_threshold_by_category, output_type, blocklist_names, halt_on_blocklist_hit, streaming controls below.
aliyun_text_moderation	region, access_key_id, access_key_secret	endpoint override, risk_level_threshold (low, medium, high — default high), streaming controls below.
bedrock	guardrail_id, guardrail_version, region, aws_credentials ({kind: static, access_key_id, secret_access_key}), latency_mode ({kind: serial} or {kind: timed, timeout_ms: 100–5000})	—

For streamed output, the two text-moderation kinds accept stream_processing_mode (window for sliding-window incremental release — the default — or buffer_full for whole-response hold-back), window_size, and window_overlap_size. The buffering kinds (pii, lakera, presidio, and the text-moderation kinds in buffer_full mode) accept max_buffer_bytes (default 262144) and on_buffer_exceeded (fail_closed default, or fail_open).

Provider credentials (api_key, access_key_secret, aws_credentials.secret_access_key) are secrets — supply them as ${VAR}. See the guardrail guides for per-provider behavior.

_format_version: "1"

guardrails:
  # In-process keyword blocklist on the request side only.
  - name: block-secrets
    kind: keyword
    hook_point: input
    patterns:
      - kind: literal
        value: internal-project-codename
      - kind: regex
        value: '\bAKIA[0-9A-Z]{16}\b'

  # Remote moderation; blocks when a listed category reaches its threshold.
  - name: content-moderation
    kind: openai_moderation
    api_key: ${OPENAI_API_KEY}
    category_thresholds:
      violence: 0.5

MCP Servers

An MCP server entry registers an upstream MCP server whose tools the gateway exposes to MCP clients as <name>__<tool>. The gateway holds the upstream credential; it is never forwarded from or exposed to the calling client.

Field	Type	Required	Description
name	string	yes	Entry identity, unique within mcp_servers, and the namespace prefix for the server's tools. Must not contain the reserved separator __. display_name is accepted as an alternative spelling; use one, not both.
url	string	yes	The upstream server's MCP endpoint, such as https://api.example.com/mcp.
transport	enum	no	streamable_http — the only supported transport, and the default.
auth_type	enum	no	How the gateway authenticates upstream: none (default), bearer (secret sent as Authorization: Bearer), api_key (secret sent as x-api-key), or oauth2 (client credentials grant; the token is cached until shortly before expiry).
secret	string	no	The bearer token, API key, or OAuth client secret, per auth_type. Supply it as ${VAR}.
client_id	string	no	OAuth client identifier. Used with auth_type: oauth2.
token_url	string	no	OAuth token endpoint where client credentials are exchanged. Used with auth_type: oauth2.
scopes	array of strings	no	OAuth scopes, joined with spaces into the token request.
timeout_ms	integer	no	Deadline per upstream operation (session setup, tool listing, tool calls). Minimum 1. Default: 30,000 ms.
enabled	boolean	no	false removes the server's tools from listings and calls. Default true.

Caller access to tools is granted per key through api_keys[].allowed_tools. See the MCP Gateway Overview for the caller connection flow.

_format_version: "1"

mcp_servers:
  - name: github
    url: https://api.example.com/mcp
    auth_type: bearer
    secret: ${GITHUB_MCP_TOKEN}

A2A Agents

An A2A agent entry registers an upstream agent that the gateway exposes to callers under /a2a/<name>, serving its agent card with URLs rewritten to the gateway. As with MCP servers, the upstream credential stays in the gateway.

Field	Type	Required	Description
name	string	yes	Entry identity, unique within a2a_agents, and the caller-facing path segment. display_name is accepted as an alternative spelling; use one, not both.
url	string	yes	The upstream agent's base URL.
protocol_version	enum	no	A2A wire format: "1.0" (default) or "0.3". Quote the value so YAML keeps it a string.
auth_type	enum	no	none (default), bearer, or api_key — the same semantics as MCP servers.
secret	string	no	The upstream credential, per auth_type. Supply it as ${VAR}.
timeout_ms	integer	no	Deadline per upstream operation, including agent-card fetches. Minimum 1. Default: 30,000 ms.
enabled	boolean	no	false stops serving the agent. Default true.

Caller access is granted per key through api_keys[].allowed_agents. See the Agent Gateway Overview for the caller connection flow.

_format_version: "1"

a2a_agents:
  - name: invoice-processor
    url: https://agents.example.com/a2a
    auth_type: bearer
    secret: ${INVOICE_AGENT_TOKEN}

Cache Policies

A cache policy caches eligible non-streaming chat-completions responses for the requests it matches; other endpoint families and streaming responses are not cached. The proxy picks the first matching enabled policy per request. See Response Caching for key matching and verification.

Field	Type	Required	Description
name	string	yes	Entry identity, unique within cache_policies. 1–120 characters. Surfaces in metric labels and cache headers.
enabled	boolean	no	false stages the policy without applying it. Default true.
backend	enum	no	memory (default) or redis. The redis backend requires the gateway's static cache.redis configuration; without it, matching requests are not cached.
ttl_seconds	integer	no	Entry time-to-live, 1–604,800 seconds (7 days). Default 3600.
applies_to	string	no	Scope selector: all (default), model:<alias> (requests targeting that model alias, compared before router fan-out), or api_key:<id> (requests authenticated by the caller API key with that resource ID).

applies_to values are matched at request time, not resolved at load: a model alias that matches nothing simply never caches, and an unrecognized prefix is treated as all — a conservative fallback that keeps caching on rather than silently disabling it.

_format_version: "1"

provider_keys:
  - display_name: openai-prod
    provider: openai
    api_key: ${OPENAI_API_KEY}

models:
  - display_name: gpt-4o
    provider: openai
    model_name: gpt-4o-2024-11-20
    provider_key: openai-prod

cache_policies:
  - name: gpt-4o-cache
    backend: memory
    ttl_seconds: 600
    applies_to: "model:gpt-4o"

Observability Exporters

An observability exporter delivers request telemetry to an external system. The kind field selects the backend, and that kind's fields sit directly on the entry. Each kind is closed: unknown fields — including any plaintext credential field — are rejected.

Fields shared by every kind:

Field	Type	Required	Description
name	string	yes	Entry identity, unique within observability_exporters. 1–120 characters.
kind	enum	yes	otlp_http, aliyun_sls, datadog, or object_store.
enabled	boolean	no	Disabled exporters stay configured but receive no telemetry. Default true.

Kind-specific fields:

kind	Required fields	Optional fields
otlp_http	endpoint — full OTLP/HTTP traces URL including the receiver path, such as https://api.honeycomb.io/v1/traces. Must be https:// except for loopback and test hosts.	headers (static headers per export request — put API keys in ${VAR} values), sample_rate (0.0–1.0; absent exports every request), content_mode, content_max_bytes.
aliyun_sls	endpoint (regional *.aliyuncs.com host, no scheme), project, logstore, credential_ref	content_mode, content_max_bytes.
datadog	site (a known Datadog site such as datadoghq.com or datadoghq.eu), service, credential_ref	ddsource (default aisix-ai-gateway), tags (rendered into ddtags), content_mode, content_max_bytes.
object_store	provider (s3, gcs, or azure_blob), bucket, prefix	region (S3 signature scope), endpoint (S3-compatible override for MinIO, OSS, R2; https:// required except loopback and test hosts), compression (gzip default, or none), auth_mode, credential_ref.

content_mode (metadata_only default, or full) controls whether exported records include prompt and response content; content_max_bytes (default 131,072; 1–1,048,576) truncates captured content in full mode. See Observability Exporters for delivery and content-capture behavior.

Remote credentials never live in this resource. credential_ref is an indirection: the gateway resolves it from its own environment variables, keyed by the ref in uppercase with hyphens as underscores — SLS_CRED_<REF>_AK_ID and SLS_CRED_<REF>_AK_SECRET for aliyun_sls, DD_CRED_<REF>_API_KEY for datadog, and OBJSTORE_CRED_<REF>_* for object_store. For object_store, auth_mode: cloud_identity (S3 and GCS only) uses the host's attached cloud identity instead and makes credential_ref optional; the default credential_ref mode requires it.

_format_version: "1"

observability_exporters:
  - name: honeycomb-prod
    kind: otlp_http
    endpoint: https://api.honeycomb.io/v1/traces
    headers:
      x-honeycomb-team: ${HONEYCOMB_API_KEY}
    sample_rate: 0.25

  # The Datadog API key comes from the gateway environment variable
  # DD_CRED_DATADOG_PROD_API_KEY, never from this file.
  - name: datadog-prod
    kind: datadog
    site: datadoghq.com
    credential_ref: datadog-prod
    service: ai-gateway
    tags: ["team:platform", "tier:prod"]

  # S3 NDJSON export using the host's attached cloud identity (no keys).
  - name: s3-events
    kind: object_store
    provider: s3
    bucket: acme-aisix-events
    prefix: ai-gateway
    region: us-east-1
    auth_mode: cloud_identity

Rate Limit Policies

A rate limit policy caps requests or tokens for one subject over a fixed window, independent of the inline rate_limit blocks on models and caller API keys. Unknown fields are rejected.

Field	Type	Required	Description
name	string	yes	Entry identity, unique within rate_limit_policies. Counters key on the derived ID, so they survive reloads.
scope	enum	yes	The subject type: api_key, model, team (one shared bucket for the whole team), member, or team_member (the team's limit, but an independent counter per member).
scope_ref	string	yes	The specific subject. For scope: api_key or scope: model, the display_name of an entry defined in the same file — resolved at load, and an unknown name is a load error. For team, member, and team_member, a team or user ID matched verbatim against the caller API key's team_id or user_id.
window	enum	yes	second, minute, or hour.
max_requests	integer	no*	Requests allowed per window, minimum 1. At least one of max_requests and max_tokens is required.
max_tokens	integer	no*	Tokens allowed per window, minimum 1. Token caps are enforced on minute windows only; on second and hour windows the cap is accepted but not applied. Pair max_tokens with window: minute — see Rate Limits.

_format_version: "1"

provider_keys:
  - display_name: openai-prod
    provider: openai
    api_key: ${OPENAI_API_KEY}

models:
  - display_name: gpt-4o
    provider: openai
    model_name: gpt-4o-2024-11-20
    provider_key: openai-prod

rate_limit_policies:
  - name: cap-gpt4o
    scope: model
    scope_ref: gpt-4o
    window: minute
    max_requests: 300
    max_tokens: 100000