AI Rate Limiting | APISIX & API7 API Gateway Docs

Parameters

See plugin common configurations for configuration options available to all plugins.

note

In API7 Enterprise (from 3.8.17) and in APISIX (from 3.16.0), you should configure one of the following parameter sets, but not both:

rules
Any combination of limit and time_window and/or instances

limit
integer | string
vaild vaule:
greater than 0
The maximum number of tokens allowed to consume within a given time interval.
In API7 Enterprise (from 3.8.17) and in APISIX (from 3.16.0), this parameter also supports the string data type and allows the use of built-in variables prefixed with a dollar sign ($). In earlier APISIX versions, only the integer type is supported.
time_window
integer | string
vaild vaule:
greater than 0
The time interval corresponding to the rate limiting limit in seconds.
In API7 Enterprise (from 3.8.17) and in APISIX (from 3.16.0), this parameter also supports the string data type and allows the use of built-in variables prefixed with a dollar sign ($). In earlier APISIX versions, only the integer type is supported.
show_limit_quota_header
boolean
default: true
If true, includes the rate limiting response headers. Specifically, when using limit/time_window or instances, the headers include the instance name as a suffix:
- X-AI-RateLimit-Limit-{name} shows the total quota.
- X-AI-RateLimit-Remaining-{name} shows the remaining quota.
- X-AI-RateLimit-Reset-{name} shows the number of seconds until the counter resets.
When rules is set, the headers use a prefix instead. See rules.header_prefix for details.
limit_strategy
string
default: total_tokens
vaild vaule:
total_tokens, prompt_tokens, completion_tokens, or expression
Type of token to apply rate limiting. total_tokens, prompt_tokens, and completion_tokens values are returned in each model response, where total_tokens is the sum of prompt_tokens and completion_tokens.
When set to expression, rate limiting cost is calculated using a custom Lua arithmetic expression defined in cost_expr. Available in API7 Enterprise from version 3.9.8 and APISIX from version 3.17.0.
cost_expr
string
vaild vaule:
any non-empty string (must be a valid Lua arithmetic expression)
Lua arithmetic expression for dynamic token cost calculation using raw usage response fields. Available field names depend on the API format and response mode. This field is required and validated when limit_strategy is expression.
Missing variables evaluate to 0. A final cost of 0 is not added to the rate-limit counter.
The expression can use arithmetic operators and the Lua math library. The functions abs, ceil, floor, max, and min are also available without the math. prefix.
Native Anthropic Messages expressions can combine input_tokens, cache_read_input_tokens, cache_creation_input_tokens, and output_tokens. OpenAI Chat Completions expressions can use prompt_tokens and completion_tokens. OpenAI Responses expressions can use input_tokens and output_tokens.
Numeric fields nested inside the usage response can be referenced by joining their keys with __. For example, input_tokens_details__cached_tokens references cached_tokens inside input_tokens_details. Top-level fields keep their original names.
Support for joined names was introduced in API7 Enterprise 3.10.8. Joined names are not available in API7 Enterprise 3.9.x or APISIX 3.19.0, where they evaluate to 0.
Available in API7 Enterprise from version 3.9.8 and APISIX from version 3.17.0.
instances
array[object]
LLM instance rate limiting configurations.
- name
  string
  required
  Name of the LLM service instance.
- limit
  integer | string
  required
  vaild vaule:
  greater than 0
  The maximum number of tokens allowed to consume within a given time interval.
  In API7 Enterprise (from 3.8.17) and in APISIX (from 3.16.0), this parameter also supports the string data type and allows the use of built-in variables prefixed with a dollar sign ($). In earlier APISIX versions, only the integer type is supported.
- time_window
  integer | string
  required
  vaild vaule:
  greater than 0
  The time interval corresponding to the rate limiting limit in seconds.
  In API7 Enterprise (from 3.8.17) and in APISIX (from 3.16.0), this parameter also supports the string data type and allows the use of built-in variables prefixed with a dollar sign ($). In earlier APISIX versions, only the integer type is supported.
rejected_code
integer
default: 503
vaild vaule:
between 200 and 599 inclusive
The HTTP status code returned when a request exceeding the quota is rejected.
rejected_msg
string
vaild vaule:
any non-empty string
The response body returned when a request exceeding the quota is rejected.
policy
string
default: local
vaild vaule:
local, redis, redis-cluster, or redis-sentinel
The policy for rate-limiting counters. API7 Gateway requires this field; configure local when the counters should not use Redis. APISIX defaults to local when the field is omitted.
Redis-backed policies are available in API7 Enterprise from version 3.8.19 and in APISIX from version 3.18.0.
Set to local to store the counter in memory locally.
Set to redis to store the counter on a Redis instance.
Set to redis-cluster to store the counter in a Redis cluster.
Set to redis-sentinel to store the counter on the Redis primary node managed by Redis Sentinel, which ensures high availability by automatically promoting a replica to primary in case of failure. Redis Sentinel provides high availability for Redis when not using Redis Cluster.
redis_host
string
The address of the Redis node. Required when policy is redis.
redis_port
integer
default: 6379
vaild vaule:
greater than or equal to 1
The port of the Redis node when policy is redis.
redis_username
string
The username for Redis data-node authentication when policy is redis. Support for this field with redis-sentinel was introduced in API7 Enterprise 3.10.5 and APISIX 3.18.0. If Redis uses only requirepass, configure redis_password without a username.
redis_password
string
Password for Redis data-node authentication when policy is redis or redis-cluster. Support for this field with redis-sentinel was introduced in API7 Enterprise 3.10.5 and APISIX 3.18.0.
In API7 Gateway 3.10.2 or later in the 3.10 release series, and 3.9.16 or later in the 3.9 release series, the value is encrypted with AES256 before being saved to the database.
In APISIX 3.18.0 or later, the value is encrypted with AES before being stored in etcd.
redis_database
integer
default: 0
vaild vaule:
greater than or equal to 0
The database number in Redis when policy is redis or redis-sentinel.
redis_ssl
boolean
default: false
If true, use SSL to connect to Redis when policy is redis.
redis_ssl_verify
boolean
default: false
If true, verify the server SSL certificate when policy is redis.
redis_server_name
string
TLS server name used for SNI and, when certificate verification is enabled, hostname verification. Applies when policy is redis and redis_ssl is true. Defaults to redis_host. If the selected value is an IP address, no server name is sent. To verify a DNS-named certificate when connecting by IP, set this field to the certificate's DNS name and enable redis_ssl_verify.
Introduced in API7 Enterprise 3.9.21 and 3.10.8, and in APISIX 3.19.0. This field applies to the single-node Redis connector, not Redis Cluster or Sentinel.
redis_timeout
integer
default: 1000
vaild vaule:
greater than or equal to 1
The Redis timeout value in milliseconds when policy is redis or redis-cluster.
redis_cluster_nodes
array[string]
List of Redis cluster nodes with at least one address. Required when policy is redis-cluster.
redis_cluster_name
string
The name of the Redis cluster. Required when policy is redis-cluster.
redis_cluster_ssl
boolean
default: false
If true, use SSL to connect to Redis cluster when policy is redis-cluster.
redis_cluster_ssl_verify
boolean
default: false
If true, verify the server SSL certificate when policy is redis-cluster.
redis_sentinels
array[object]
An array of Redis Sentinel nodes (host and port). Required when policy is redis-sentinel.
- host
  string
  required
  vaild vaule:
  at least 2 characters
  Hostname or IP address of the Redis Sentinel node.
- port
  integer
  required
  vaild vaule:
  between 1 and 65535 inclusive
  Port of the Redis Sentinel node.
redis_master_name
string
The name of the Redis master group that Sentinels are monitoring. Required when policy is redis-sentinel.
redis_role
string
default: master
vaild vaule:
master or slave
The Redis node role to connect to. Configurable when policy is redis-sentinel. Set to master to connect to the current Redis master, and set to slave to connect to a Redis replica.
redis_connect_timeout
integer
default: 1000
vaild vaule:
greater than or equal to 1
Timeout in milliseconds for establishing a connection to a Redis node. Configurable when policy is redis-sentinel.
redis_read_timeout
integer
default: 1000
vaild vaule:
greater than or equal to 1
Timeout in milliseconds for reading data from a Redis node. Configurable when policy is redis-sentinel.
redis_keepalive_timeout
integer
default: `10000` for `redis` or `redis-cluster`; `60000` for `redis-sentinel`
vaild vaule:
redis and redis-cluster: greater than or equal to 1000; redis-sentinel: greater than or equal to 1
Time in milliseconds that an idle Redis connection is kept alive in the connection pool before being closed. Used by all Redis-backed policies in APISIX 3.18.0. In API7 Enterprise, it is used by redis-sentinel; redis and redis-cluster support it from version 3.9.16 on the 3.9 release series and version 3.10.3 on the 3.10 release series.
redis_keepalive_pool
integer
default: 100
vaild vaule:
greater than or equal to 1
Maximum number of idle Redis connections in the keepalive pool. Used when policy is redis or redis-cluster. Available in API7 Enterprise from version 3.9.16 on the 3.9 release series and version 3.10.3 on the 3.10 release series, and in APISIX from version 3.18.0.
sentinel_username
string
Username used to authenticate with the Redis Sentinel instance. Configurable when policy is redis-sentinel.
sentinel_password
string
Password used to authenticate with the Redis Sentinel instance. Configurable when policy is redis-sentinel.
In API7 Gateway 3.10.2 or later in the 3.10 release series, and 3.9.16 or later in the 3.9 release series, the value is encrypted with AES256 before being saved to the database.
In APISIX 3.18.0 or later, the value is encrypted with AES before being stored in etcd.
allow_degradation
boolean
default: false
If true, allow the gateway to continue handling requests without the plugin when the plugin or its dependencies become unavailable.
Available in API7 Enterprise from version 3.8.19 and in APISIX from version 3.18.0.
rules
array[object]
An array of rate-limiting rules that are applied sequentially.
Available in API7 Enterprise from 3.8.17 and in APISIX from 3.16.0.
- count
  integer | string
  required
  vaild vaule:
  greater than 0
  The maximum number of tokens allowed to consume within a given time interval.
  This parameter also supports the string data type and allows the use of built-in variables prefixed with a dollar sign ($).
- time_window
  integer | string
  required
  vaild vaule:
  greater than 0
  The time interval corresponding to the rate limiting count in seconds.
  This parameter also supports the string data type and allows the use of built-in variables prefixed with a dollar sign ($).
- key
  string
  required
  A variable expression that resolves to the key used to count requests for this rule. Prefix each variable with $; both $variable and ${variable} forms are accepted. For example, use ${http_custom_a} or ${http_custom_a}_${http_custom_b}. See built-in variables for available variables.
  A rule applies when at least one variable reference resolves to a request value or its ?? default. A missing variable without a default contributes an empty string but does not count as resolved, so a partial expression can still apply when another variable resolves. An expression with no resolved variables, including a value with no variable references, is skipped.
  If rules is configured and no rule applies to a request, the gateway returns HTTP 500 by default. Setting allow_degradation to true lets the request continue without rate limiting.
  In an ADC configuration, escape a gateway expression with a simple name, such as ${http_user}, as ${http_user}. ADC treats an unescaped ${NAME} containing only word characters as load-time environment interpolation; dotted gateway variable names pass through unchanged.
- header_prefix
  string
  Prefix for all rate limiting response headers. Available in API7 Enterprise from version 3.8.19 and in APISIX from version 3.17.0.
  When configured, the prefix is inserted after X-AI- in the header name. For example, with header_prefix set to test, the headers become X-AI-Test-RateLimit-Limit, X-AI-Test-RateLimit-Remaining, and X-AI-Test-RateLimit-Reset.
  When not configured, the index of the rule in the rules array is used as the prefix. For example, headers for the first rule will be X-AI-1-RateLimit-Limit, X-AI-1-RateLimit-Remaining, and X-AI-1-RateLimit-Reset.

Parameters​

Parameters