AI Rate Limiting | APISIX & API7 API Gateway Docs

Parameters

See plugin common configurations for configuration options available to all plugins.

note

In API7 Enterprise (from 3.8.17), you should configure one of the following parameter sets, but not both:

rules
Any combination of limit and time_window and/or instances

rules is not available in APISIX yet.

limit
integer | string
vaild vaule:
greater than 0
The maximum number of tokens allowed to consume within a given time interval.
In API7 Enterprise (from 3.8.17), this parameter also supports the string data type and allows the use of built-in variables prefixed with a dollar sign ($). In APISIX, only the integer type is supported.
time_window
integer | string
vaild vaule:
greater than 0
The time interval corresponding to the rate limiting limit in seconds.
In API7 Enterprise (from 3.8.17), this parameter also supports the string data type and allows the use of built-in variables prefixed with a dollar sign ($). In APISIX, only the integer type is supported.
show_limit_quota_header
boolean
default: true
If true, include X-AI-RateLimit-Limit-* to show the total quota, X-AI-RateLimit-Remaining-* to show the remaining quota in the response header, and X-AI-RateLimit-Reset-* to show the number of seconds left for the counter to reset, where * is the instance name.
limit_strategy
string
default: total_tokens
vaild vaule:
total_tokens, prompt_tokens, or completion_tokens
Type of token to apply rate limiting. total_tokens, prompt_tokens, and completion_tokens values are returned in each model response, where total_tokens is the sum of prompt_tokens and completion_tokens.
instances
array[object]
LLM instance rate limiting configurations.
- name
  string
  required
  Name of the LLM service instance.
- limit
  integer | string
  required
  vaild vaule:
  greater than 0
  The maximum number of tokens allowed to consume within a given time interval.
  In API7 Enterprise (from 3.8.17), this parameter also supports the string data type and allows the use of built-in variables prefixed with a dollar sign ($). In APISIX, only the integer type is supported.
- time_window
  integer | string
  required
  vaild vaule:
  greater than 0
  The time interval corresponding to the rate limiting limit in seconds.
  In API7 Enterprise (from 3.8.17), this parameter also supports the string data type and allows the use of built-in variables prefixed with a dollar sign ($). In APISIX, only the integer type is supported.
rejected_code
integer
default: 503
vaild vaule:
between 200 and 599 inclusive
The HTTP status code returned when a request exceeding the quota is rejected.
rejected_msg
string
vaild vaule:
any non-empty string
The response body returned when a request exceeding the quota is rejected.
rules
array[object]
An array of rate-limiting rules to be applied simultaneously.
Available in API7 Enterprise from 3.8.17. Not available in APISIX yet.
- count
  integer | string
  required
  vaild vaule:
  greater than 0
  The maximum number of requests allowed within a given time interval.
  This parameter also supports the string data type and allows the use of built-in variables prefixed with a dollar sign ($).
- time_window
  integer | string
  required
  vaild vaule:
  greater than 0
  The time interval corresponding to the rate limiting count in seconds.
  This parameter also supports the string data type and allows the use of built-in variables prefixed with a dollar sign ($).
- key
  string
  required
  The key to count requests by. If the configured key does not exist, the rule will not be executed.
  If the key_type is var, the key is interpreted as a variable. The variable does not need to be prefixed by a dollar sign ($). See built-in variables for available variables.
  If the key_type is var_combination, the key is interpreted as a combination of variables. All variables should be prefixed by dollar signs ($). For example, to configure the key to use a combination of two request headers custom-a and custom-b, the key should be configured as $http_custom_a $http_custom_b.
  If the key_type is constant, the key is interpreted as a constant value.

Parameters​

Parameters