# AI Cache Configuration

The `ai-cache` plugin stores exact and semantically similar LLM responses in Redis, reducing response latency and repeated upstream model usage.<!-- --> <!-- -->Configuration covers plugin parameters.

## Parameters[​](#parameters "Direct link to Parameters")

See plugin [common configurations](https://docs.api7.ai/apisix/reference/plugin-common-configurations.md) for configuration options available to all plugins.

This plugin supports gateway references in string-valued configuration fields. Use the `$env://` prefix for environment variables or the `$secret://` prefix for values from a secret manager, such as HashiCorp Vault’s [KV secrets engine](https://developer.hashicorp.com/vault/docs/secrets/kv). The gateway resolves these references at runtime. ADC `${NAME}` interpolation is different because ADC resolves it while loading the configuration file, before synchronization. For more information, see [environment variables in plugin](https://docs.api7.ai/apisix/reference/environment-variables.md#plugins) and [secrets](https://docs.api7.ai/apisix/key-concepts/secrets.md).

* exact

  object

  ***

  Settings for exact-match caching, where a response is reused only when the normalized request is identical to a previously cached one.

  * ttl

    integer

    default: `3600`

    valid value:

    greater than or equal to 1

    ***

    Time-to-live in seconds for a cached entry.

* cache\_key

  object

  ***

  Settings that control how the cache key is scoped.

  * share\_across\_routes

    boolean

    default: `false`

    ***

    If true, the cache key does not include the route ID, so identical requests on different routes can share cached responses. If false, each route has its own cache scope.

  * include\_consumer

    boolean

    default: `false`

    ***

    If true, the consumer name is included in the cache key, so cached responses are isolated per consumer.

  * include\_vars

    array\[string]

    default: `[]`

    ***

    Names of additional context variables to include in the cache key scope, so requests with different values of these variables do not share cached responses.

* max\_cache\_body\_size

  integer

  default: `1048576`

  valid value:

  greater than or equal to 0

  ***

  Maximum size in bytes of a response body that will be cached. Larger responses are not cached.

* cache\_headers

  boolean

  default: `true`

  ***

  If true, the plugin adds the `X-AI-Cache-Status` response header (and `X-AI-Cache-Age` on a cache hit). Set to false to omit these headers.

* fail\_mode

  string

  default: `skip`

  valid value:

  `skip`, `warn`, or `error`

  ***

  Behavior when the request cannot be cached because no AI instance was selected, for example when the route does not also configure `ai-proxy` or `ai-proxy-multi`. With `skip`, the request is passed through unchecked. With `warn`, the request is passed through and a warning is logged. With `error`, the request is rejected with HTTP 500.

* bypass\_on

  array\[object]

  ***

  A list of request-header matching rules. If a request matches any rule, the cache is bypassed and the response is marked with `X-AI-Cache-Status` set to `BYPASS`.

  * header

    string

    required

    valid value:

    non-empty

    ***

    Name of the request header to match.

  * equals

    string

    required

    ***

    Value that the header must equal for the rule to match.

* policy

  string

  default: `redis`

  valid value:

  `redis`

  ***

  Cache storage backend. Currently only `redis` is supported. The API7 control-plane API requires an explicit `policy` even though the gateway Lua schema has this default.

* layers

  array\[string]

  default: `["exact"]`

  valid value:

  `exact`, or both `exact` and `semantic`

  ***

  Cache layers to enable. `exact` is always required. Add `semantic` to look for similar prompts through embeddings and vector search after an exact miss.

  Semantic caching was introduced in API7 Enterprise 3.9.16 and 3.10.3, and APISIX 3.18.0.

* semantic

  object

  ***

  Settings for semantic caching. Required when `layers` includes `semantic`.

  Introduced in API7 Enterprise 3.9.16 and 3.10.3, and APISIX 3.18.0.

  * similarity\_threshold

    number

    default: `0.95`

    valid value:

    between 0 and 1 inclusive

    ***

    Minimum similarity score required for a semantic cache hit.

  * top\_k

    integer

    default: `1`

    valid value:

    greater than or equal to 1

    ***

    Number of nearest vector matches to retrieve.

  * distance\_metric

    string

    default: `cosine`

    valid value:

    `cosine`

    ***

    Distance metric used by vector search.

  * ttl

    integer

    default: `86400`

    valid value:

    greater than or equal to 1

    ***

    Time-to-live in seconds for semantic cache entries.

  * match

    object

    ***

    Settings that control which conversation content is embedded for semantic matching.

    * message\_countback

      integer

      default: `1`

      valid value:

      greater than or equal to 1

      ***

      Number of recent user-message turns to include in the embedding input.

    * ignore\_system\_prompts

      boolean

      default: `true`

      ***

      If true, system prompts are excluded from the embedding input.

    * ignore\_assistant\_prompts

      boolean

      default: `true`

      ***

      If true, assistant messages are excluded from the embedding input.

    * ignore\_tool\_prompts

      boolean

      default: `true`

      ***

      If true, tool messages are excluded from the embedding input.

  * embedding

    object

    required

    ***

    Embedding provider configuration. Configure exactly one of `openai` or `azure_openai`.

    * openai

      object

      ***

      OpenAI-compatible embedding provider settings. Requires `model` and `api_key`.

      * endpoint

        string

        ***

        OpenAI-compatible embedding API endpoint. When not configured, the public OpenAI embeddings endpoint is used.

      * model

        string

        required

        ***

        Embedding model name, such as `text-embedding-3-small`.

      * api\_key

        string

        required

        ***

        API key used to authenticate with the embedding provider. The value is encrypted with AES before being stored in etcd.

      * dimensions

        integer

        valid value:

        greater than or equal to 1

        ***

        Number of dimensions in the embedding output. Configure this only for models that support overriding the output dimensions.

      * ssl\_verify

        boolean

        default: `true`

        ***

        If true, verify the embedding provider's TLS certificate.

      * timeout

        integer

        default: `5000`

        valid value:

        greater than or equal to 1

        ***

        Timeout in milliseconds for requests to the embedding provider.

    * azure\_openai

      object

      ***

      Azure OpenAI embedding provider settings. Requires `endpoint` and `api_key`.

      * endpoint

        string

        required

        ***

        Azure OpenAI embeddings endpoint.

      * api\_key

        string

        required

        ***

        API key used to authenticate with Azure OpenAI. The value is encrypted with AES before being stored in etcd.

      * dimensions

        integer

        valid value:

        greater than or equal to 1

        ***

        Number of dimensions in the embedding output. Configure this only for models that support overriding the output dimensions.

      * ssl\_verify

        boolean

        default: `true`

        ***

        If true, verify the Azure OpenAI endpoint's TLS certificate.

      * timeout

        integer

        default: `5000`

        valid value:

        greater than or equal to 1

        ***

        Timeout in milliseconds for requests to Azure OpenAI.

  * vector\_search

    object

    required

    ***

    Vector search backend configuration.

    * redis

      object

      required

      ***

      Redis Search vector index settings.

      * index

        string

        default: `ai-cache`

        ***

        Name of the Redis Search index used by semantic caching.

* redis\_host

  string

  required

  valid value:

  at least 2 characters

  ***

  Address of the Redis server. Required when `policy` is `redis`.

* redis\_port

  integer

  default: `6379`

  valid value:

  greater than or equal to 1

  ***

  Port of the Redis server.

* redis\_username

  string

  ***

  Username for Redis authentication when using Redis ACLs.

* redis\_password

  string

  ***

  Password for Redis authentication.

  In API7 Gateway 3.10.2 or later in the 3.10 release series, and 3.9.16 or later in the 3.9 release series, the value is encrypted with AES256 before being saved to the database.

  In APISIX 3.18.0 or later, the value is encrypted with AES before being stored in etcd.

* redis\_database

  integer

  default: `0`

  valid value:

  greater than or equal to 0

  ***

  Database number of the Redis server.

* redis\_timeout

  integer

  default: `1000`

  valid value:

  greater than or equal to 1

  ***

  Timeout in milliseconds for Redis operations.

* redis\_ssl

  boolean

  default: `false`

  ***

  If true, use TLS for the connection to Redis.

* redis\_ssl\_verify

  boolean

  default: `false`

  ***

  If true, verify the TLS certificate of the Redis server.

* redis\_server\_name

  string

  ***

  TLS server name used for SNI and, when certificate verification is enabled, hostname verification. Applies when `redis_ssl` is `true`. Defaults to `redis_host`. If the selected value is an IP address, no server name is sent. To verify a DNS-named certificate when connecting by IP, set this field to the certificate's DNS name and enable `redis_ssl_verify`.

  Introduced in API7 Enterprise 3.9.21 and 3.10.8, and in APISIX 3.19.0. This field applies to the single-node Redis connector, not Redis Cluster or Sentinel.

* redis\_keepalive\_timeout

  integer

  default: `10000`

  valid value:

  greater than or equal to 1000

  ***

  Keepalive timeout in milliseconds for the Redis connection pool. Available in API7 Enterprise from version 3.9.17 on the 3.9 line and from version 3.10.4 on the 3.10 line, and in APISIX from version 3.18.0.

* redis\_keepalive\_pool

  integer

  default: `100`

  valid value:

  greater than or equal to 1

  ***

  Keepalive pool size for Redis connections. Available in API7 Enterprise from version 3.9.17 on the 3.9 line and from version 3.10.4 on the 3.10 line, and in APISIX from version 3.18.0.
