Response Caching

Response caching lets AISIX reuse an earlier non-streaming chat-completions response when a later request has the same cache key. It is gateway-side response caching, not provider prompt caching and not the managed configuration snapshot cache.

In this guide, you will create a cache policy, verify cache miss and hit behavior with the x-aisix-cache response header, and review how cache scope and backend settings work.

How Response Caching Works

AISIX caches exact non-streaming chat-completions responses. Streaming responses and other proxy API families do not use this response-cache path.

Cache policies can apply to all eligible requests, one model alias, or one caller API key. Each policy stores responses in the gateway's in-process memory cache or in Redis when Redis is configured at startup.

Before AISIX can cache a response, the gateway must have the selected backend available and the request must match an enabled cache policy.

Cache Key Matching

AISIX creates the cache key from the normalized chat-completions request that affects the upstream response. Two requests share a cache entry only when the model alias, message roles and normalized content, sampling settings, response length, and extra OpenAI-compatible request options match.

JSON object key order does not affect cache matching, including nested objects. Array order does affect matching, so two requests with the same tool definitions in a different order use different cache entries.

For multi-target models, the cache key uses the alias the caller requested, not the target model that served the miss.

Choose a Cache Backend

Each cache policy chooses where matching responses are stored:

Backend	Behavior
Memory	Default. Uses the in-process cache on the gateway instance that handled the miss.
Redis	Uses the shared Redis cache when cache.redis is configured at startup.

Use memory for single-instance deployments or for policies where node-local cache entries are acceptable. Use Redis when several gateway instances should share cached responses for the same policy.

Redis cache storage can connect to a single Redis endpoint, Redis Cluster, or Redis Sentinel.

Configure a Cache Policy

The examples below show how to configure and verify a model-scoped policy with either memory or Redis cache storage. Use one backend for the example model alias at a time. If another enabled cache policy already matches the same requests, disable or delete it before trying the next example.

Prerequisites

Before starting either example, prepare the following:

• A self-hosted AISIX gateway with the admin and proxy listeners available.
• The admin key from the gateway config.yaml.
• A working model alias and caller API key that can send non-streaming chat-completions requests.

Configure an In-Memory Cache Policy

The example below creates a model-scoped policy that uses the default in-process memory cache.

Set the values used by the example requests:

export AISIX_ADMIN_KEY="YOUR_ADMIN_KEY"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="gpt-4o-mini"
export CACHE_PROMPT="cache-check-$(date +%s)"

Create a cache policy for the example model alias:

curl -sS -X POST "http://127.0.0.1:3001/admin/v1/cache_policies" \
  -H "Authorization: Bearer ${AISIX_ADMIN_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "default-chat-cache",
    "enabled": true,
    "backend": "memory",
    "applies_to": "model:'"${AISIX_MODEL}"'",
    "ttl_seconds": 3600
  }'

You should see a response similar to the following:

{
  "id": "8d86d1cf-4a46-4a71-b4ef-c01e51f77f21",
  "value": {
    "name": "default-chat-cache",
    "enabled": true,
    "backend": "memory",
    "ttl_seconds": 3600,
    "applies_to": "model:gpt-4o-mini"
  },
  "revision": 1
}

Save the returned ID if you want to update or delete the policy later.

Verify In-Memory Cache Behavior

Send a request with the cache prompt:

curl -sSi -X POST "http://127.0.0.1:3000/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"${AISIX_MODEL}"'",
    "messages": [{"role": "user", "content": "'"${CACHE_PROMPT}"'"}]
  }'

The first matching request should include this response header:

x-aisix-cache: miss

The miss value means AISIX called the upstream provider and wrote the response into the cache.

Repeat the request with the same body and model alias:

curl -sSi -X POST "http://127.0.0.1:3000/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"${AISIX_MODEL}"'",
    "messages": [{"role": "user", "content": "'"${CACHE_PROMPT}"'"}]
  }'

The repeated request should include this response header:

x-aisix-cache: hit

The hit value means AISIX served the cached copy without calling the upstream provider.

Change the prompt to confirm that the cache key is tied to the request body:

curl -sSi -X POST "http://127.0.0.1:3000/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"${AISIX_MODEL}"'",
    "messages": [{"role": "user", "content": "'"${CACHE_PROMPT}"' different"}]
  }'

The changed request should return a cache miss.

Configure a Redis Cache Policy

Use Redis when several gateway instances should share cached responses for the same policy.

Before starting the gateway, make sure a Redis instance is already running. Then configure AISIX with the Redis connection details:

config.yaml

cache:
  redis:
    mode: single
    url: redis://127.0.0.1:6379/

Start or restart the gateway after adding the Redis configuration.

Setting only cache.backend: redis does not make Redis available, and AISIX fails startup without a Redis configuration block.

Set the values used by the example requests:

export AISIX_ADMIN_KEY="YOUR_ADMIN_KEY"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="gpt-4o-mini"
export CACHE_PROMPT="redis-cache-check-$(date +%s)"

Create a policy that selects the Redis backend:

curl -sS -X POST "http://127.0.0.1:3001/admin/v1/cache_policies" \
  -H "Authorization: Bearer ${AISIX_ADMIN_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "shared-chat-cache",
    "enabled": true,
    "backend": "redis",
    "applies_to": "model:'"${AISIX_MODEL}"'",
    "ttl_seconds": 3600
  }'

Save the returned ID if you want to update or delete the policy later.

Verify Redis Cache Behavior

Send a request with the cache prompt:

curl -sSi -X POST "http://127.0.0.1:3000/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"${AISIX_MODEL}"'",
    "messages": [{"role": "user", "content": "'"${CACHE_PROMPT}"'"}]
  }'

The first matching request should include this response header:

x-aisix-cache: miss

The miss value means AISIX called the upstream provider and wrote the response into Redis.

Repeat the request with the same body and model alias:

curl -sSi -X POST "http://127.0.0.1:3000/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"${AISIX_MODEL}"'",
    "messages": [{"role": "user", "content": "'"${CACHE_PROMPT}"'"}]
  }'

The repeated request should include this response header:

x-aisix-cache: hit

The hit value means AISIX served the cached copy from Redis without calling the upstream provider.

If a matching policy selects Redis but the gateway process did not start with cache.redis, AISIX disables caching for that policy's matching requests. It does not silently fall back to memory.

Tune Policy Scope

After verifying the basic cache flow, tune the policy scope for the traffic pattern you want to cache.

The applies_to field controls which requests match a cache policy:

Value	Scope
all	Every eligible non-streaming chat-completions request.
model:<alias>	Requests that use the caller-visible model alias.
api_key:<id>	Requests authenticated with the caller API key resource ID.

Start with a narrow policy, such as a model-scoped or caller-key-scoped policy. Use a global policy only when every eligible chat-completions request in the environment should participate in response caching.

Avoid overlapping enabled policies for the same requests. When more than one policy matches, AISIX uses one matching policy to choose the cache backend and TTL, so overlapping policies can make cache behavior harder to reason about.

Avoid unsupported matcher prefixes. The gateway treats unknown forms as global, so a typo can make a policy broader than intended.

Next Steps

You have now configured a response cache policy and verified miss and hit behavior. Next, continue with Guardrails to add request and response checks before and after provider calls.