Prompt Caching

Automatic prompt caching lets AISIX add Anthropic prompt-cache markers to requests for a direct Anthropic model, so callers get the provider's prompt-cache discount without changing their requests. This is provider-side prompt caching on Claude models, not the gateway-side Response Caching that reuses stored responses.

In this guide, you will enable automatic prompt caching on an Anthropic model, choose a cache lifetime, and verify that repeated requests with a stable prefix produce cache reads at the discounted rate.

How Automatic Prompt Caching Works

Anthropic prompt caching is opt-in per request: the caller marks a stable prefix with a cache_control marker, and the provider then serves that prefix from cache at a reduced input rate on later requests. Many clients never set these markers, so they pay the full input rate every turn.

When automatic prompt caching is enabled on a model, AISIX adds the markers for callers that send none:

One marker on the last system block, which caches the stable tools and system prefix.
One marker on the last content block of the final message, which caches the whole conversation prefix. Because it rides the final turn, the cached prefix advances as the conversation grows and re-caches incrementally.

AISIX adds at most these two markers, and only to a request that carried none of its own, so the provider's four-breakpoint-per-request limit is never exceeded.

If the caller already sent any cache_control marker, AISIX forwards the request unchanged and adds nothing. The caller's own caching strategy always wins.

Configure Automatic Prompt Caching

Automatic prompt caching is a model-level setting on a direct Anthropic model. It is off unless you enable it.

Add the auto_prompt_caching block when you create or update the model:

{
  "enabled": true,
  "ttl": "5m"
}

Field	Type	Description
`enabled`	boolean	Whether AISIX injects prompt-cache markers for this model.
`ttl`	string	Cache lifetime for injected markers: `5m` (default when omitted) or `1h`. See Choose a cache lifetime.

Prerequisites

Before you start, prepare the following:

A self-hosted AISIX gateway with the admin and proxy listeners available.
The admin key from the gateway config.yaml.
An Anthropic provider key and a caller API key that can send chat-completions requests to a Claude model.

Enable It on a Model

Create a direct Anthropic model with automatic prompt caching enabled:

# Replace with your values
export AISIX_ADMIN_KEY="YOUR_ADMIN_KEY"
export PROVIDER_KEY_ID="YOUR_ANTHROPIC_PROVIDER_KEY_ID"

curl -sS -X POST "http://127.0.0.1:3001/admin/v1/models" \
  -H "Authorization: Bearer ${AISIX_ADMIN_KEY}" \
  -H "Content-Type: application/json" \
  --data-binary @- <<EOF
  {
    "display_name": "claude-prod",
    "provider": "anthropic",
    "model_name": "claude-sonnet-4-5",
    "provider_key_id": "${PROVIDER_KEY_ID}",
    "auto_prompt_caching": {
      "enabled": true,
      "ttl": "5m"
    }
  }
EOF

To enable it on an existing model, send the same auto_prompt_caching block in a PATCH request to the model resource. An empty object clears the setting.

note

Automatic prompt caching applies to direct Anthropic models only. Setting auto_prompt_caching on a routing, ensemble, semantic, or embedding model is rejected.

In AISIX Cloud, enable it from the model form in the dashboard: open the Automatic prompt caching section, turn it on, and choose a cache lifetime.

Choose a Cache Lifetime

Anthropic supports exactly two cache lifetimes, and AISIX injects the one you select:

Lifetime	`ttl`	Cache write cost	Cache read cost
5 minutes (default)	`5m`	1.25x the base input rate	0.1x the base input rate
1 hour	`1h`	2x the base input rate	0.1x the base input rate

Both lifetimes read from cache at the same discounted rate. The difference is the write cost and how long the prefix stays cached. A cache read also refreshes the entry, so an actively used prefix stays warm.

Use 5m in most cases. It has the lower write cost and breaks even after a single cache read. Choose 1h only when the same prefix is reused after gaps longer than five minutes, such as long agent sessions with idle periods between turns, where the higher write cost is offset by avoiding repeated writes.

For current multipliers and the supported lifetimes, see the Anthropic prompt caching documentation.

Verify the Cache Discount

Automatic prompt caching pays off when a large, stable prefix is reused. Send two requests that share the same system prompt to observe a cache write followed by a cache read.

Send the first request with a substantial system prompt:

# Replace with your values
export AISIX_API_KEY="YOUR_CALLER_API_KEY"

curl -sS -X POST "http://127.0.0.1:3000/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-prod",
    "messages": [
      {"role": "system", "content": "<a long, stable system prompt above the model minimum>"},
      {"role": "user", "content": "First question"}
    ]
  }'

Send a second request that keeps the same system prompt:

curl -sS -X POST "http://127.0.0.1:3000/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-prod",
    "messages": [
      {"role": "system", "content": "<the same long, stable system prompt>"},
      {"role": "user", "content": "Second question"}
    ]
  }'

The gateway records cache-write tokens for the first request and cache-read tokens for the second, and prices the cache-read tokens at the discounted rate. Cache-write and cache-read token counts appear in usage records alongside prompt and completion tokens, so the discount is reflected automatically in budgets and spend reporting. To inspect per-request cache tokens, see Metrics and logs.

note

Anthropic caches a prefix only when it meets a per-model minimum length. A prefix below that minimum is processed without caching and returns no error, so short prompts see no change. Automatic prompt caching benefits requests with a large, stable system prompt or tool definitions.

Scope and Limitations

Automatic prompt caching covers direct Anthropic models. It is not yet applied to Claude models served through Amazon Bedrock or Google Vertex AI. Callers on those models can still set their own cache_control markers, which AISIX forwards unchanged.
The provider caches a prefix within one Anthropic account, not per AISIX organization or caller API key. Callers that share one Anthropic provider key share the same provider-side cache. Assign a separate provider key per tenant when you need cache isolation between tenants.
Cache-read tokens do not count toward Anthropic's input-token-per-minute rate limit, so caching a large prefix also increases usable throughput under that limit.

Next Steps

You have enabled automatic prompt caching on an Anthropic model and verified the cache-read discount. Next:

Track the resulting cost in Budgets.
Add gateway-side Response Caching to reuse whole responses for repeated identical requests.

How Automatic Prompt Caching Works​

Configure Automatic Prompt Caching​

Prerequisites​

Enable It on a Model​

Choose a Cache Lifetime​

Verify the Cache Discount​

Scope and Limitations​

Next Steps​