Anthropic Prompt Caching

Automatic Anthropic prompt caching lets AISIX add cache markers to eligible requests for a direct Claude model. Callers can receive provider cache discounts without adding the markers themselves. This is provider-side caching for repeated prompt prefixes, not gateway-side Response Caching, which stores and reuses complete responses.

In this guide, you will enable automatic prompt caching through AISIX Cloud or a declarative resources file. You will then choose a cache lifetime and verify cache creation and cache read tokens for a stable prefix.

How Automatic Prompt Caching Works

Anthropic prompt caching requires a cache_control marker on each eligible request. Callers can place markers themselves, or AISIX can add them automatically for a configured model. Anthropic can serve a marked prefix from cache at a reduced input rate on later requests; without a marker, it processes the prompt at the standard input rate.

When automatic prompt caching is enabled on a model, AISIX adds up to two markers to requests that contain no caller-supplied markers:

If the request has a nonempty system block, AISIX marks its last block to cache the tools and system prefix.
If the final message has a nonempty last content block, AISIX marks that block to cache the conversation prefix. Because the marker is attached to the final turn, the cached prefix advances and new content is cached incrementally as the conversation grows.

AISIX adds at most these two markers, and only to a request that carried none of its own, so the provider's four-breakpoint-per-request limit is not exceeded.

If the caller already sent any cache_control marker, AISIX preserves the caller's markers and adds nothing. The caller's caching strategy takes precedence.

Enable Automatic Prompt Caching

Automatic prompt caching is a model-level setting and is disabled by default. Choose either the AISIX Cloud or open-source configuration path below.

Prerequisites

Before starting, prepare the following:

One of these configuration paths:
- AISIX Cloud with an environment, an attached gateway, and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
- An open-source AISIX gateway that loads a declarative resources.yaml file.
An Anthropic provider key, a direct Claude model, and a caller API key that is allowed to use the model.
A real Anthropic credential for verifying provider cache creation and reads. The configuration shape can be validated without sending a live provider request.
For open-source verification, a configured Datadog, Alibaba Cloud SLS, or object storage exporter. The OTLP exporter does not expose the cache-specific token counters.
curl and jq for the verification requests.

Automatic prompt caching is supported only on direct Anthropic models. Do not set it on routing, ensemble, semantic, or embedding models. AISIX also does not inject markers for Claude models served through Amazon Bedrock or Google Vertex AI.

AISIX Cloud

Export the control-plane connection values and the ID of the direct Claude model:

# AISIX_CP includes /api and has no trailing slash.
# The local On-Premises quickstart uses http://localhost:8080/api.
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_BASE_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"
export MODEL_ID="YOUR_DIRECT_MODEL_ID"

Enable automatic prompt caching on the model:

curl -sS -X PATCH "$AISIX_CP/environments/$ENV_ID/models/$MODEL_ID" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "auto_prompt_caching": {
      "enabled": true,
      "ttl": "5m"
    }
  }' | jq

Send an empty auto_prompt_caching object in another PATCH request to clear the setting.

You can also use the dashboard model form. Open Automatic prompt caching, turn it on, and choose a cache lifetime.

Open-Source AISIX Gateway

Add auto_prompt_caching to the direct Anthropic model in resources.yaml:

resources.yaml
models:
  - display_name: claude-prod
    provider: anthropic
    model_name: claude-sonnet-4-5
    provider_key: anthropic-prod
    auto_prompt_caching:
      enabled: true
      ttl: 5m

The provider_key value must name an Anthropic entry under provider_keys. Validate the complete resources file, then reload the gateway. See Reload a Resources File for the runnable Docker workflow.

Configuration Fields

Both configuration paths use the fields below. enabled is required whenever the object contains configuration fields.

Field	Type	Description
`enabled`	boolean	Whether AISIX injects prompt-cache markers for this model.
`ttl`	string	Cache lifetime for injected markers: `5m` (default when omitted) or `1h`.

Choose a Cache Lifetime

Anthropic supports two cache lifetimes, and AISIX injects the one you select:

Lifetime	`ttl`	Cache creation cost	Cache read cost
5 minutes (default)	`5m`	1.25x the base input rate	0.1x the base input rate
1 hour	`1h`	2x the base input rate	0.1x the base input rate

Both lifetimes read from cache at the same discounted rate. The difference is the cache creation cost and how long the prefix remains cached. A cache read also refreshes the entry, extending its lifetime while the prefix remains active.

Use 5m by default. It has the lower cache creation cost and breaks even after a single cache read. Choose 1h only when the same prefix is reused after gaps longer than five minutes. For example, the higher creation cost can be worthwhile for long agent sessions with idle periods between turns because it avoids repeatedly creating the cache entry.

For current multipliers and supported lifetimes, see the Anthropic prompt caching documentation.

Verify the Cache Discount

Automatic prompt caching pays off when a large, stable prefix is reused. To verify it, send two requests that share the same prefix and inspect the cache creation and read tokens.

Anthropic requires a model-specific minimum prompt length for caching. A shorter prefix is processed without caching and returns no error, so use a prefix that meets the minimum for the selected Claude model.

Export the gateway connection and request values:

# AISIX_PROXY has no trailing slash or endpoint path such as /v1.
# The local quickstarts use http://127.0.0.1:3000.
export AISIX_PROXY="YOUR_AISIX_GATEWAY_URL"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="YOUR_DIRECT_MODEL_ALIAS"

Send the first request with a sufficiently long, stable system prompt. Replace the placeholder with a real prompt, and use the exact same text in the second request:

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"${AISIX_MODEL}"'",
    "messages": [
      {"role": "system", "content": "<a stable system prompt that meets the model minimum>"},
      {"role": "user", "content": "First question"}
    ]
  }' | jq

The first request should record cache creation tokens. Send a second request that keeps the exact same system prompt:

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"${AISIX_MODEL}"'",
    "messages": [
      {"role": "system", "content": "<a stable system prompt that meets the model minimum>"},
      {"role": "user", "content": "Second question"}
    ]
  }' | jq

The second request should record cache read tokens. Inspect the two requests through the applicable path:

AISIX Cloud: Open the environment's Logs page. The first request shows Cache creation tokens, while the second shows Cache read tokens.
Open-source AISIX gateway: Inspect cache_creation_tokens and cache_read_tokens in an object storage or SLS record, or aisix.cache_creation_tokens and aisix.cache_read_tokens in Datadog.

The token_type="total" gateway metric also includes Anthropic cache creation and read tokens, but it does not separate them.

AISIX records cache creation and read tokens separately from ordinary input tokens. In AISIX Cloud, provider-discounted cache reads are reflected in budgets and spend reporting. Both cache token types count toward AISIX token-based rate limits.

Scope and Limitations

Automatic prompt caching applies to normalized requests sent through /v1/chat/completions or /v1/responses to a direct Anthropic model. The native /v1/messages route is raw passthrough for Anthropic models: AISIX does not add automatic markers there, but it preserves markers supplied by the caller.
Anthropic isolates caches by its organization and workspace boundaries, not by AISIX organization or caller API key. Callers that share one Anthropic provider key can therefore share the same provider-side cache. For cache isolation between tenants, use provider keys from separate Anthropic workspaces.
Cache read tokens do not count toward Anthropic's input-token-per-minute rate limit. This provider behavior is separate from AISIX token-based rate limits, which include cache read and cache creation tokens.

Next Steps

You have enabled automatic prompt caching and verified the provider cache discount. Next:

In AISIX Cloud, track the resulting cost in Budgets.
Add gateway-side Response Caching when identical requests can reuse the complete response.

How Automatic Prompt Caching Works​

Enable Automatic Prompt Caching​

Prerequisites​

AISIX Cloud​

Open-Source AISIX Gateway​

Configuration Fields​

Choose a Cache Lifetime​

Verify the Cache Discount​

Scope and Limitations​

Next Steps​