Anthropic Prompt Caching

Automatic Anthropic prompt caching lets AISIX add cache markers to eligible requests for a direct Claude model on the Anthropic API, Amazon Bedrock, or Vertex AI. Callers can receive provider cache discounts without adding the markers themselves. This is provider-side caching for repeated prompt prefixes, not gateway-side Response Caching, which stores and reuses complete responses.

In this guide, you will enable automatic prompt caching through the AISIX Cloud Admin API or a declarative resources file. You will then choose a cache lifetime and verify cache creation and cache read tokens for a stable prefix.

How Automatic Prompt Caching Works

Claude prompt caching requires a cache_control marker on each eligible request. Callers can place markers themselves, or AISIX can add them automatically for a configured model. The provider can serve a marked prefix from cache at a reduced input rate on later requests; without a marker, it processes the prompt at the standard input rate.

When automatic prompt caching is enabled on a model, AISIX adds up to two markers to requests that contain no caller-supplied markers:

• If the request has a nonempty system block, AISIX marks its last block to cache the tools and system prefix.
• If the final message has a nonempty last content block, AISIX marks that block to cache the conversation prefix. Because the marker is attached to the final turn, the cached prefix advances and new content is cached incrementally as the conversation grows.

AISIX adds at most these two markers, and only to a request that carried none of its own, so the provider's four-breakpoint-per-request limit is not exceeded.

If the caller already sent any cache_control marker, AISIX preserves the caller's markers and adds nothing. The caller's caching strategy takes precedence.

Enable Automatic Prompt Caching

Automatic prompt caching is a model-level setting and is disabled by default. Choose either the control-plane or AISIX OSS configuration path below.

Prerequisites

Before starting, prepare the following:

• One of these configuration paths:
  • AISIX Enterprise or AISIX Cloud with an environment, an attached gateway, and a write-scoped admin token. For AISIX Enterprise, follow the AISIX Enterprise getting-started guide. To request AISIX Cloud access, contact API7.
  • An open-source AISIX gateway that loads a declarative resources.yaml file.
• A provider key for Anthropic, AWS Bedrock, or Gemini Enterprise Agent Platform, a direct Claude model on it, and a caller API key that is allowed to use the model.
• A real provider credential for verifying provider cache creation and reads. The configuration shape can be validated without sending a live provider request.
• For open-source verification, a configured observability exporter of any kind.
• curl and jq for the verification requests.

Automatic prompt caching is supported only on direct Claude models:

• On the Anthropic API, a model whose provider key uses the anthropic adapter.
• On Amazon Bedrock, a model whose model_name starts with anthropic. after any region or cross-region inference-profile prefix such as us. or global.. For an ARN, AISIX reads the model ID in its last segment.
• On Vertex AI, a model whose model_name starts with claude-.

AISIX Enterprise and AISIX Cloud reject the setting with 400 on any other model, including Amazon Nova, other Bedrock and Vertex AI publishers, and routing, ensemble, semantic, or embedding models. An open-source gateway ignores it on those models. A gateway at 1.6.0 or earlier loads the setting but adds markers only for the Anthropic API.

AISIX Enterprise or AISIX Cloud

Export the control-plane connection values and the ID of the direct Claude model:

# AISIX_CP includes /api and has no trailing slash.
# The local AISIX Enterprise quickstart uses http://localhost:8080/api.
export AISIX_CP="YOUR_AISIX_CONTROL_PLANE_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"
export MODEL_ID="YOUR_DIRECT_MODEL_ID"

Enable automatic prompt caching on the model:

curl -sS -X PATCH "$AISIX_CP/environments/$ENV_ID/models/$MODEL_ID" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "auto_prompt_caching": {
      "enabled": true,
      "ttl": "5m"
    }
  }' | jq

To clear the setting, send auto_prompt_caching as null or as an empty object in another PATCH request. A PATCH that omits the field leaves the setting unchanged.

You can also use the dashboard model form. Open Automatic prompt caching, turn it on, and choose a cache lifetime.

AISIX OSS

Add auto_prompt_caching to the direct Anthropic model in resources.yaml:

resources.yaml (model prompt caching)

models:
  - display_name: claude-prod
    provider: anthropic
    model_name: claude-sonnet-4-5
    provider_key: anthropic-prod
    auto_prompt_caching:
      enabled: true
      ttl: 5m

The provider_key value must name an Anthropic entry under provider_keys. A Claude model on a Bedrock or Vertex AI provider key takes the same auto_prompt_caching block. Validate the complete resources file, then reload the gateway. See Reload a Resources File for the runnable Docker workflow.

Configuration Fields

Both configuration paths use the fields below. enabled is required whenever the object contains configuration fields.

Field	Type	Description
enabled	boolean	Whether AISIX injects prompt-cache markers for this model.
ttl	string	Cache lifetime for injected markers: 5m (default when omitted) or 1h. Some Claude models on Amazon Bedrock accept only the 5-minute cache, and Bedrock rejects a request to such a model that asks for 1h.

Choose a Cache Lifetime

Anthropic supports two cache lifetimes, and AISIX injects the one you select:

Lifetime	ttl	Cache creation cost	Cache read cost
5 minutes (default)	5m	1.25x the base input rate	0.1x the base input rate
1 hour	1h	2x the base input rate	0.1x the base input rate

Both lifetimes read from cache at the same discounted rate. The difference is the cache creation cost and how long the prefix remains cached. A cache read also refreshes the entry, extending its lifetime while the prefix remains active.

Use 5m by default. It has the lower cache creation cost and breaks even after a single cache read. Choose 1h only when the same prefix is reused after gaps longer than five minutes. For example, the higher creation cost can be worthwhile for long agent sessions with idle periods between turns because it avoids repeatedly creating the cache entry.

Some Claude models on Amazon Bedrock support only the 5-minute lifetime, and Bedrock rejects a request to such a model that asks for 1h, whether the caller wrote the ttl or automatic prompt caching injected it. AISIX forwards the ttl as configured rather than changing it.

The usage event records how many cache creation tokens were written at each lifetime when the provider reports the split. AISIX Enterprise and AISIX Cloud bill a one-hour write on a Claude model served by Anthropic, Amazon Bedrock, or Google Vertex AI at twice the input rate unless the organization's price sets a one-hour cache write rate, and every other one-hour write at the model's cache write rate. See Model Pricing and Cache Creation by Lifetime.

For current multipliers and supported lifetimes, see the Anthropic prompt caching documentation.

Verify the Cache Discount

Automatic prompt caching pays off when a large, stable prefix is reused. To verify it, send two requests that share the same prefix and inspect the cache creation and read tokens.

Anthropic requires a model-specific minimum prompt length for caching. A shorter prefix is processed without caching and returns no error, so use a prefix that meets the minimum for the selected Claude model.

Export the gateway connection and request values:

# AISIX_PROXY has no trailing slash or endpoint path such as /v1.
# The local quickstarts use http://127.0.0.1:3000.
export AISIX_PROXY="YOUR_AISIX_GATEWAY_URL"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="YOUR_DIRECT_MODEL_ALIAS"

Send the first request with a sufficiently long, stable system prompt. Replace the placeholder with a real prompt, and use the exact same text in the second request:

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"${AISIX_MODEL}"'",
    "messages": [
      {"role": "system", "content": "<a stable system prompt that meets the model minimum>"},
      {"role": "user", "content": "First question"}
    ]
  }' | jq

The first request should record cache creation tokens. Send a second request that keeps the exact same system prompt:

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"${AISIX_MODEL}"'",
    "messages": [
      {"role": "system", "content": "<a stable system prompt that meets the model minimum>"},
      {"role": "user", "content": "Second question"}
    ]
  }' | jq

The second request should record cache read tokens. Inspect the two requests through the applicable path:

• AISIX Enterprise or AISIX Cloud: Open the environment's Logs page. The first request shows Cache creation tokens, while the second shows Cache read tokens.
• Open-source AISIX gateway: Inspect cache_creation_tokens and cache_read_tokens in an object storage or SLS record, aisix.cache_creation_tokens and aisix.cache_read_tokens in Datadog, or gen_ai.usage.cache_creation.input_tokens and gen_ai.usage.cache_read.input_tokens on an OTLP span. The OTLP gen_ai.usage.input_tokens already includes both, so do not add them to it. A gateway at 1.6.0 or earlier does not export the cache counts over OTLP. When the provider reports how the write splits between lifetimes, the record also carries cache_creation_5m_tokens and cache_creation_1h_tokens, or aisix.cache_creation_5m_tokens and aisix.cache_creation_1h_tokens in Datadog and on an OTLP span.

An OpenAI-compatible caller can also read the same two counts straight off the response, without an exporter: prompt_tokens_details.cached_tokens and prompt_tokens_details.cache_creation_tokens on /v1/chat/completions, or the input_tokens_details equivalents on /v1/responses. These are the client-facing projection of the counters above — the recorded ones keep the provider's own accounting, so the two are reported in different shapes. See Token Usage.

The token_type="total" gateway metric also includes Anthropic cache creation and read tokens, but it does not separate them.

AISIX records cache creation and read tokens separately from ordinary input tokens. In AISIX Enterprise and AISIX Cloud, provider-discounted cache reads are reflected in budgets and spend reporting. Both cache token types count toward AISIX token-based rate limits.

Scope and Limitations

• Automatic prompt caching applies to normalized requests sent through /v1/chat/completions or /v1/responses to a supported Claude model, streaming or not. The /v1/messages route forwards the caller's body for Claude on Anthropic, Bedrock, and Vertex AI: AISIX does not add automatic markers there, but it preserves markers supplied by the caller.
• Toward Amazon Bedrock, a cache_control marker keeps only type and ttl, for caller-supplied and injected markers alike. Toward Vertex AI, its scope field is removed. On Bedrock Converse, which serves streaming chat requests and every non-Claude model, a marker becomes a cachePoint block after the marked content; Amazon Nova receives checkpoints in system and messages without a ttl, and other publishers receive none. Automatic prompt caching does not cover Nova.
• Anthropic isolates caches by its organization and workspace boundaries, not by AISIX organization or caller API key. Callers that share one Anthropic provider key can therefore share the same provider-side cache. For cache isolation between tenants, use provider keys from separate Anthropic workspaces.
• Cache read tokens do not count toward Anthropic's input-token-per-minute rate limit. This provider behavior is separate from AISIX token-based rate limits, which include cache read and cache creation tokens.

Next Steps

You have enabled automatic prompt caching and verified the provider cache discount. Next:

• In AISIX Enterprise and AISIX Cloud, track the resulting cost in Budgets.
• Add gateway-side Response Caching when identical requests can reuse the complete response.