AWS Bedrock

Amazon Bedrock is an AWS service for accessing foundation models from Amazon and other providers through a managed API. AISIX gives applications one OpenAI-compatible interface for Bedrock-hosted Claude, Llama, Mistral, Amazon Nova, Cohere, and other models.

This configuration is for Bedrock-hosted models that should use AISIX authentication, model allowlists, rate limits, and usage accounting. AISIX signs outbound Bedrock calls with AWS SigV4, or sends a Bedrock API key as a bearer token; see Choose a Credential Mode.

Prerequisites

Before starting, prepare the following:

• One AISIX setup:
  • For AISIX Enterprise or AISIX Cloud, an environment with an attached gateway and a write-scoped admin token. For AISIX Enterprise, follow the AISIX Enterprise getting-started guide. To request AISIX Cloud access, contact API7.
  • For the open-source AISIX gateway, prepare either a local AISIX installation or the Docker setup from the AISIX OSS getting-started guide. Configure the gateway to load a declarative resources file.
• AWS credentials with bedrock:InvokeModel permission for the target model: an access key ID and secret access key, plus an STS session token for temporary credentials, or one of the other credential modes.
• For cross-Region inference, bedrock:InvokeModel permission on the inference-profile ARN and on the foundation-model ARN in the source Region and every destination Region. See the requirements for geographic and global profiles.
• Access to the target Bedrock model in the selected region, and its model ID or inference profile ID.
• curl and jq.

Configure with AISIX Enterprise or AISIX Cloud

Export the control-plane connection details:

# AISIX_CP is the Admin API base URL; include /api and omit a trailing slash
# The local AISIX Enterprise quickstart uses http://localhost:8080/api
export AISIX_CP="YOUR_AISIX_CONTROL_PLANE_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Create the provider key with the Bedrock catalog provider and a structured config credential:

PROVIDER_KEY_ID=$(curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "bedrock-prod",
    "provider": "amazon-bedrock",
    "config": {
      "access_key_id": "YOUR_AWS_ACCESS_KEY_ID",
      "secret_access_key": "YOUR_AWS_SECRET_ACCESS_KEY",
      "region": "us-west-2"
    },
    "allowed_environments": ["'"$ENV_ID"'"]
  }' | jq -r '.provider_key.id')

The credential travels in config, so the request omits api_key. Add session_token to config when you use temporary STS credentials. To authenticate another way, add auth_type to config; see Choose a Credential Mode.

api_base is optional. Without it, the control plane stores the runtime endpoint of the credential's region, https://bedrock-runtime.us-west-2.amazonaws.com in this example, or .amazonaws.com.cn in the China regions, and returns it like an explicit value. Set api_base for a private Bedrock endpoint. A region that is not a valid AWS Region name, or one in an isolated partition, needs an explicit api_base. When you later replace the credential with one for another region, an endpoint the control plane derived follows it to the new region.

In the dashboard, the provider key form's Authentication field selects the credential mode and shows the fields that mode takes.

Create an alias for Claude Sonnet 5. In us-west-2, in-Region inference is unavailable for this model, so the example uses its US geographic inference profile:

MODEL_ID=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "claude-bedrock",
    "model_name": "us.anthropic.claude-sonnet-5",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }' | jq -r '.model.id')

Create a caller API key that can access the model:

BEDROCK_CALLER_KEY=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "bedrock-caller",
    "allowed_models": ["'"${MODEL_ID}"'"]
  }' | jq -r '.plaintext')

Configure with AISIX OSS

Export the AWS credential and choose the caller API key that applications will send to AISIX:

# Replace with your values
export BEDROCK_CREDENTIALS='{"access_key_id":"YOUR_AWS_ACCESS_KEY_ID","secret_access_key":"YOUR_AWS_SECRET_ACCESS_KEY","region":"us-west-2"}'
export BEDROCK_CALLER_KEY="YOUR_CALLER_API_KEY"

For a new gateway, use this complete resources file. For an existing gateway, merge these entries into its current file, preserving its other resources.

resources.yaml

_format_version: "1"

provider_keys:
  - display_name: bedrock-prod
    provider: amazon-bedrock
    adapter: bedrock
    api_key: ${BEDROCK_CREDENTIALS}

models:
  - display_name: claude-bedrock
    provider: amazon-bedrock
    model_name: us.anthropic.claude-sonnet-5
    provider_key: bedrock-prod

api_keys:
  - display_name: bedrock-caller
    key_env: BEDROCK_CALLER_KEY
    allowed_models: ["claude-bedrock"]

❶ provider labels the upstream.

❷ adapter selects Bedrock.

❸ api_key is a JSON string with access_key_id, secret_access_key, and region. The other credential modes use the same JSON string with different fields. Bedrock's endpoint is region-keyed, for example bedrock-runtime.us-west-2.amazonaws.com, so the region is required. Leave api_base unset for standard AWS, or set it to a private Bedrock endpoint if you use one.

❹ model_name is the Bedrock model ID or full inference profile ID. The example's us. profile is available from us-west-2 and keeps inference within the United States and Canada.

❺ provider_key attaches the model to the credential by the provider key display_name. The model's provider uses the same upstream label as the provider key.

To use Meta Llama in the example file, replace the claude-bedrock model entry and update bedrock-caller to allow the new alias. Keep bedrock-prod and the other resources unchanged:

resources.yaml (Meta Llama model access)

models:
  - display_name: llama-bedrock
    provider: amazon-bedrock
    model_name: us.meta.llama3-3-70b-instruct-v1:0
    provider_key: bedrock-prod

api_keys:
  - display_name: bedrock-caller
    key_env: BEDROCK_CALLER_KEY
    allowed_models: ["llama-bedrock"]

For Amazon Nova, use a Bedrock model or inference profile ID, such as the Nova 2 Lite profile us.amazon.nova-2-lite-v1:0 from us-west-2. As in the Claude and Llama examples, the us. prefix selects a US geographic inference profile. The gateway reads the plaintext caller key from BEDROCK_CALLER_KEY and stores only a hash.

The verification below uses claude-bedrock. If you applied the Meta Llama block, send llama-bedrock instead and expect that alias in the response.

Include session_token in the credential JSON when you use temporary STS credentials. Omit it for long-lived static keys. Provider key secrets follow the credential-handling behavior described in Provider Keys.

Validate and Load the Configuration

If AISIX is installed locally, validate the complete file before loading it:

aisix validate --resources resources.yaml

After validation, start the gateway with the referenced environment variables in its process environment. Reload an existing gateway only if those variables are already available to the process; otherwise, restart it with the updated environment.

If you use Docker, adapt the validation and startup commands in the AISIX OSS getting-started guide. Mount this resources.yaml file and pass every environment variable it references with -e in both commands.

Choose a Credential Mode

The Bedrock credential is one JSON object: config in the AISIX Cloud Admin API, or the JSON string in a resources file's api_key. Its auth_type selects how AISIX authenticates to Bedrock, and region is required in every mode:

auth_type	Fields	Authentication
access_key (default when omitted)	access_key_id and secret_access_key, plus session_token for temporary credentials	SigV4 with these keys.
api_key	api_key, a Bedrock API key	Sent as Authorization: Bearer, without SigV4.
default_chain	No credential fields	SigV4 with the gateway's own AWS credentials: environment variables, web identity (such as IRSA on EKS), the ECS container credentials endpoint, or the EC2 instance profile.

With access_key or default_chain, role_arn makes AISIX exchange those credentials for the role's through sts:AssumeRole. external_id and role_session_name are optional and require role_arn. Without role_session_name, the session is named aisix-session-<unix seconds>. When the base credentials already belong to that role, AISIX uses them as they are rather than asking the role to assume itself. For example, to assume a role with the gateway's own credentials:

Bedrock credential with AssumeRole

{
  "auth_type": "default_chain",
  "role_arn": "arn:aws:iam::123456789012:role/bedrock-invoke",
  "external_id": "YOUR_EXTERNAL_ID",
  "region": "us-west-2"
}

A field that belongs to another mode is rejected: the Admin API refuses to save it with 400, and in a resources file the gateway fails each request through the key with 401 authentication_error, naming the field but never its value. A missing credential source in the default chain, or an AssumeRole that STS refuses, is also a non-retryable 401. A failing credential service, such as an STS 5xx or an unreachable metadata endpoint, is a transport error that routing can retry and fail over. Error messages name the auth_type and whether a role was involved, never the text STS returns; the cause is written to the gateway log.

Temporary credentials from the default chain or AssumeRole are cached per configuration and refreshed 60 seconds before they expire. STS is reached through the standard AWS SDK resolution, so AWS_ENDPOINT_URL_STS and AWS_ENDPOINT_URL in the gateway's environment apply. Every Bedrock call honors the mode, including streaming, token counting, and embeddings.

The api_key and default_chain modes, and the role fields in any mode, require gateway 1.7.0 or later. AISIX Enterprise and AISIX Cloud refuse to save such a key with 422 DP_INCOMPATIBLE while an environment the key is allowed in has an older gateway registered, because an older gateway either fails every request through the key or signs as the base identity instead of the role.

Verify the Provider Connection

Export the AISIX gateway origin:

# AISIX_PROXY has no trailing slash or endpoint path
# The local quickstarts use http://127.0.0.1:3000
export AISIX_PROXY="YOUR_AISIX_GATEWAY_URL"

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer ${BEDROCK_CALLER_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-bedrock",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from Bedrock."
      }
    ]
  }'

The gateway returns an OpenAI-compatible response with the caller-facing alias:

{
  "id": "msg_01example",
  "object": "chat.completion",
  "model": "claude-bedrock",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello from Bedrock!"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 9,
    "completion_tokens": 5,
    "total_tokens": 14
  }
}

Check Bedrock invocation metrics, CloudTrail, or provider-side logs for the test request. If AISIX returns an upstream authentication or authorization error, check the AWS credential, region, IAM permissions, and Bedrock model access.

Reasoning Effort

A reasoning_effort sent to /v1/chat/completions, or a reasoning.effort sent to /v1/responses, reaches a Claude model on Bedrock the same way it reaches Anthropic directly. Claude Opus 4.5, Claude 4.6, and later models receive output_config.effort. Claude Haiku 4.5, Claude Sonnet 4.5, and earlier families reject that field, so they receive an extended-thinking budget instead, and a request without an output limit gets one large enough for the budget, carried in inferenceConfig.maxTokens on Converse. AISIX reads the Claude name out of the bare, cross-region, and inference-profile ARN spellings of the model ID. See Reasoning Effort on Older Claude Models for the budgets.

Reasoning the model returns reaches the caller on every Bedrock model, Claude or not, on streaming and non-streaming requests alike. /v1/chat/completions returns it as message.reasoning_content, or as delta.reasoning_content on a stream. /v1/responses returns it as a reasoning output item. Separate reasoning blocks are joined by a newline, and Bedrock's reasoning signatures and redacted reasoning are not returned on these two routes. /v1/messages to a Claude model returns Bedrock's own thinking blocks with their signatures, and redacted_thinking blocks, as described in Claude on the Messages API. /v1/messages to any other model returns reasoning as a thinking block with signature: "", streamed with thinking_delta events and no signature_delta.

Claude on the Messages API

/v1/messages and /v1/messages/count_tokens addressed to a Claude model on Bedrock go to Bedrock's own Anthropic Messages operations with the caller's body, instead of being translated through the chat format. A Claude model is an anthropic. model ID, with or without a region or inference-profile prefix, or an ARN that names a Claude model. An application inference profile ARN, which names no model, keeps the translation. /v1/chat/completions and /v1/responses are not affected.

Caller route	Bedrock operation
/v1/messages	InvokeModel, or InvokeModelWithResponseStream for a streaming request
/v1/messages/count_tokens	CountTokens with the invokeModel input, falling back to Bedrock Mantle as described below

The caller therefore gets Bedrock's response as Claude produced it: thinking blocks with their signatures, redacted_thinking blocks, images inside a tool_result, the native stop reasons, the upstream message ID, and full usage on the stream's message_start event, cache counters included. A Bedrock exception raised inside a stream after content has started becomes an Anthropic error event, for example throttlingException as rate_limit_error and serviceUnavailableException as overloaded_error. An exception raised before any content fails the attempt instead, so routing can fail over.

On the way, AISIX applies only these changes to the body:

• model is removed, because the URL names the model, anthropic_version is set to bedrock-2023-05-31, and stream is removed.
• Top-level fields are limited to the ones Bedrock accepts: anthropic_version, anthropic_beta, max_tokens, messages, system, stop_sequences, temperature, top_p, top_k, tools, tool_choice, thinking, metadata, output_config, safeguards, and context_management. A legacy top-level output_format moves into output_config.format, and metadata keeps only user_id.
• anthropic-beta values are written into the body's anthropic_beta and filtered to the ones Bedrock runs, as described in Claude Beta Values. advanced-tool-use-2025-11-20 is sent as Bedrock's tool-search-tool-2025-10-19, and AISIX adds effort-2025-11-24 when output_config.effort is set on a Claude family before 4.6. Values Bedrock refuses, such as prompt-caching-scope-2026-01-05, extended-cache-ttl-2025-04-11, and files-api-2025-04-14, are dropped. The accepted set was measured on Claude Sonnet 4.5, and newer Claude models on Bedrock may accept more.
• A thinking.display other than summarized or omitted is dropped, because Bedrock rejects it, so the model's default display applies. Claude requests on /v1/chat/completions and /v1/responses get the same thinking.display rule.
• context_management.edits keeps only compact_20260112 and clear_tool_uses_20250919.
• A cache_control marker keeps only type and ttl.
• Media given by URL, including inside a tool_result, is downloaded and sent inline; see Media Inputs and URL Fetching.
• The Anthropic attribution line and thinking blocks with an empty signature are removed, as for every upstream that is not Anthropic's own API; see Anthropic Attribution Line.

A structured-output request, output_format or output_config.format, to a Claude family before 4.5, which has no native structured output, stays on these operations too. AISIX moves the schema onto the synthetic json_tool_call tool in the Anthropic body, sealed and narrowed as described in How AISIX Adapts the Schema, and forces that tool unless the request sets its own tool_choice or enables thinking. The model's call to the tool is returned as the JSON text the caller asked for. Everything else in the body reaches the model as written. The upstream call does not stream, so a streaming caller receives the answer as an SSE stream once it is complete.

/v1/messages/count_tokens counts with Bedrock's CountTokens, using the same body adjustments, except that the caller's and the provider key's betas are not sent. Bedrock's CountTokens does not count some Claude models, such as those Bedrock serves only through cross-region inference. When it answers 400 or 404 for a Claude model, AISIX counts on Bedrock Mantle instead, at https://bedrock-mantle.<region>.api.aws/anthropic/v1/messages/count_tokens, authenticated with the provider key's credentials. For a VPC endpoint or GovCloud, set upstream.bedrock.mantle_endpoint in the gateway's startup configuration; one value serves every Bedrock provider key.

Claude Beta Values

Claude on Bedrock reads its betas from the request body's anthropic_beta field, so AISIX never sends an anthropic-beta header to a Claude model on Bedrock, whichever setting supplied the value. Every beta it does send is renamed where Bedrock's name differs and filtered to the betas Bedrock runs, and a value Bedrock refuses is dropped rather than failing the request.

The provider key's anthropic-beta is the value its request headers resolve to: a request.default_headers entry, or the caller's own header when request.forward_client_headers admits it. When both supply it, the default_headers entry wins, as for any other header. Which betas reach anthropic_beta then depends on the route:

Caller route	Betas in anthropic_beta
/v1/messages, streaming or not	The caller's anthropic-beta header, even when forward_client_headers does not name it, merged with the provider key's anthropic-beta and with the betas the body's features need.
/v1/messages/count_tokens	Only the betas the body's features need.
/v1/chat/completions and /v1/responses	The provider key's anthropic-beta, added to any anthropic_beta the body already carries. On Converse that field is additionalModelRequestFields.anthropic_beta, and a single string there counts as one value. No betas are added for the body's features.

These rules apply to the Claude model IDs described in Claude on the Messages API. An application inference profile ARN names no model, so AISIX cannot tell that it serves Claude: the provider key's anthropic-beta goes to it as an unfiltered header, as it does for every other Bedrock model.

GPT and Grok Models

A Grok model (xai.grok-*) or OpenAI GPT 5.6 or later (openai.gpt-5.6 and newer, gpt-oss excluded), addressed through a cross-region inference profile such as global. or us., is served by Bedrock's OpenAI-compatible Chat Completions endpoint, /openai/v1/chat/completions on the runtime host or api_base. The request keeps its OpenAI shape and is authenticated with the provider key's credentials. In-Region model IDs, ARNs, and gpt-oss stay on Converse.

A request still goes to Converse when it needs something only Converse offers:

• Any of guardrailConfig, performanceConfig, serviceTier, requestMetadata, outputConfig, additionalModelRequestFields, additionalModelResponseFieldPaths, or promptVariables.
• A json_schema response format without its json_schema object.
• Function tools on a GPT model while it reasons, which is whenever reasoning_effort is absent or not "none". Grok takes tools on Chat Completions at any effort.

On the Chat Completions endpoint, every parameter goes out as the caller sent it, including parameters AWS refuses for the model, such as n and stop, the penalties and reasoning_effort: "none" on Grok, and temperature, top_p, the penalties, logprobs, and top_logprobs on GPT while it reasons. AWS's error then reaches the caller with its status and message. developer messages keep their role. The only rewrite is the output limit: max_tokens is sent as max_completion_tokens, and an explicit max_completion_tokens wins. Remote image URLs are downloaded and sent inline. A gateway at 1.7.0 or earlier drops the parameters AWS refuses on this endpoint, and sends a request with thinking, top_k, or a json_object response format to Converse instead. See Request Parameter Handling.

Stop Sequences and Output Limits on Converse

A chat request served by Converse or ConverseStream carries stop and max_completion_tokens in typed fields, so AISIX checks them before sending instead of forwarding an invalid value for Bedrock to reject. This covers every streaming request that does not go to the Chat Completions endpoint, every non-streaming request to a model other than Claude, and a non-streaming Claude request that carries requestMetadata or outputConfig.

• stop must be a non-empty string or an array of non-empty strings. A number, an object, or an array holding a non-string or an empty string is refused with 400 and invalid_request_error, and the message names stop. A single empty string, an empty array, and null are treated as no stop.
• max_completion_tokens must be a non-negative integer. Any other value, such as -1, 1.5, or "100", is refused the same way, and the message names max_completion_tokens.

The stop check applies to every Converse model, so whether a request is refused does not depend on which target a Model Group picks. A non-streaming Claude request on InvokeModel translates both values as on the Anthropic route, which refuses an invalid stop or max_completion_tokens with 400 naming the field as well.

A valid stop is sent as inferenceConfig.stopSequences on every Converse model, including models AWS refuses it for, such as OpenAI gpt-oss (openai.gpt-oss-*), whose 400 then reaches the caller. A gateway at 1.7.0 or earlier sends gpt-oss, Grok, and GPT 5.6 or later no stop sequences and ignores a valid stop. A Claude request's own stop_sequences wins over stop.

max_completion_tokens, or max_tokens when it is absent, is sent as inferenceConfig.maxTokens exactly. A gateway at 1.7.0 or earlier raises a limit below 16 to 16 for GPT, Grok, and Kimi K3.

Converse Request Fields

Converse has typed fields for the common sampling settings and a free-form additionalModelRequestFields object for anything model-specific. AISIX places each field of a chat request by what the model's Bedrock documentation defines for it, following Request Parameter Handling:

• temperature and top_p go into inferenceConfig with the caller's exact value.
• OpenAI models, gpt-oss, GPT, and Grok, receive every OpenAI field that Converse has no field of its own for in additionalModelRequestFields, as AWS documents for those models. This includes seed, frequency_penalty, presence_penalty, user, safety_identifier, logit_bias, parallel_tool_calls, metadata, and a response_format other than json_schema. n, logprobs, and top_logprobs are dropped, because Converse answers with one message and no log probabilities.
• Cohere Command R receives seed and the penalties, and AI21 Jamba receives the penalties, in additionalModelRequestFields. Other publishers have no place for these OpenAI fields, and AISIX drops them.
• reasoning_effort goes into additionalModelRequestFields.reasoning_effort as sent for gpt-oss, DeepSeek other than R1, Grok, Zhipu AI GLM (zai.*), and Kimi K3, and as reasoning.effort for GPT. Amazon Nova 2 receives low, medium, or high as reasoningConfig with that maxReasoningEffort, and none as reasoningConfig: {"type": "disabled"}; a tier Nova 2 has no setting for is dropped. Claude receives the translation described in Reasoning Effort. Other publishers have no reasoning setting, and AISIX drops the field.
• top_k reaches Claude in additionalModelRequestFields and Amazon Nova in inferenceConfig.topK. Other publishers do not receive it.
• service_tier auto is sent as serviceTier default, and default, flex, and priority are sent as written. Any other tier is dropped.
• A function tool's strict is sent as toolSpec.strict. The synthetic structured-output tool carries the caller's json_schema.strict the same way. A model that does not take strict answers with Bedrock's own 400; Nova Lite, for example, returns This model doesn't support the strict field.
• additionalModelResponseFieldPaths and promptVariables are sent as the top-level Converse fields of those names, and a Claude request that carries either one is served by Converse rather than InvokeModel.
• Any other field that is not an OpenAI Chat Completions field is a model-specific parameter, and reaches the model in additionalModelRequestFields.
• An assistant turn that carries only replayed reasoning, as reasoning_content or as reasoning content parts such as a Mistral thinking or Databricks reasoning part, is left out of messages.

Converse refuses the following with 400 invalid_request_error naming the field, because it cannot carry them for Bedrock to judge: a tool_choice that is an unknown string or an object without a type or function name, a tool without a function or name, parameters that are not an object, tools that is not an array, a replayed tool call without an id or function name, and a service_tier or reasoning_effort that is not a string. A gateway at 1.7.0 or earlier ignores most of these values instead, and drops the OpenAI fields listed above for OpenAI models, strict, and reasoning_effort on Grok, GLM, and Kimi K3.

Model IDs

model_name can be a bare model ID, a cross-region inference-profile ID, or an ARN. An inference-profile ARN is classified by the model ID in its last segment, so a Claude profile ARN takes the Claude paths above. An application inference profile or provisioned-throughput ARN names no model, so AISIX serves it through Converse. A model ID AISIX does not recognize, or one with invalid characters, fails as a configuration error that is not retried, and a Model Group does not fail over from it.

Structured Output

A response_format carrying a JSON schema — sent to /v1/chat/completions, or translated from a Responses text.format or an Anthropic Messages output_config.format — is honored on Bedrock. Which shape it takes depends on the model:

• Claude 4.5 and later constrain their own decoding, so the schema goes on the wire natively. AISIX reads the Claude name out of the Bedrock model ID first, so the bare (anthropic.claude-sonnet-4-5-...), cross-region (us.anthropic.claude-...), and inference-profile ARN spellings are all recognized. A non-streaming Claude request goes out on the Anthropic Messages wire and carries the schema in output_config.format; Converse and ConverseStream carry it in outputConfig.textFormat, where the schema is a JSON string and the schema name is the caller's own where they sent one.
• GPT 5.6 and later, and Grok, on the Chat Completions endpoint receive the json_schema response format in its OpenAI shape.
• Every other model takes the synthetic tool route: a json_tool_call tool whose input is the answer, with the reply translated back into ordinary content and a streaming request served as a simulated stream. AISIX forces the tool only on model families that take a forced tool choice on Bedrock: Anthropic Claude, Amazon Nova, OpenAI (gpt-oss and GPT), xAI, Qwen 3, DeepSeek V3, Kimi K2.5 and K3, MiniMax M2.x, Nemotron 3, Mistral Large 2407, and the zai. and mistral.devstral-2 models. Elsewhere the tool is only offered, and the model is trusted to take the only tool on the table.

Tools the caller sent are always forwarded, whatever the model: an explicitly requested tool the publisher does not support is theirs to see rejected. {"type": "json_object"} names no schema and puts nothing on the wire on either route.

For Claude, and for a model ID that names no publisher, such as an application inference profile ARN, Bedrock documents the same JSON Schema subset Anthropic does, so the schema is sealed and narrowed the same way; see How AISIX Adapts the Schema. A model of any other publisher on the synthetic tool route receives the caller's schema as sent.

Media and Prompt Caching

Image, document, audio, and video parts in a chat request reach Bedrock as Converse media blocks, or as Claude image and document blocks on the Messages wire. Claude models take images and PDF documents only, and a part the model cannot take is rejected with 400. Bedrock reads no media URL itself, so AISIX downloads an http or https URL and sends the bytes inline. See Media Inputs and URL Fetching for the download limits and the startup settings that control them.

Prompt-cache markers reach Claude and Amazon Nova models on every route. On Converse and ConverseStream, a cache_control marker becomes a cachePoint block after the marked content. Claude takes checkpoints in system, messages, and tools with the marker's ttl, and Nova takes them in system and messages without a ttl. Other publishers, including openai.*, receive no checkpoint. Toward Bedrock, a marker keeps only type and ttl. Some Claude models on Bedrock accept only the 5-minute cache and reject a request that asks for 1h. To add markers automatically, see Anthropic Prompt Caching.

Prepare for Production

If applications use streaming, add bedrock:InvokeModelWithResponseStream to the AWS permissions and confirm streaming behavior with the target model.

Send at least one user or assistant message. Bedrock does not accept requests that hold only system or developer messages. AISIX sends such a request to Bedrock on every route, and Bedrock's own error reaches the caller. A gateway at 1.7.0 or earlier rejects it before calling Bedrock with 400 invalid_request_error.

An error Bedrock returns reaches the caller with Bedrock's own message text. That text can include AWS identifiers such as account IDs and ARNs. An error with no message is named by its status. ModelTimeoutException is reported as a timeout, ModelErrorException as an upstream server error that routing can retry and fail over, and a context-window overflow carries code: context_length_exceeded.

Bedrock runtime calls, the Bedrock guardrail, and the AWS STS calls that assume a role go through the gateway's HTTPS_PROXY, HTTP_PROXY, or ALL_PROXY unless NO_PROXY covers the host, like other upstream calls. To keep Bedrock traffic direct, such as to a VPC endpoint, add its host to NO_PROXY. Requests to the AWS credential endpoints, such as instance metadata, ECS task credentials, EKS Pod Identity, and a loopback credential agent, always stay direct. A gateway at 1.6.x or earlier connects to Bedrock directly.

AISIX records Bedrock's x-amzn-requestid response header as the provider request ID whenever the response body carries no ID of its own. On Converse, it is also the response id the caller sees. See Trace Provider Attempts.

Next Steps

You have now connected AISIX to AWS Bedrock and verified the model alias. Continue with these guides:

• Anthropic: configure Claude through the Anthropic API instead.
• Model Aliases: configure routing, retry behavior, or cost metadata for the alias.
• Provider Compatibility: review supported proxy endpoints and provider-specific boundaries.