Skip to main content
API7.ai Logo
Version: Dev

AWS Bedrock

Amazon Bedrock is an AWS service for accessing foundation models from Amazon and other providers through a managed API. AISIX gives applications one OpenAI-compatible interface for Bedrock-hosted Claude, Llama, Mistral, Amazon Nova, Cohere, and other models.

This configuration is for Bedrock-hosted models that should use AISIX authentication, model allowlists, rate limits, and usage accounting. AISIX signs outbound Bedrock calls with AWS SigV4.

Prerequisites​

Before starting, prepare the following:

  • One AISIX setup:
    • For AISIX Cloud, an environment with an attached gateway and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
    • For the open-source AISIX gateway, prepare either a local AISIX installation or the Docker setup from the Open-Source AISIX Gateway Quickstart. Configure the gateway to load a declarative resources file.
  • An AWS access key ID and secret access key with bedrock:InvokeModel permission for the target model. Prepare an STS session token when using temporary credentials.
  • For cross-Region inference, bedrock:InvokeModel permission on the inference-profile ARN and on the foundation-model ARN in the source Region and every destination Region. See the requirements for geographic and global profiles.
  • Access to the target Bedrock model in the selected region, and its model ID or inference profile ID.
  • curl and jq.

Configure with AISIX Cloud​

Export the AISIX Cloud connection details:

# AISIX_CP is the Admin API base URL; include /api and omit a trailing slash
# The local On-Premises quickstart uses http://localhost:8080/api
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Create the provider key with the Bedrock catalog provider and a structured config credential:

PROVIDER_KEY_ID=$(curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "bedrock-prod",
    "provider": "amazon-bedrock",
    "api_key": "",
    "api_base": "https://bedrock-runtime.us-west-2.amazonaws.com",
    "config": {
      "access_key_id": "YOUR_AWS_ACCESS_KEY_ID",
      "secret_access_key": "YOUR_AWS_SECRET_ACCESS_KEY",
      "region": "us-west-2"
    },
    "allowed_environments": ["'"$ENV_ID"'"]
  }' | jq -r '.provider_key.id')

The empty api_key is intentional. Create requests require the field even when config supplies the structured Bedrock credential. When updating config, omit api_key instead of sending another empty string.

The open-source gateway can derive the standard AWS endpoint, but AISIX Cloud currently requires an explicit api_base for Bedrock. Set the region in the hostname and config to the same value. The dashboard exposes the access key ID, secret access key, and region fields. If you use temporary STS credentials through the AISIX Cloud Admin API, also add session_token to config.

Create an alias for Claude Sonnet 5. In us-west-2, in-Region inference is unavailable for this model, so the example uses its US geographic inference profile:

MODEL_ID=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "claude-bedrock",
    "model_name": "us.anthropic.claude-sonnet-5",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }' | jq -r '.model.id')

Create a caller API key that can access the model:

BEDROCK_CALLER_KEY=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "bedrock-caller",
    "allowed_models": ["'"${MODEL_ID}"'"]
  }' | jq -r '.plaintext')

Configure with the Open-Source AISIX Gateway​

Export the AWS credential and choose the caller API key that applications will send to AISIX:

# Replace with your values
export BEDROCK_CREDENTIALS='{"access_key_id":"YOUR_AWS_ACCESS_KEY_ID","secret_access_key":"YOUR_AWS_SECRET_ACCESS_KEY","region":"us-west-2"}'
export BEDROCK_CALLER_KEY="YOUR_CALLER_API_KEY"

For a new gateway, use this complete resources file. For an existing gateway, merge these entries into its current file, preserving its other resources.

resources.yaml
_format_version: "1"

provider_keys:
  - display_name: bedrock-prod
    provider: amazon-bedrock
    adapter: bedrock
    api_key: ${BEDROCK_CREDENTIALS}

models:
  - display_name: claude-bedrock
    provider: amazon-bedrock
    model_name: us.anthropic.claude-sonnet-5
    provider_key: bedrock-prod

api_keys:
  - display_name: bedrock-caller
    key_env: BEDROCK_CALLER_KEY
    allowed_models: ["claude-bedrock"]

❶ provider labels the upstream.

❷ adapter selects Bedrock.

❸ api_key is a JSON string with access_key_id, secret_access_key, and region. Bedrock's endpoint is region-keyed, for example bedrock-runtime.us-west-2.amazonaws.com, so the region is required. Leave api_base unset for standard AWS, or set it to a private Bedrock endpoint if you use one.

❹ model_name is the Bedrock model ID or full inference profile ID. The example's us. profile is available from us-west-2 and keeps inference within the United States and Canada.

❺ provider_key attaches the model to the credential by the provider key display_name. The model's provider uses the same upstream label as the provider key.

To use Meta Llama in the example file, replace the claude-bedrock model entry and update bedrock-caller to allow the new alias. Keep bedrock-prod and the other resources unchanged:

resources.yaml (Meta Llama model access)
models:
  - display_name: llama-bedrock
    provider: amazon-bedrock
    model_name: us.meta.llama3-3-70b-instruct-v1:0
    provider_key: bedrock-prod

api_keys:
  - display_name: bedrock-caller
    key_env: BEDROCK_CALLER_KEY
    allowed_models: ["llama-bedrock"]

For Amazon Nova, use a Bedrock model or inference profile ID, such as the Nova 2 Lite profile us.amazon.nova-2-lite-v1:0 from us-west-2. As in the Claude and Llama examples, the us. prefix selects a US geographic inference profile. The gateway reads the plaintext caller key from BEDROCK_CALLER_KEY and stores only a hash.

The verification below uses claude-bedrock. If you applied the Meta Llama block, send llama-bedrock instead and expect that alias in the response.

Include session_token in the credential JSON when you use temporary STS credentials. Omit it for long-lived static keys. Provider key secrets follow the credential-handling behavior described in Provider Keys.

Validate and Load the Configuration​

If AISIX is installed locally, validate the complete file before loading it:

aisix validate --resources resources.yaml

After validation, start the gateway with the referenced environment variables in its process environment. Reload an existing gateway only if those variables are already available to the process; otherwise, restart it with the updated environment.

If you use Docker, adapt the validation and startup commands in the Open-Source AISIX Gateway Quickstart. Mount this resources.yaml file and pass every environment variable it references with -e in both commands.

Verify the Provider Connection​

Export the AISIX gateway origin:

# AISIX_PROXY has no trailing slash or endpoint path
# The local quickstarts use http://127.0.0.1:3000
export AISIX_PROXY="YOUR_AISIX_GATEWAY_URL"

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer ${BEDROCK_CALLER_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-bedrock",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from Bedrock."
      }
    ]
  }'

The gateway returns an OpenAI-compatible response with the caller-facing alias:

{
  "id": "msg_01example",
  "object": "chat.completion",
  "model": "claude-bedrock",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello from Bedrock!"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 9,
    "completion_tokens": 5,
    "total_tokens": 14
  }
}

Check Bedrock invocation metrics, CloudTrail, or provider-side logs for the test request. If AISIX returns an upstream authentication or authorization error, check the AWS credential, region, IAM permissions, and Bedrock model access.

Structured Output​

A response_format carrying a JSON schema — sent to /v1/chat/completions, or translated from a Responses text.format or an Anthropic Messages output_config.format — is honored on Bedrock. Which shape it takes depends on the model:

  • Claude 4.5 and later constrain their own decoding, so the schema goes on the wire natively. AISIX reads the Claude name out of the Bedrock model ID first, so the bare (anthropic.claude-sonnet-4-5-...), cross-region (us.anthropic.claude-...), and inference-profile ARN spellings are all recognized. A non-streaming Claude request goes out on the Anthropic Messages wire and carries the schema in output_config.format; Converse and ConverseStream carry it in outputConfig.textFormat, where the schema is a JSON string and the schema name is the caller's own where they sent one.
  • Every other model whose Converse implementation supports tool use — Amazon Nova, Meta, Mistral, and Cohere, alongside older Claude families — takes the synthetic tool route: a json_tool_call tool whose input is the answer, with the reply translated back into ordinary content and a streaming request served as a simulated stream. The forcing AISIX adds applies only where Converse honors an explicit tool choice, which is Anthropic Claude and Amazon Nova; elsewhere the tool is offered and the model is trusted to take the only tool on the table.
  • Models whose Converse implementation has no tool use at all — Amazon Titan Text, DeepSeek, and publishers AISIX has not classified — leave the field unhonored and answer as they did before. Attaching a tool configuration to them fails the whole request, including one that carries no tools of its own.

Tools the caller sent are always forwarded, whatever the model: an explicitly requested tool the publisher does not support is theirs to see rejected. {"type": "json_object"} names no schema and puts nothing on the wire on either route.

Bedrock documents the same JSON Schema subset Anthropic does, so the schema is sealed and narrowed the same way; see How AISIX Adapts the Schema.

Prepare for Production​

If applications use streaming, add bedrock:InvokeModelWithResponseStream to the AWS permissions and confirm streaming behavior with the target model.

Send at least one user or assistant message. Bedrock Converse does not accept requests that hold only system or developer messages, so AISIX rejects them before calling the provider. The check covers every request on the Converse path: all non-Claude models, and streaming Claude requests too. Only non-streaming Claude requests use the invoke path instead, together with streaming Claude requests whose response_format takes the synthetic tool route.

Upstream error detail from AWS is redacted in the caller-visible error to avoid leaking AWS identifiers such as ARNs, region, and account ID.

Next Steps​

You have now connected AISIX to AWS Bedrock and verified the model alias. Continue with these guides:

  • Anthropic: configure Claude through the Anthropic API instead.
  • Model Aliases: configure routing, retry behavior, or cost metadata for the alias.
  • Provider Compatibility: review supported proxy endpoints and provider-specific boundaries.