Cohere

Cohere provides Command models and APIs for generation, embeddings, and reranking. AISIX places these capabilities behind gateway-managed credentials, caller access, rate limits, and usage accounting.

Cohere publishes a native API and a compatibility API for OpenAI-shaped requests. This guide uses the compatibility API for chat completions and embeddings, then configures the native API separately for reranking.

Prerequisites

Before starting, prepare the following:

One AISIX setup:
- For AISIX Cloud, an environment with an attached gateway and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
- For the open-source AISIX gateway, prepare either a local AISIX installation or the Docker setup from the Open-Source AISIX Gateway Quickstart. Configure the gateway to load a declarative resources file.
A Cohere API key from the Cohere dashboard.
curl and jq.

Configure with AISIX Cloud

Export the AISIX Cloud connection details:

# AISIX_CP is the Admin API base URL; include /api and omit a trailing slash
# The local On-Premises quickstart uses http://localhost:8080/api
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Create a provider key, model alias, and caller API key for the Cohere-backed chat-completions route.

AISIX connects to Cohere's compatibility API through the openai adapter. Chat completions and embeddings need no request translation. Cohere rerank uses the native API and requires the separate provider key configured in Add a Rerank Model.

Create a Provider Key

Create the provider key that stores the Cohere credential and API root:

# Replace with your value
export COHERE_API_KEY="YOUR_PROVIDER_API_KEY"

PROVIDER_KEY_ID=$(curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "cohere-prod",
    "provider": "cohere",
    "api_key": "'"${COHERE_API_KEY}"'",
    "api_base": "https://api.cohere.ai/compatibility/v1",
    "allowed_environments": ["'"${ENV_ID}"'"]
  }' | jq -r '.provider_key.id')

echo "$PROVIDER_KEY_ID"

❶ provider is cohere. The AISIX Cloud Admin API derives the adapter from the catalog provider; the adapter field is only accepted on BYO provider keys.

❷ api_key stores the Cohere API key. It follows the credential-handling behavior in Provider Keys.

❸ api_base points at Cohere's compatibility API. The path carries a /compatibility segment before the /v1 version segment, because the OpenAI-shaped routes are served on a separate prefix from Cohere's native API. AISIX appends /chat/completions to this base. For the cohere catalog provider the field is optional — the AISIX Cloud Admin API fills in the same value when you omit it — but the examples set it explicitly so the surface each key targets stays visible in the configuration.

The command captures the returned provider key ID in PROVIDER_KEY_ID.

Create a Model

Cohere model IDs combine a family name with a release date: command-a-03-2025 is the March 2025 snapshot of the command-a family, and capability variants add a suffix before the date, as in command-a-reasoning-08-2025 or command-a-vision-07-2025. Because the date is part of the ID, a model alias created from a catalog ID pins one snapshot. Check the Cohere models list for current IDs.

Create the model alias callers will send in requests:

MODEL_ID=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "cohere-command-a-prod",
    "model_name": "command-a-03-2025",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }' | jq -r '.model.id')

echo "$MODEL_ID"

❶ display_name is the alias callers send in model.

❷ model_name is the Cohere model ID, for example command-a-03-2025, command-a-plus-05-2026, or command-a-reasoning-08-2025.

❸ provider_key_id attaches the alias to the Cohere provider key.

Create a Caller API Key

Create the caller API key that can access the model alias. The plaintext key is server-generated and returned once in the response — store it securely:

AISIX_API_KEY=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "cohere-caller",
    "allowed_models": ["'"${MODEL_ID}"'"]
  }' | jq -r '.plaintext')

echo "$AISIX_API_KEY"

The allowed_models value must reference the model ID captured in the previous step. After the write, the configuration projects to attached gateways automatically.

Configure with the Open-Source AISIX Gateway

Export the upstream credential and choose the caller API key that applications will send to the gateway:

export COHERE_API_KEY="YOUR_PROVIDER_API_KEY"
export CALLER_API_KEY="YOUR_CALLER_API_KEY"

Create a complete declarative resources file for this provider:

resources.yaml
_format_version: "1"

provider_keys:
  - display_name: "cohere-prod"
    provider: "cohere"
    adapter: "openai"
    api_key: ${COHERE_API_KEY}
    api_base: "https://api.cohere.ai/compatibility/v1"

models:
  - display_name: "cohere-command-a-prod"
    provider: "cohere"
    model_name: "command-a-03-2025"
    provider_key: "cohere-prod"

api_keys:
  - display_name: "cohere-caller"
    key_env: CALLER_API_KEY
    allowed_models:
      - "cohere-command-a-prod"

If AISIX is installed locally, validate the file before loading it:

aisix validate --resources resources.yaml

After validation, start the gateway with the referenced environment variables in its process environment. Reload an existing gateway only if those variables are already available to the process; otherwise, restart it with the updated environment.

If you use Docker, adapt the validation and startup commands in the Open-Source AISIX Gateway Quickstart. Mount this resources.yaml file and pass every environment variable it references with -e in both commands. After the resources load, prepare the shared verification request below:

export AISIX_API_KEY="$CALLER_API_KEY"

Verify the Provider Connection

Export the AISIX gateway origin:

# The local quickstarts use http://127.0.0.1:3000
export AISIX_PROXY="YOUR_AISIX_GATEWAY_ORIGIN"

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cohere-command-a-prod",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from Cohere."
      }
    ]
  }'

The gateway returns an OpenAI-compatible response that echoes the caller-facing alias cohere-command-a-prod. If the request fails, check the provider key api_key, api_base, and the Cohere model ID in model_name. A 404 from the upstream usually means api_base points at Cohere's native API root instead of the compatibility path.

Control Reasoning Effort

Cohere's reasoning-capable Command models, such as command-a-reasoning-08-2025 and command-a-plus-05-2026, expose thinking differently on each API surface. Because AISIX routes to the compatibility API, callers control it with the OpenAI-shaped reasoning_effort field:

{
  "reasoning_effort": "none"
}

Cohere's compatibility API accepts none and high on this field, which map to disabled and enabled thinking. The native API's token-budget control is not documented on this surface, so a request cannot cap the reasoning token count through the compatibility route. See Cohere's Reasoning documentation for the current field behavior.

AISIX preserves reasoning_content in responses and normalizes reasoning to that canonical field. The Cohere provider key carries no reasoning-field override. If your model streams reasoning under a different delta path, set response.reasoning_field on the provider key.

Add a Rerank Model

The /v1/rerank route accepts a model whose provider value is openai, cohere, or jina, so a Cohere-backed alias is admitted on this route. Two details decide whether the request reaches Cohere:

Cohere rerank is not part of the compatibility API. It is published on the native API host.
The rerank route appends /rerank to the provider key base and inserts a /v1 segment when the base does not already end in /v1. A key that points at https://api.cohere.ai/compatibility/v1 therefore builds https://api.cohere.ai/compatibility/v1/rerank, which is not a rerank endpoint.

In AISIX Cloud, create a second Cohere provider key for the native API root:

RERANK_PROVIDER_KEY_ID=$(curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "cohere-rerank",
    "provider": "cohere",
    "api_key": "'"${COHERE_API_KEY}"'",
    "api_base": "https://api.cohere.com",
    "allowed_environments": ["'"${ENV_ID}"'"]
  }' | jq -r '.provider_key.id')

echo "$RERANK_PROVIDER_KEY_ID"

❶ api_base is the native API root without a version segment. The rerank route adds the version itself and resolves the request to Cohere's v1 rerank endpoint.

caution

The rerank route inserts the /v1 segment whenever the base does not already end in /v1, so Cohere's v2 rerank path is not reachable through /v1/rerank. Setting api_base to https://api.cohere.com/v2 builds https://api.cohere.com/v2/v1/rerank and fails upstream.

Create the rerank model alias and a caller key scoped to it:

RERANK_MODEL_ID=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "cohere-rerank-prod",
    "model_name": "rerank-v3.5",
    "provider_key_id": "'"${RERANK_PROVIDER_KEY_ID}"'"
  }' | jq -r '.model.id')

RERANK_API_KEY=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "cohere-rerank-caller",
    "allowed_models": ["'"${RERANK_MODEL_ID}"'"]
  }' | jq -r '.plaintext')

For the open-source AISIX gateway, add a second provider key for the native API root, add the rerank model, and allow the existing caller key to use both aliases:

resources.yaml
provider_keys:
  - display_name: "cohere-prod"
    provider: "cohere"
    adapter: "openai"
    api_key: ${COHERE_API_KEY}
    api_base: "https://api.cohere.ai/compatibility/v1"
  - display_name: "cohere-rerank"
    provider: "cohere"
    adapter: "openai"
    api_key: ${COHERE_API_KEY}
    api_base: "https://api.cohere.com"

models:
  - display_name: "cohere-command-a-prod"
    provider: "cohere"
    model_name: "command-a-03-2025"
    provider_key: "cohere-prod"
  - display_name: "cohere-rerank-prod"
    provider: "cohere"
    model_name: "rerank-v3.5"
    provider_key: "cohere-rerank"

api_keys:
  - display_name: "cohere-caller"
    key_env: CALLER_API_KEY
    allowed_models:
      - "cohere-command-a-prod"
      - "cohere-rerank-prod"

Validate and reload or restart the declarative resources file as described above, then use the existing caller key for the rerank request:

export RERANK_API_KEY="$CALLER_API_KEY"

Rerank model IDs follow their own naming, separate from the Command families: rerank-v3.5, and the two Rerank 4.0 variants rerank-v4.0-fast and rerank-v4.0-pro. Because this route resolves to Cohere's v1 rerank endpoint, confirm on the rerank model overview that the ID you pick is served there before you point a production alias at it.

Send a rerank request through the proxy:

curl -sS -X POST "$AISIX_PROXY/v1/rerank" \
  -H "Authorization: Bearer ${RERANK_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cohere-rerank-prod",
    "query": "How do I rotate a provider credential?",
    "documents": [
      "Provider keys store the upstream credential.",
      "Caller API keys authorize model access.",
      "Rate limits apply per caller key."
    ],
    "top_n": 2
  }'

AISIX rewrites only the model field to rerank-v3.5 and forwards the body unchanged, so Cohere's top_n parameter reaches the upstream as written. The response keeps Cohere's rerank shape: a results array ordered by relevance_score, plus a meta object. AISIX reads meta.billed_units.input_tokens from that response for usage accounting, so rerank traffic appears in gateway logs and budget totals alongside chat traffic.

Endpoint Support for Cohere

The routes below behave differently depending on which Cohere provider key backs the model alias.

Route	Behavior with a Cohere-backed model
`/v1/chat/completions`	Supported through the compatibility API base.
`/v1/embeddings`	Supported. The `openai` adapter appends `/embeddings` to the compatibility base, which is Cohere's OpenAI-shaped embeddings route. Use a Cohere embedding model ID such as `embed-v4.0` in `model_name`.
`/v1/responses`	Bridged through the chat adapter. AISIX returns a Responses-shaped result rather than forwarding to a native Responses API, and ignores OpenAI-specific Responses fields without a chat equivalent. See Responses API.
`/v1/rerank`	Supported with a second provider key on the native API root. See Add a Rerank Model.
`/v1/images/generations`	Not supported. The route accepts only models whose provider value is `openai`.
`/v1/videos`	Not supported. Cohere is not in the video route's provider allowlist.
`/v1/messages`	Supported for Anthropic-shaped callers through translation. Token counting at `/v1/messages/count_tokens` requires an Anthropic-backed model.
`/passthrough/cohere/*`	Available through a configured passthrough route; grant the route on the caller key's `allowed_routes`. A route binds one fixed `target_url` and provider key, so reaching both the compatibility base and the native root takes two routes under distinct prefixes, one per Cohere provider key.

Next Steps

You have now connected AISIX to Cohere, verified the model alias, and added a rerank route. Continue with these guides:

Model Aliases: configure routing, retry behavior, or cost metadata for these aliases.
Rerank: review the rerank request contract and its provider requirement.
Routing and Failover: fail over between Cohere and a second provider.
Provider Compatibility: review supported proxy endpoints and provider-specific boundaries.

Prerequisites​

Configure with AISIX Cloud​

Create a Provider Key​

Create a Model​

Create a Caller API Key​

Configure with the Open-Source AISIX Gateway​

Verify the Provider Connection​

Control Reasoning Effort​

Add a Rerank Model​

Endpoint Support for Cohere​

Next Steps​