Cohere

Connect AISIX AI Gateway to Cohere so applications can call Cohere Command models through the gateway's OpenAI-compatible API. AISIX keeps the Cohere credential on the gateway side. It authorizes model access with caller API keys and allowlists, and applies the same rate-limit and usage-accounting controls as other model aliases.

Cohere publishes two API surfaces: a native API that uses Cohere's own request shape, and a compatibility API that accepts OpenAI-shaped requests. AISIX uses the openai adapter against the compatibility API, so chat completions and embeddings need no request translation. Cohere rerank stays on the native API and needs a second provider key — see Add a Rerank Model.

Prerequisites

The following examples configure the upstream through the AISIX Cloud Admin API. Before configuring the upstream, prepare the following:

A running AISIX managed control plane with an environment and an attached gateway. Follow the AISIX On-Premises Quickstart to set up the stack, create an admin token with write scope, and capture the environment ID.
A Cohere API key from the Cohere dashboard.

Export the connection details used by every request below:

# Replace with your values
export AISIX_CP="http://localhost:8080/api"
export AISIX_TOKEN="aisix_pat_YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Configure the Cohere Upstream

Create a provider key, model alias, and caller API key for the Cohere-backed chat-completions route.

Create a Provider Key

Create the provider key that stores the Cohere credential and API root:

# Replace with your value
export COHERE_API_KEY="YOUR_PROVIDER_API_KEY"

PROVIDER_KEY_ID=$(curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "cohere-prod",
    "provider": "cohere",
    "api_key": "'"${COHERE_API_KEY}"'",
    "api_base": "https://api.cohere.ai/compatibility/v1",
    "allowed_environments": ["'"${ENV_ID}"'"]
  }' | jq -r '.provider_key.id')

echo "$PROVIDER_KEY_ID"

❶ provider is cohere. The Cloud Admin API derives the adapter from the catalog provider; the adapter field is only accepted on BYO provider keys.

❷ api_key stores the Cohere API key. It follows the credential-handling behavior in Provider Credentials.

❸ api_base points at Cohere's compatibility API. The path carries a /compatibility segment before the /v1 version segment, because the OpenAI-shaped routes are served on a separate prefix from Cohere's native API. AISIX appends /chat/completions to this base. For the cohere catalog provider the field is optional — the Cloud Admin API fills in the same value when you omit it — but the examples set it explicitly so the surface each key targets stays visible in the configuration.

The command captures the returned provider key ID in PROVIDER_KEY_ID.

Create a Model

Cohere model IDs combine a family name with a release date: command-a-03-2025 is the March 2025 snapshot of the command-a family, and capability variants add a suffix before the date, as in command-a-reasoning-08-2025 or command-a-vision-07-2025. Because the date is part of the ID, a model alias created from a catalog ID pins one snapshot. Check the Cohere models list for current IDs.

Create the model alias callers will send in requests:

MODEL_ID=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "cohere-command-a-prod",
    "model_name": "command-a-03-2025",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }' | jq -r '.model.id')

echo "$MODEL_ID"

❶ display_name is the alias callers send in model.

❷ model_name is the Cohere model ID, for example command-a-03-2025, command-a-plus-05-2026, or command-a-reasoning-08-2025.

❸ provider_key_id attaches the alias to the Cohere provider key.

Create a Caller API Key

Create the caller API key that can access the model alias. The plaintext key is server-generated and returned once in the response — store it securely:

AISIX_API_KEY=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "cohere-caller",
    "allowed_models": ["'"${MODEL_ID}"'"]
  }' | jq -r '.plaintext')

echo "$AISIX_API_KEY"

The allowed_models value must reference the model ID captured in the previous step. After the write, the configuration projects to attached gateways automatically.

Verify the Provider Connection

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "http://127.0.0.1:3000/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cohere-command-a-prod",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from Cohere."
      }
    ]
  }'

The gateway returns an OpenAI-compatible response that echoes the caller-facing alias cohere-command-a-prod. If the request fails, check the provider key api_key, api_base, and the Cohere model ID in model_name. A 404 from the upstream usually means api_base points at Cohere's native API root instead of the compatibility path.

Control Reasoning Effort

Cohere's reasoning-capable Command models, such as command-a-reasoning-08-2025 and command-a-plus-05-2026, expose thinking differently on each API surface. Because AISIX routes to the compatibility API, callers control it with the OpenAI-shaped reasoning_effort field:

{
  "reasoning_effort": "none"
}

Cohere's compatibility API accepts none and high on this field, which map to disabled and enabled thinking. The native API's token-budget control is not documented on this surface, so a request cannot cap the reasoning token count through the compatibility route. See Cohere's Reasoning documentation for the current field behavior.

AISIX preserves reasoning_content in responses and normalizes reasoning to that canonical field. The Cohere provider key carries no reasoning-field override. If your model streams reasoning under a different delta path, set response.reasoning_field on the provider key.

Add a Rerank Model

The /v1/rerank route accepts a model whose provider value is openai, cohere, or jina, so a Cohere-backed alias is admitted on this route. Two details decide whether the request reaches Cohere:

Cohere rerank is not part of the compatibility API. It is published on the native API host.
The rerank route appends /rerank to the provider key base and inserts a /v1 segment when the base does not already end in /v1. A key that points at https://api.cohere.ai/compatibility/v1 therefore builds https://api.cohere.ai/compatibility/v1/rerank, which is not a rerank endpoint.

Create a second Cohere provider key for the native API root:

RERANK_PROVIDER_KEY_ID=$(curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "cohere-rerank",
    "provider": "cohere",
    "api_key": "'"${COHERE_API_KEY}"'",
    "api_base": "https://api.cohere.com",
    "allowed_environments": ["'"${ENV_ID}"'"]
  }' | jq -r '.provider_key.id')

echo "$RERANK_PROVIDER_KEY_ID"

❶ api_base is the native API root without a version segment. The rerank route adds the version itself and resolves the request to Cohere's v1 rerank endpoint.

note

The rerank route inserts the /v1 segment whenever the base does not already end in /v1, so Cohere's v2 rerank path is not reachable through /v1/rerank. Setting api_base to https://api.cohere.com/v2 builds https://api.cohere.com/v2/v1/rerank and fails upstream.

Create the rerank model alias and a caller key scoped to it:

RERANK_MODEL_ID=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "cohere-rerank-prod",
    "model_name": "rerank-v3.5",
    "provider_key_id": "'"${RERANK_PROVIDER_KEY_ID}"'"
  }' | jq -r '.model.id')

RERANK_API_KEY=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "cohere-rerank-caller",
    "allowed_models": ["'"${RERANK_MODEL_ID}"'"]
  }' | jq -r '.plaintext')

Rerank model IDs follow their own naming, separate from the Command families: rerank-v3.5, and the two Rerank 4.0 variants rerank-v4.0-fast and rerank-v4.0-pro. Because this route resolves to Cohere's v1 rerank endpoint, confirm on the rerank model overview that the ID you pick is served there before you point a production alias at it.

Send a rerank request through the proxy:

curl -sS -X POST "http://127.0.0.1:3000/v1/rerank" \
  -H "Authorization: Bearer ${RERANK_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cohere-rerank-prod",
    "query": "How do I rotate a provider credential?",
    "documents": [
      "Provider keys store the upstream credential.",
      "Caller API keys authorize model access.",
      "Rate limits apply per caller key."
    ],
    "top_n": 2
  }'

AISIX rewrites only the model field to rerank-v3.5 and forwards the body unchanged, so Cohere's top_n parameter reaches the upstream as written. The response keeps Cohere's rerank shape: a results array ordered by relevance_score, plus a meta object. AISIX reads meta.billed_units.input_tokens from that response for usage accounting, so rerank traffic appears in gateway logs and budget totals alongside chat traffic.

Endpoint Support for Cohere

The routes below behave differently depending on which Cohere provider key backs the model alias.

Route	Behavior with a Cohere-backed model
`/v1/chat/completions`	Supported through the compatibility API base.
`/v1/embeddings`	Supported. The `openai` adapter appends `/embeddings` to the compatibility base, which is Cohere's OpenAI-shaped embeddings route. Use a Cohere embedding model ID such as `embed-v4.0` in `model_name`.
`/v1/responses`	Bridged through the chat adapter. AISIX returns a Responses-shaped result for supported request features rather than forwarding to a native Responses API.
`/v1/rerank`	Supported with a second provider key on the native API root. See Add a Rerank Model.
`/v1/images/generations`	Not supported. The route accepts only models whose provider value is `openai`.
`/v1/videos`	Not supported. Cohere is not in the video route's provider allowlist.
`/v1/messages`	Supported for Anthropic-shaped callers through translation. Token counting at `/v1/messages/count_tokens` requires an Anthropic-backed model.
`/passthrough/cohere/*`	Supported. The route borrows the base URL from the first Cohere model the caller key can access, so a key scoped to the compatibility model reaches compatibility paths and a key scoped to the rerank model reaches native paths.

Next Steps

You have now connected AISIX to Cohere, verified the model alias, and added a rerank route. Continue with these guides:

Model Aliases: add routing, cost metadata, and rate limits for these aliases.
Rerank: review the rerank request contract and its provider requirement.
Routing and Failover: fail over between Cohere and a second provider.
Provider Compatibility: review supported proxy endpoints and provider-specific boundaries.

Prerequisites​

Configure the Cohere Upstream​

Create a Provider Key​

Create a Model​

Create a Caller API Key​

Verify the Provider Connection​

Control Reasoning Effort​

Add a Rerank Model​

Endpoint Support for Cohere​

Next Steps​