Together AI

In this guide, you will connect AISIX AI Gateway to Together AI. Callers reach Together-hosted open-source models through the gateway's OpenAI-compatible API, while AISIX owns the credential, model allowlist, rate limits, and usage accounting.

Together AI exposes an OpenAI-compatible API, so it uses the openai adapter with a Together api_base. Use this configuration when applications should authenticate to AISIX with a caller API key instead of holding the Together key directly.

Prerequisites

Before starting, prepare the following:

A gateway with the admin API on :3001 and the proxy API on :3000.
The admin key from the gateway config.yaml.
A Together API key from the Together console. Together's OpenAI-compatible API root is https://api.together.xyz/v1.

Supported Capabilities

Together AI is a chat-completions upstream.

Endpoint	Supported	Streaming	Notes
`/v1/chat/completions`	Yes	Yes	Primary path. Together uses `<publisher>/<model>` IDs, for example `meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8`.
`/v1/images/generations`	No	—	Rejected: image generation requires `provider: "openai"`.
`/v1/rerank`	No	—	Together is not in the rerank allowlist (`openai`, `cohere`, `jina`).

See Provider Compatibility for the exact per-endpoint rules.

Configure the Together Upstream

Create a provider key, model alias, and caller API key for the Together-backed route.

Create a Provider Key

export AISIX_ADMIN_KEY="YOUR_ADMIN_KEY"
export TOGETHER_API_KEY="YOUR_PROVIDER_API_KEY"

curl -sS -X POST "http://127.0.0.1:3001/admin/v1/provider_keys" \
  -H "Authorization: Bearer ${AISIX_ADMIN_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "together-prod",
    "provider": "together",
    "adapter": "openai",
    "secret": "'"${TOGETHER_API_KEY}"'",
    "api_base": "https://api.together.xyz/v1"
  }'

❶ provider is together.

❷ adapter is openai, because Together accepts OpenAI chat-completions requests.

❸ secret stores the Together API key. It follows the credential-handling behavior in Provider Credentials.

❹ api_base is required and already includes the /v1 path. AISIX appends /chat/completions to it.

Copy the returned provider key ID.

Create a Model

export PROVIDER_KEY_ID="YOUR_PROVIDER_KEY_ID"

curl -sS -X POST "http://127.0.0.1:3001/admin/v1/models" \
  -H "Authorization: Bearer ${AISIX_ADMIN_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "together-llama-prod",
    "provider": "together",
    "model_name": "meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }'

❶ display_name is the alias callers send in model.

❷ model_name is the Together model ID in <publisher>/<model> form. A flat OpenAI-style name such as gpt-4o returns a 404 from Together.

❸ provider_key_id attaches the alias to the Together provider key.

Create a Caller API Key

Choose the caller API key value that the application will send to AISIX, then hash it for the admin resource:

export AISIX_API_KEY="YOUR_CALLER_API_KEY"

CALLER_KEY_HASH=$(printf '%s' "${AISIX_API_KEY}" | shasum -a 256 | awk '{print $1}')

curl -sS -X POST "http://127.0.0.1:3001/admin/v1/apikeys" \
  -H "Authorization: Bearer ${AISIX_ADMIN_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "key_hash": "'"${CALLER_KEY_HASH}"'",
    "allowed_models": ["together-llama-prod"]
  }'

❶ allowed_models must match the model alias you created.

Verify the Upstream

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "http://127.0.0.1:3000/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "together-llama-prod",
    "messages": [{"role": "user", "content": "Say hello from Together AI."}]
  }'

The gateway returns an OpenAI-compatible response that echoes the caller-facing alias together-llama-prod. If AISIX returns an upstream authentication error, check the provider key secret; if it returns an upstream route error, check api_base and the <publisher>/<model> ID in model_name.

Behavior and Limits

Together's catalog changes frequently. Look up the exact <publisher>/<model> ID in the Together models list before creating a model alias.

Response extensions beyond the OpenAI envelope are not normalized by default.

Next Steps

Model Aliases — add routing, cost metadata, and rate limits for this alias.
Routing and Failover — fail over between Together and a second provider.
Provider Compatibility — the endpoint-by-endpoint support rules.

Prerequisites​

Supported Capabilities​

Configure the Together Upstream​

Create a Provider Key​

Create a Model​

Create a Caller API Key​

Verify the Upstream​

Behavior and Limits​

Next Steps​