Skip to main content
API7.ai Logo

Together AI

Together AI provides hosted inference for open and partner models. Applications call selected models through stable AISIX aliases while the gateway holds the Together API key.

Prerequisites​

Before starting, prepare the following:

  • One AISIX setup:
    • For AISIX Cloud, an environment with an attached gateway and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
    • For the open-source AISIX gateway, prepare either a local AISIX installation or the Docker setup from the Open-Source AISIX Gateway Quickstart. Configure the gateway to load a declarative resources file.
  • A Together API key from the Together console.
  • curl and jq.

Configure with AISIX Cloud​

Export the AISIX Cloud connection details:

# AISIX_CP is the Admin API base URL; include /api and omit a trailing slash
# The local On-Premises quickstart uses http://localhost:8080/api
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Create a provider key, model alias, and caller API key for the Together-backed chat-completions route.

Because Together AI exposes an OpenAI-compatible API, AISIX connects through the openai adapter and uses the Together API root as api_base.

Create a Provider Key​

Create the provider key that stores the Together credential and API root, and allow it into the environment:

# Replace with your value
export TOGETHER_API_KEY="YOUR_PROVIDER_API_KEY"

PROVIDER_KEY_ID=$(curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "together-prod",
    "provider": "togetherai",
    "api_key": "'"${TOGETHER_API_KEY}"'",
    "api_base": "https://api.together.ai/v1",
    "allowed_environments": ["'"$ENV_ID"'"]
  }' | jq -r '.provider_key.id')

❶ provider is togetherai, the catalog provider ID.

❷ api_key stores the Together API key. It follows the credential-handling behavior in Provider Keys.

❸ api_base already includes the /v1 path. AISIX appends /chat/completions to it. Together currently documents https://api.together.ai/v1; AISIX Cloud still fills in the equivalent https://api.together.xyz/v1 root when the field is omitted. Set the official root explicitly so the target stays visible on the resource and does not depend on that fallback.

The command captures the returned provider key ID in PROVIDER_KEY_ID.

Create a Model​

Together's catalog changes frequently. Look up the exact <publisher>/<model> ID in the Together models list before creating a model alias.

Create the model alias callers will send in requests:

MODEL_ID=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "together-gptoss-prod",
    "model_name": "openai/gpt-oss-120b",
    "provider_key_id": "'"$PROVIDER_KEY_ID"'"
  }' | jq -r '.model.id')

❶ display_name is the alias callers send in model.

❷ model_name is the Together model ID in <publisher>/<model> form. A flat OpenAI-style name such as gpt-4o returns a 404 from Together.

❸ provider_key_id attaches the alias to the Together provider key.

Create a Caller API Key​

Create the caller API key that can access the model alias. The API generates the key value and returns the plaintext once:

AISIX_API_KEY=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "together-caller",
    "allowed_models": ["'"$MODEL_ID"'"]
  }' | jq -r '.plaintext')

The allowed_models value references the model by its ID. The plaintext key is returned only in this response, so store it securely.

The new resources project to the attached gateway automatically.

Configure with the Open-Source AISIX Gateway​

Export the upstream credential and choose the caller API key that applications will send to the gateway:

export TOGETHER_API_KEY="YOUR_PROVIDER_API_KEY"
export CALLER_API_KEY="YOUR_CALLER_API_KEY"

Create a complete declarative resources file for this provider:

resources.yaml
_format_version: "1"

provider_keys:
  - display_name: "together-prod"
    provider: "togetherai"
    adapter: "openai"
    api_key: ${TOGETHER_API_KEY}
    api_base: "https://api.together.ai/v1"

models:
  - display_name: "together-gptoss-prod"
    provider: "togetherai"
    model_name: "openai/gpt-oss-120b"
    provider_key: "together-prod"

api_keys:
  - display_name: "together-caller"
    key_env: CALLER_API_KEY
    allowed_models:
      - "together-gptoss-prod"

If AISIX is installed locally, validate the file before loading it:

aisix validate --resources resources.yaml

After validation, start the gateway with the referenced environment variables in its process environment. Reload an existing gateway only if those variables are already available to the process; otherwise, restart it with the updated environment.

If you use Docker, adapt the validation and startup commands in the Open-Source AISIX Gateway Quickstart. Mount this resources.yaml file and pass every environment variable it references with -e in both commands. After the resources load, prepare the shared verification request below:

export AISIX_API_KEY="$CALLER_API_KEY"

Verify the Provider Connection​

Export the AISIX gateway origin:

# The local quickstarts use http://127.0.0.1:3000
export AISIX_PROXY="YOUR_AISIX_GATEWAY_ORIGIN"

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "together-gptoss-prod",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from Together AI."
      }
    ]
  }'

The gateway returns an OpenAI-compatible response that echoes the caller-facing alias together-gptoss-prod. If the request fails, check the provider key api_key, api_base, and the <publisher>/<model> ID in model_name.

Endpoint Coverage​

Together exposes several OpenAI-shaped APIs on the same base, but normalized route support also depends on the model's togetherai provider value:

RouteBehavior with a Together AI alias
/v1/chat/completions and /v1/completionsSupported, buffered and streaming where the selected model implements the route.
/v1/responsesSupported through the AISIX Responses bridge. Together does not implement a native Responses API, and fields without a chat-completions equivalent are ignored.
/v1/messagesSupported through translation to chat completions. /v1/messages/count_tokens is limited to Anthropic-backed models.
/v1/embeddingsSupported when the alias names a Together embedding model. AISIX rewrites the caller-facing alias and forwards the OpenAI-shaped request to {api_base}/embeddings.
/v1/audio/transcriptions, /v1/audio/translations, and /v1/audio/speechSupported when the alias names a Together model for the selected audio route. AISIX preserves the OpenAI request and response shapes while rewriting the model alias.
/v1/images/generationsNot supported. The normalized route accepts only models whose configured provider is openai, even though Together serves the same path natively. Use /passthrough/togetherai/images/generations instead.
/v1/videosNot supported. The video route's provider allowlist does not include togetherai; use provider passthrough with Together's native video contract.
/v1/rerankNot supported. The normalized route accepts only the openai, cohere, and jina provider values. Use /passthrough/togetherai/rerank with a Together rerank model and native request body.
/passthrough/togetherai/*restSupported for Together-native routes with limited gateway normalization. Passthrough does not rewrite a caller-facing alias inside the body, buffers upstream SSE responses, and records zero token and cost usage.

See Provider Compatibility for the full endpoint and provider matrix.

Next Steps​

You have now connected AISIX to Together AI and verified the model alias. Continue with these guides: