Groq

In this guide, you will connect AISIX AI Gateway to Groq. Callers reach Groq-hosted models through the gateway's OpenAI-compatible API, while AISIX owns the credential, model allowlist, rate limits, and usage accounting.

Groq exposes an OpenAI-compatible API, so it uses the openai adapter with a Groq api_base. Use this configuration when applications should authenticate to AISIX with a caller API key instead of holding the Groq key directly.

Prerequisites

Before starting, prepare the following:

A gateway with the admin API on :3001 and the proxy API on :3000.
The admin key from the gateway config.yaml.
A Groq API key from the Groq Console. Groq's OpenAI-compatible API root is https://api.groq.com/openai/v1.

Supported Capabilities

Groq is a chat-completions upstream.

Endpoint	Supported	Streaming	Notes
`/v1/chat/completions`	Yes	Yes	Primary path for `openai/gpt-oss-120b` and other Groq-hosted models.
`/v1/images/generations`	No	—	Rejected: image generation requires `provider: "openai"`.
`/v1/rerank`	No	—	Groq is not in the rerank allowlist (`openai`, `cohere`, `jina`).

See Provider Compatibility for the exact per-endpoint rules.

Configure the Groq Upstream

Create a provider key, model alias, and caller API key for the Groq-backed route.

Create a Provider Key

export AISIX_ADMIN_KEY="YOUR_ADMIN_KEY"
export GROQ_API_KEY="YOUR_PROVIDER_API_KEY"

curl -sS -X POST "http://127.0.0.1:3001/admin/v1/provider_keys" \
  -H "Authorization: Bearer ${AISIX_ADMIN_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "groq-prod",
    "provider": "groq",
    "adapter": "openai",
    "secret": "'"${GROQ_API_KEY}"'",
    "api_base": "https://api.groq.com/openai/v1"
  }'

❶ provider is groq.

❷ adapter is openai, because Groq accepts OpenAI chat-completions requests.

❸ secret stores the Groq API key. It follows the credential-handling behavior in Provider Credentials.

❹ api_base is required and already includes the /openai/v1 path. AISIX appends /chat/completions to it.

Copy the returned provider key ID.

Create a Model

export PROVIDER_KEY_ID="YOUR_PROVIDER_KEY_ID"

curl -sS -X POST "http://127.0.0.1:3001/admin/v1/models" \
  -H "Authorization: Bearer ${AISIX_ADMIN_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "groq-gptoss-prod",
    "provider": "groq",
    "model_name": "openai/gpt-oss-120b",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }'

❶ display_name is the alias callers send in model.

❷ model_name is the Groq model ID, for example openai/gpt-oss-120b. Groq is deprecating the standalone llama-3.x IDs, so confirm the current ID in the Groq models list.

❸ provider_key_id attaches the alias to the Groq provider key.

Create a Caller API Key

Choose the caller API key value that the application will send to AISIX, then hash it for the admin resource:

export AISIX_API_KEY="YOUR_CALLER_API_KEY"

CALLER_KEY_HASH=$(printf '%s' "${AISIX_API_KEY}" | shasum -a 256 | awk '{print $1}')

curl -sS -X POST "http://127.0.0.1:3001/admin/v1/apikeys" \
  -H "Authorization: Bearer ${AISIX_ADMIN_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "key_hash": "'"${CALLER_KEY_HASH}"'",
    "allowed_models": ["groq-gptoss-prod"]
  }'

❶ allowed_models must match the model alias you created.

Verify the Upstream

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "http://127.0.0.1:3000/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "groq-gptoss-prod",
    "messages": [{"role": "user", "content": "Say hello from Groq."}]
  }'

The gateway returns an OpenAI-compatible response that echoes the caller-facing alias groq-gptoss-prod. If AISIX returns an upstream authentication error, check the provider key secret; if it returns an upstream route error, check api_base and the Groq model ID in model_name.

Behavior and Limits

As of mid-2026, Groq began deprecating the standalone llama-3.x model IDs in favor of IDs such as openai/gpt-oss-120b. Groq model availability changes over time, so check the Groq models list for the current IDs before creating a model alias.

Response extensions beyond the OpenAI envelope are not normalized by default.

Next Steps

Model Aliases — add routing, cost metadata, and rate limits for this alias.
Routing and Failover — fail over between Groq and a second provider.
Provider Compatibility — the endpoint-by-endpoint support rules.

Prerequisites​

Supported Capabilities​

Configure the Groq Upstream​

Create a Provider Key​

Create a Model​

Create a Caller API Key​

Verify the Upstream​

Behavior and Limits​

Next Steps​