Skip to main content
API7.ai Logo

Groq

Groq provides hosted inference for a catalog of open models. AISIX gives applications stable model aliases and caller keys while keeping the Groq credential at the gateway.

Prerequisites​

Before starting, prepare the following:

  • One AISIX setup:
    • For AISIX Cloud, an environment with an attached gateway and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
    • For the open-source AISIX gateway, prepare either a local AISIX installation or the Docker setup from the Open-Source AISIX Gateway Quickstart. Configure the gateway to load a declarative resources file.
  • A Groq API key from the Groq Console.
  • curl.

Configure with AISIX Cloud​

Export the AISIX Cloud connection details:

# AISIX_CP is the Admin API base URL; include /api and omit a trailing slash
# The local On-Premises quickstart uses http://localhost:8080/api
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Create a provider key, model alias, and caller API key for the Groq-backed chat-completions route.

Because Groq exposes an OpenAI-compatible API, AISIX connects through the openai adapter and uses the Groq API root as api_base.

Create a Provider Key​

Create the provider key that stores the Groq credential and API root:

# Replace with your values
export GROQ_API_KEY="YOUR_PROVIDER_API_KEY"

curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "groq-prod",
    "provider": "groq",
    "api_key": "'"${GROQ_API_KEY}"'",
    "api_base": "https://api.groq.com/openai/v1",
    "allowed_environments": ["'"${ENV_ID}"'"]
  }'

❶ provider is groq. The AISIX Cloud Admin API derives the adapter from the catalog provider; the adapter field is only accepted on BYO provider keys.

❷ api_key stores the Groq API key. The value is encrypted before storage and never returned by read endpoints. It follows the credential-handling behavior in Provider Keys.

❸ api_base already includes the /openai/v1 path. AISIX appends /chat/completions to it. The field is optional for this catalog provider because the AISIX Cloud Admin API fills in the same value when you omit it. The example sets it explicitly so the upstream root stays visible on the resource.

AISIX Cloud currently retains a legacy compatibility rename from max_completion_tokens to max_tokens. Groq still accepts max_tokens but now deprecates it in favor of max_completion_tokens.

The open-source configuration below intentionally omits this override so current clients can send max_completion_tokens directly.

Copy the provider_key.id value from the response (for example with jq -r '.provider_key.id') and export it:

export PROVIDER_KEY_ID="YOUR_PROVIDER_KEY_ID"

Create a Model​

Groq model availability changes over time. Check the Groq models list for a current model ID before creating a model alias.

Create the model alias callers will send in requests:

curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "groq-gptoss-prod",
    "model_name": "openai/gpt-oss-120b",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }'

❶ display_name is the alias callers send in model.

❷ model_name is the Groq model ID, for example openai/gpt-oss-120b.

❸ provider_key_id attaches the alias to the Groq provider key.

Capture the model.id value from the response:

export MODEL_ID="YOUR_MODEL_ID"

Create a Caller API Key​

Create the caller API key resource that can access the model alias. The gateway generates the key value and returns the plaintext once in the response:

curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "groq-caller",
    "allowed_models": ["'"${MODEL_ID}"'"]
  }'

The allowed_models value must reference the model ID you captured. Copy the plaintext value from the response — it is shown only once — and export it as the caller key:

export AISIX_API_KEY="YOUR_CALLER_API_KEY"

After the write, the configuration projects to attached gateways automatically.

Configure with the Open-Source AISIX Gateway​

Export the upstream credential and choose the caller API key that applications will send to the gateway:

export GROQ_API_KEY="YOUR_PROVIDER_API_KEY"
export CALLER_API_KEY="YOUR_CALLER_API_KEY"

Create a complete declarative resources file for this provider:

resources.yaml
_format_version: "1"

provider_keys:
  - display_name: "groq-prod"
    provider: "groq"
    adapter: "openai"
    api_key: ${GROQ_API_KEY}
    api_base: "https://api.groq.com/openai/v1"

models:
  - display_name: "groq-gptoss-prod"
    provider: "groq"
    model_name: "openai/gpt-oss-120b"
    provider_key: "groq-prod"

api_keys:
  - display_name: "groq-caller"
    key_env: CALLER_API_KEY
    allowed_models:
      - "groq-gptoss-prod"

If AISIX is installed locally, validate the file before loading it:

aisix validate --resources resources.yaml

After validation, start the gateway with the referenced environment variables in its process environment. Reload an existing gateway only if those variables are already available to the process; otherwise, restart it with the updated environment.

If you use Docker, adapt the validation and startup commands in the Open-Source AISIX Gateway Quickstart. Mount this resources.yaml file and pass every environment variable it references with -e in both commands. After the resources load, prepare the shared verification request below:

export AISIX_API_KEY="$CALLER_API_KEY"

Verify the Provider Connection​

Export the AISIX gateway origin:

# The local quickstarts use http://127.0.0.1:3000
export AISIX_PROXY="YOUR_AISIX_GATEWAY_ORIGIN"

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "groq-gptoss-prod",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from Groq."
      }
    ]
  }'

The gateway returns an OpenAI-compatible response that echoes the caller-facing alias groq-gptoss-prod. If the request fails, check the provider key api_key, api_base, and the Groq model ID in model_name.

Endpoint and Compatibility Boundaries​

Groq is mostly, but not completely, OpenAI-compatible. AISIX forwards additional chat request fields instead of filtering them, so Groq enforces its own model and parameter rules. Groq's compatibility guide says logprobs, logit_bias, top_logprobs, and messages[].name return 400, and n must be 1. Groq's API reference also marks frequency_penalty, presence_penalty, metadata, and store as unsupported.

A temperature of 0 is converted upstream to 1e-8. For audio transcription and translation, Groq does not support vtt or srt output. See Groq's OpenAI compatibility guide for the current list.

RouteBehavior with a groq provider key
/v1/chat/completionsSupported, including streaming. AISIX forwards fields such as tools, response_format, and reasoning_effort; support still depends on the selected Groq model. The example GPT-OSS model supports tool use, structured output, and low, medium, or high reasoning effort.
/v1/responsesSupported through the chat-based Responses bridge, not Groq's native beta Responses API. Fields and response items without a chat equivalent are not preserved.
/v1/messagesSupported through translation to chat, not a native Groq Messages API. /v1/messages/count_tokens is unavailable because the configured provider is not Anthropic.
/v1/audio/transcriptions, /v1/audio/translations, and /v1/audio/speechSupported with separate model aliases for current Groq speech-to-text or text-to-speech models. Check Groq's model list before configuring an audio alias.
/v1/files and /v1/batchesSupported through Groq's OpenAI-compatible file and batch APIs. Use the Groq alias as the routing model as described in Batch, Files, and Fine-Tuning. Groq applies discounted batch pricing, but AISIX estimates batch cost from the alias's configured synchronous prices; do not treat that estimate as the provider invoice.
/v1/fine_tuning/jobsNot compatible with Groq fine-tuning. AISIX forwards the OpenAI /fine_tuning/jobs contract under api_base, while Groq's closed-beta API uses /v1/fine_tunings outside the /openai/v1 root and has a different body.
/v1/embeddings and /v1/completionsNot supported because Groq does not expose those upstream endpoints.
/v1/images/generations, /v1/videos, and /v1/rerankNot supported by Groq or by these AISIX routes' provider rules.

The normalized chat response preserves standard content, tool calls, reasoning text, and token totals. AISIX returns Groq's native message.reasoning value as reasoning_content. It does not preserve Groq-specific metadata such as x_groq, usage timing details, usage_breakdown, or unmodeled citation and annotation fields.

To use Groq's native Responses wire shape, call /passthrough/groq/responses. The path is relative to the configured https://api.groq.com/openai/v1 base. Send the exact Groq model ID because passthrough does not rewrite AISIX aliases. AISIX relays the provider response body without normalization unless a guardrail blocks it. Passthrough requests record zero tokens and therefore do not provide normalized token or cost accounting. The configured base also means generic passthrough cannot reach Groq's sibling /v1/fine_tunings API.

Next Steps​

You have now connected AISIX to Groq and verified the model alias. Continue with these guides: