Skip to main content
API7.ai Logo
Version: 1.4.0

Groq

Groq provides hosted inference for a catalog of open models. AISIX gives applications stable model aliases and caller keys while keeping the Groq credential at the gateway.

Prerequisites​

Before starting, prepare the following:

  • One AISIX setup:
    • For AISIX Cloud, an environment with an attached gateway and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
    • For the open-source AISIX gateway, prepare either a local AISIX installation or the Docker setup from the Open-Source AISIX Gateway Quickstart. Configure the gateway to load a declarative resources file.
  • A Groq API key from the Groq Console.
  • curl.

Configure with AISIX Cloud​

Export the AISIX Cloud connection details:

# AISIX_CP is the Admin API base URL; include /api and omit a trailing slash
# The local On-Premises quickstart uses http://localhost:8080/api
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Create a provider key, model alias, and caller API key for Groq-backed Chat Completions and native Responses.

Because Groq exposes an OpenAI-compatible API, AISIX connects through the openai adapter and uses the Groq API root as api_base.

Create a Provider Key​

Create the provider key that stores the Groq credential and API root:

# Replace with your values
export GROQ_API_KEY="YOUR_PROVIDER_API_KEY"

curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "groq-prod",
    "provider": "groq",
    "api_key": "'"${GROQ_API_KEY}"'",
    "api_base": "https://api.groq.com/openai/v1",
    "apis": {
      "responses": {}
    },
    "allowed_environments": ["'"${ENV_ID}"'"]
  }'

❶ provider is groq. The AISIX Cloud Admin API derives the adapter from the catalog provider; the adapter field is only accepted on BYO provider keys.

❷ api_key stores the Groq API key. The value is encrypted before storage and never returned by read endpoints. It follows the credential-handling behavior in Provider Keys.

❸ api_base already includes the /openai/v1 path. AISIX appends the selected endpoint path to it. The field is optional for this catalog provider because the AISIX Cloud Admin API fills in the same value when you omit it. The example sets it explicitly so the upstream root stays visible on the resource.

❹ apis.responses declares that Groq serves the Responses API at the same root. AISIX can then use Groq's native request and response format instead of translating through Chat Completions.

AISIX Cloud currently retains a legacy compatibility rename from max_completion_tokens to max_tokens. Groq still accepts max_tokens but now deprecates it in favor of max_completion_tokens.

The open-source configuration below intentionally omits this override so current clients can send max_completion_tokens directly.

Copy the provider_key.id value from the response (for example with jq -r '.provider_key.id') and export it:

export PROVIDER_KEY_ID="YOUR_PROVIDER_KEY_ID"

Create a Model​

Groq model availability changes over time. Check the Groq models list for a current model ID before creating a model alias.

Create the model alias callers will send in requests:

curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "groq-gptoss-prod",
    "model_name": "openai/gpt-oss-120b",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }'

❶ display_name is the alias callers send in model.

❷ model_name is the Groq model ID, for example openai/gpt-oss-120b.

❸ provider_key_id attaches the alias to the Groq provider key.

Capture the model.id value from the response:

export MODEL_ID="YOUR_MODEL_ID"

Create a Caller API Key​

Create the caller API key resource that can access the model alias. The gateway generates the key value and returns the plaintext once in the response:

curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "groq-caller",
    "allowed_models": ["'"${MODEL_ID}"'"]
  }'

The allowed_models value must reference the model ID you captured. Copy the plaintext value from the response — it is shown only once — and export it as the caller key:

export AISIX_API_KEY="YOUR_CALLER_API_KEY"

After the write, the configuration projects to attached gateways automatically.

Configure with the Open-Source AISIX Gateway​

Export the upstream credential and choose the caller API key that applications will send to the gateway:

export GROQ_API_KEY="YOUR_PROVIDER_API_KEY"
export CALLER_API_KEY="YOUR_CALLER_API_KEY"

For a new gateway, use this complete resources file. For an existing gateway, merge the entries into its current file, preserving its other resources:

resources.yaml
_format_version: "1"

provider_keys:
  - display_name: "groq-prod"
    provider: "groq"
    adapter: "openai"
    api_key: ${GROQ_API_KEY}
    api_base: "https://api.groq.com/openai/v1"
    apis:
      responses: {}

models:
  - display_name: "groq-gptoss-prod"
    provider: "groq"
    model_name: "openai/gpt-oss-120b"
    provider_key: "groq-prod"

api_keys:
  - display_name: "groq-caller"
    key_env: CALLER_API_KEY
    allowed_models:
      - "groq-gptoss-prod"

If AISIX is installed locally, validate the file before loading it:

aisix validate --resources resources.yaml

After validation, start the gateway with the referenced environment variables in its process environment. Reload an existing gateway only if those variables are already available to the process; otherwise, restart it with the updated environment.

If you use Docker, adapt the validation and startup commands in the Open-Source AISIX Gateway Quickstart. Mount this resources.yaml file and pass every environment variable it references with -e in both commands. After the resources load, prepare the shared verification request below:

export AISIX_API_KEY="$CALLER_API_KEY"

Verify the Provider Connection​

Export the AISIX gateway origin:

# AISIX_PROXY has no trailing slash or endpoint path
# The local quickstarts use http://127.0.0.1:3000
export AISIX_PROXY="YOUR_AISIX_GATEWAY_URL"

Send a Responses request through the AISIX proxy:

curl -sS -X POST "$AISIX_PROXY/v1/responses" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "groq-gptoss-prod",
    "input": "Say hello from Groq."
  }'

AISIX sends the request to Groq's native Responses endpoint, replaces the AISIX alias with the configured Groq model ID upstream, and restores the alias in the response. If the request fails, check the provider key credential, API root, and Groq model ID.

Endpoint and Compatibility Boundaries​

Groq is mostly, but not completely, OpenAI-compatible. AISIX forwards additional chat request fields instead of filtering them, so Groq enforces its own model and parameter rules. Groq's compatibility guide says logprobs, logit_bias, top_logprobs, and messages[].name return 400, and n must be 1. Groq's API reference also marks frequency_penalty, presence_penalty, metadata, and store as unsupported.

A temperature of 0 is converted upstream to 1e-8. For audio transcription and translation, Groq does not support vtt or srt output. See Groq's OpenAI compatibility guide for the current list.

RouteBehavior with a groq provider key
/v1/chat/completionsSupported, including streaming. AISIX forwards fields such as tools, response_format, and reasoning_effort; support still depends on the selected Groq model. The example GPT-OSS model supports tool use, structured output, and low, medium, or high reasoning effort.
/v1/responsesSent to Groq's native Responses API because this guide declares apis.responses. Groq currently labels the API beta and does not support every OpenAI Responses field. Without the declaration, AISIX uses the chat-based Responses bridge.
/v1/messagesSupported through translation to chat, not a native Groq Messages API. /v1/messages/count_tokens is unavailable because the configured provider is not Anthropic.
/v1/audio/transcriptions, /v1/audio/translations, and /v1/audio/speechSupported with separate model aliases for current Groq speech-to-text or text-to-speech models. Check Groq's model list before configuring an audio alias.
/v1/files and /v1/batchesSupported through Groq's OpenAI-compatible file and batch APIs. Use the Groq alias as the routing model as described in Batch, Files, and Fine-Tuning. Groq applies discounted batch pricing, but AISIX estimates batch cost from the alias's configured synchronous prices; do not treat that estimate as the provider invoice.
/v1/fine_tuning/jobsNot compatible with Groq fine-tuning. AISIX forwards the OpenAI /fine_tuning/jobs contract under api_base, while Groq's closed-beta API uses /v1/fine_tunings outside the /openai/v1 root and has a different body.
/v1/embeddings and /v1/completionsNot supported because Groq does not expose those upstream endpoints.
/v1/images/generations, /v1/videos, and /v1/rerankNot supported by Groq or by these AISIX routes' provider rules.

The normalized chat response preserves standard content, tool calls, reasoning text, and token totals. AISIX returns Groq's native message.reasoning value as reasoning_content. It does not preserve Groq-specific metadata such as x_groq, usage timing details, usage_breakdown, or unmodeled citation and annotation fields.

The native declaration keeps model-alias rewriting and model access controls on the normalized AISIX route. A separate passthrough route is no longer needed for Groq Responses. For another Groq endpoint under the same API root that AISIX does not normalize, configure a passthrough route with your chosen prefix and https://api.groq.com/openai/v1 as its target_url. Grant the route name in the caller key's allowed_routes, and send the upstream model ID because passthrough does not rewrite aliases.

Next Steps​

You have now connected AISIX to Groq and verified the model alias. Continue with these guides: