Skip to main content
API7.ai Logo

Other OpenAI-Compatible Providers

Many public model providers expose an OpenAI-compatible API. AISIX Cloud can connect to a provider when it accepts bearer-authenticated OpenAI chat-completions requests and appears in the AISIX provider catalog. The open-source AISIX gateway can use any reachable endpoint with the same protocol and authentication shape under an operator-chosen provider label.

When a provider has a dedicated setup listed under Provider Upstreams, follow that page instead. The configuration below is a parameterized template for another public provider; the AISIX Cloud path assumes that the provider is in its catalog.

For a private or customer-operated server, use the dedicated Ollama, vLLM, or Bring Your Own Endpoint guide.

Prerequisites​

Before starting, prepare the following:

  • One AISIX setup:
    • For AISIX Cloud, an environment with an attached gateway and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
    • For the open-source AISIX gateway, prepare either a local AISIX installation or the Docker setup from the Open-Source AISIX Gateway Quickstart. Configure the gateway to load a declarative resources file.
  • An API key for an OpenAI-compatible provider. For AISIX Cloud, the provider must be listed in the AISIX provider catalog; the open-source resources workflow does not use that catalog for admission.
  • curl and jq.

Select Provider Values​

For AISIX Cloud, choose the exact provider ID shown by the AISIX provider catalog. For an open-source resources file, choose a stable provider label, such as the provider's lowercase name. Then copy the API root and model ID from the provider's official API reference:

export PROVIDER_ID="YOUR_PROVIDER_ID"
export PROVIDER_API_KEY="YOUR_PROVIDER_API_KEY"
export PROVIDER_API_BASE="https://api.provider.example/v1"
export UPSTREAM_MODEL_ID="publisher/model-id"
export MODEL_ALIAS="provider-model-prod"

The address and IDs above are fictional placeholders. Replace every value before running the setup.

Configure with AISIX Cloud​

Export the AISIX Cloud connection details:

# AISIX_CP is the Admin API base URL; include /api and omit a trailing slash
# The local On-Premises quickstart uses http://localhost:8080/api
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Create a provider key, model alias, and caller API key for the provider-backed chat-completions route.

AISIX connects through the openai adapter and uses the provider's API root as api_base. Give the provider key a descriptive label so operators can identify the upstream later.

Create a Provider Key​

PROVIDER_KEY_RESPONSE=$(
  curl -sS -X POST "$AISIX_CP/provider_keys" \
    -H "Authorization: Bearer $AISIX_TOKEN" \
    -H "Content-Type: application/json" \
    --data-binary @- <<EOF
{
  "display_name": "community-provider-prod",
  "provider": "${PROVIDER_ID}",
  "api_key": "${PROVIDER_API_KEY}",
  "api_base": "${PROVIDER_API_BASE}",
  "allowed_environments": ["${ENV_ID}"]
}
EOF
)

PROVIDER_KEY_ID=$(printf '%s' "$PROVIDER_KEY_RESPONSE" | jq -er '.provider_key.id')
echo "$PROVIDER_KEY_ID"

provider must exactly match an ID in the AISIX catalog. The AISIX Cloud Admin API derives the openai adapter and bearer authentication for community catalog providers. Do not send an adapter field; that field is accepted only when provider is byo.

A provider without a dedicated setup page is admitted from the synced public catalog rather than from the gateway's built-in provider list. If the create returns 400 INVALID_REQUEST naming the catalog, the control plane does not have that provider in its current catalog. Connected deployments sync at start and every 24 hours; packaged On-Premises deployments use a bundled snapshot by default and do not refresh it. Retry after an online sync, review the On-Premises pricing-catalog settings, or configure the upstream with Bring Your Own Endpoint.

AISIX appends the endpoint path to api_base. Include provider-specific prefixes such as /v1, /openai, or /openai/v1 when the official endpoint requires them. Setting the root explicitly also avoids depending on the API base cached by the latest catalog sync.

Provider key secrets follow the credential-handling behavior described in Provider Keys.

Create a Model​

Map the caller-facing alias to the provider's exact model ID:

MODEL_RESPONSE=$(
  curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
    -H "Authorization: Bearer $AISIX_TOKEN" \
    -H "Content-Type: application/json" \
    --data-binary @- <<EOF
{
  "display_name": "${MODEL_ALIAS}",
  "model_name": "${UPSTREAM_MODEL_ID}",
  "provider_key_id": "${PROVIDER_KEY_ID}"
}
EOF
)

MODEL_ID=$(printf '%s' "$MODEL_RESPONSE" | jq -er '.model.id')
echo "$MODEL_ID"

display_name is the alias callers send in model. model_name is sent to the upstream unchanged, so preserve any publisher namespace, casing, punctuation, and version suffix.

To configure cost metadata for budget accounting or usage reports, see Model Aliases.

Create a Caller API Key​

Create a caller key limited to the model resource:

AISIX_API_KEY=$(
  curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
    -H "Authorization: Bearer $AISIX_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
      "display_name": "community-provider-caller",
      "allowed_models": ["'"${MODEL_ID}"'"]
    }' | jq -er '.plaintext'
)

echo "$AISIX_API_KEY"

The plaintext is returned only when the caller key is created. Store it securely.

Configure with the Open-Source AISIX Gateway​

Export the upstream credential and choose the caller API key that applications will send to the gateway:

export PROVIDER_ID="YOUR_PROVIDER_ID"
export PROVIDER_API_BASE="YOUR_PROVIDER_API_BASE"
export UPSTREAM_MODEL_ID="YOUR_UPSTREAM_MODEL_ID"
export MODEL_ALIAS="YOUR_MODEL_ALIAS"
export PROVIDER_API_KEY="YOUR_PROVIDER_API_KEY"
export CALLER_API_KEY="YOUR_CALLER_API_KEY"

Create a complete declarative resources file for this provider:

The open-source gateway validates the provider label's shape but does not require it to appear in the AISIX Cloud catalog. Use the same label on the provider key and model. The label also identifies the upstream in telemetry and in a passthrough path such as /passthrough/example-provider/*.

resources.yaml
_format_version: "1"

provider_keys:
  - display_name: "community-provider-prod"
    provider: "${PROVIDER_ID}"
    adapter: "openai"
    api_key: ${PROVIDER_API_KEY}
    api_base: "${PROVIDER_API_BASE}"

models:
  - display_name: "${MODEL_ALIAS}"
    provider: "${PROVIDER_ID}"
    model_name: "${UPSTREAM_MODEL_ID}"
    provider_key: "community-provider-prod"

api_keys:
  - display_name: "community-provider-caller"
    key_env: CALLER_API_KEY
    allowed_models:
      - "${MODEL_ALIAS}"

If AISIX is installed locally, validate the file before loading it:

aisix validate --resources resources.yaml

After validation, start the gateway with the referenced environment variables in its process environment. Reload an existing gateway only if those variables are already available to the process; otherwise, restart it with the updated environment.

If you use Docker, adapt the validation and startup commands in the Open-Source AISIX Gateway Quickstart. Mount this resources.yaml file and pass every environment variable it references with -e in both commands. After the resources load, prepare the shared verification request below:

export AISIX_API_KEY="$CALLER_API_KEY"

Verify the Provider Connection​

Export the AISIX gateway origin:

# The local quickstarts use http://127.0.0.1:3000
export AISIX_PROXY="YOUR_AISIX_GATEWAY_ORIGIN"

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer $AISIX_API_KEY" \
  -H "Content-Type: application/json" \
  --data-binary @- <<EOF
{
  "model": "${MODEL_ALIAS}",
  "messages": [
    {
      "role": "user",
      "content": "Say hello through the configured provider."
    }
  ]
}
EOF

The response should be OpenAI-compatible and should carry the caller-facing alias. Use the provider's request logs or usage page, when available, to confirm that the request reached the intended upstream account and model.

If the gateway returns an upstream authentication error, check the provider key's api_key. If it returns an upstream route error, check api_base and UPSTREAM_MODEL_ID.

Support Provider-Specific Behavior​

The provider must accept OpenAI chat-completions requests. A provider with a different request format needs a native adapter protocol family or a compatible endpoint.

The openai adapter does not make every normalized AISIX endpoint available to every provider label:

  • /v1/completions, /v1/embeddings, /v1/audio/*, /v1/files, /v1/batches, and /v1/fine_tuning/jobs can dispatch through this adapter, but they work only when the upstream implements the corresponding OpenAI route and fields.
  • /v1/responses uses the Responses bridge over chat for a community provider label. /v1/messages similarly translates the request and response through chat rather than using a provider-native Messages route.
  • /v1/images/generations, /v1/rerank, and /v1/videos enforce provider allowlists. They can reject a community provider label even when its upstream exposes a similarly named route.

Use provider passthrough when an application needs a provider-native route or contract. Passthrough does not rewrite the model alias and has different streaming and usage-accounting behavior, so review its limitations before adopting it. See Provider Compatibility for the normalized-route matrix.

AISIX preserves reasoning_content and normalizes reasoning to that canonical field. If a provider streams reasoning from a different delta path, use the response.reasoning_field override on the provider key.

Next Steps​