DeepInfra

Connect AISIX AI Gateway to DeepInfra so applications can call DeepInfra-hosted open-weight models through the gateway's OpenAI-compatible API. AISIX keeps the DeepInfra credential on the gateway side. It authorizes model access with caller API keys and allowlists, and applies the same rate-limit and usage-accounting controls as other model aliases.

DeepInfra serves open-weight models from many publishers and exposes an OpenAI-compatible API, so it uses the openai adapter with a DeepInfra api_base.

deepinfra reaches AISIX through the community catalog rather than through a curated provider entry. The Cloud Admin API accepts "provider": "deepinfra" because the ID is published in the models.dev catalog, and the catalog default assigns the openai adapter with HTTP bearer authentication. What the default does not supply is a per-provider base URL or any request and response rewrites, so this page states exactly which values you are responsible for setting.

note

Because deepinfra is a community-catalog provider, the Cloud Admin API catalog returns community_badge: true for it, and the dashboard's model-pricing view groups its models under All providers (community) with the note that wire compatibility is assumed to be OpenAI-shaped unless you override it per provider key. The provider works, and the sections below cover the two overrides that matter for DeepInfra.

Prerequisites

The following examples configure the upstream through the AISIX Cloud Admin API. Before configuring the upstream, prepare the following:

Access to AISIX Cloud with an environment, an attached gateway, and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
A DeepInfra API token from the DeepInfra dashboard.

Export the connection details used by every request below:

# Replace with your values
export AISIX_CP="http://localhost:8080/api"
export AISIX_TOKEN="aisix_pat_YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Configure the DeepInfra Upstream

Create a provider key, model alias, and caller API key for the DeepInfra-backed chat-completions route.

Create a Provider Key

Create the provider key that stores the DeepInfra credential and API root:

# Replace with your value
export DEEPINFRA_API_KEY="YOUR_PROVIDER_API_KEY"

PROVIDER_KEY_ID=$(curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "deepinfra-prod",
    "provider": "deepinfra",
    "api_key": "'"${DEEPINFRA_API_KEY}"'",
    "api_base": "https://api.deepinfra.com/v1/openai",
    "allowed_environments": ["'"${ENV_ID}"'"]
  }' | jq -r '.provider_key.id')

echo "$PROVIDER_KEY_ID"

❶ provider is deepinfra. The Cloud Admin API derives the adapter from the catalog entry; the adapter field is only accepted on BYO provider keys, and sending it on a catalog key returns a 400 error. For deepinfra the derived adapter is openai.

❷ api_key stores the DeepInfra API token. DeepInfra authenticates the OpenAI-compatible surface with HTTP bearer authentication, which is what the openai adapter already sends. The value follows the credential-handling behavior in Provider Credentials.

❸ api_base is required for DeepInfra. models.dev publishes no api field for the deepinfra entry, so there is no cached default for the Cloud Admin API to fall back on. Omitting api_base returns a 400 error stating that models.dev does not publish a default base URL for this provider.

The command captures the returned provider key ID in PROVIDER_KEY_ID.

Why the Base URL Carries Both `/v1` and `/openai`

DeepInfra documents its full OpenAI-compatible endpoint as https://api.deepinfra.com/v1/openai/chat/completions in Chat Completions, and its own SDK examples set base_url to https://api.deepinfra.com/v1/openai. AISIX appends the endpoint path, such as /chat/completions or /embeddings, to api_base, so api_base must be the root that those paths hang off — which for DeepInfra includes both the /v1 segment and the /openai segment.

Two related behaviors are worth knowing:

If you paste the full endpoint URL, AISIX strips a known endpoint suffix such as /chat/completions along with any trailing slash before building the upstream URL. Setting api_base to https://api.deepinfra.com/v1/openai/chat/completions therefore still works, but prefer the root form so the value stays readable.
AISIX does not synthesize a missing path segment for a non-OpenAI vendor. Setting api_base to the bare host https://api.deepinfra.com produces the upstream URL https://api.deepinfra.com/chat/completions, which DeepInfra does not serve. Use the root DeepInfra documents rather than relying on a shorter form to be repaired. Leaving api_base empty is not a workaround either: the gateway refuses to fall back to the OpenAI host for a non-OpenAI provider and returns an upstream configuration error instead of leaking the DeepInfra token to another vendor.

Create a Model

DeepInfra model IDs are namespaced by the weights publisher, in the form <publisher>/<Model-Name>, matching the Hugging Face repository ID for the same weights. The casing is not uniform, and the publisher segment is the Hub organization rather than the vendor's brand name — for example the GLM models are published under zai-org, not zhipuai. Copy the ID verbatim from the DeepInfra model catalog rather than retyping it.

Current IDs include:

Model ID	Notes
`deepseek-ai/DeepSeek-V3.2`	Reasoning model; returns reasoning in `reasoning_content`.
`openai/gpt-oss-120b`	Open-weight model; accepts `reasoning_effort` values `low`, `medium`, and `high`.
`meta-llama/Llama-3.3-70B-Instruct-Turbo`	Non-reasoning instruction model.

Create the model alias callers will send in requests:

MODEL_ID=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "deepinfra-deepseek-prod",
    "model_name": "deepseek-ai/DeepSeek-V3.2",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }' | jq -r '.model.id')

echo "$MODEL_ID"

❶ display_name is the alias callers send in model.

❷ model_name is the full DeepInfra model ID, including the publisher segment. A bare identifier such as DeepSeek-V3.2 is rejected by the upstream.

❸ provider_key_id attaches the alias to the DeepInfra provider key.

Create a Caller API Key

Create the caller API key that can access the model alias. The plaintext key is server-generated and returned once in the create response, so capture it now:

AISIX_API_KEY=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "deepinfra-caller",
    "allowed_models": ["'"${MODEL_ID}"'"]
  }' | jq -r '.plaintext')

echo "$AISIX_API_KEY"

The allowed_models value must reference the model ID captured in the previous step. After the write, the configuration projects to attached gateways automatically.

Verify the Provider Connection

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "http://127.0.0.1:3000/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepinfra-deepseek-prod",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from DeepInfra."
      }
    ]
  }'

The gateway returns an OpenAI-compatible response that echoes the caller-facing alias deepinfra-deepseek-prod.

If the request fails, isolate the cause by symptom:

Symptom	Likely cause
Upstream authentication error	The DeepInfra token in `api_key` is wrong or revoked.
Upstream 404	`api_base` is missing the `/openai` segment, or `model_name` omits the publisher prefix.
Upstream configuration error before any request is sent	`api_base` is empty on the provider key.

Set the Generated-Token Limit

This is the one request-shape difference most likely to affect a DeepInfra alias, and it is a direct consequence of the community-catalog path.

DeepInfra documents the generated-token cap as max_tokens in Chat Completions. Current OpenAI clients send max_completion_tokens instead. Several curated providers in the AISIX catalog carry a rename that converts the current name to the legacy one before the request leaves the gateway. deepinfra has no catalog entry, so no rename is registered for it and AISIX forwards whichever name the caller sent, unchanged.

The practical result:

A caller that sends max_tokens reaches DeepInfra with the name DeepInfra documents, and the cap applies.
A caller that sends max_completion_tokens reaches DeepInfra with a name DeepInfra does not document, and the cap may be ignored, letting a response run to the model's own output limit.

If your clients send the current OpenAI name, register the rename yourself on the provider key:

{
  "request": {
    "param_renames": {
      "max_completion_tokens": "max_tokens"
    }
  }
}

The rename applies to every model that references the provider key. If a request carries both names, AISIX uses the value from the original caller-facing name. See Provider-Specific Overrides.

Control Reasoning Output

DeepInfra accepts the standard OpenAI reasoning_effort parameter at the top level of the chat-completions body, and also accepts a reasoning object with effort and enabled fields. Setting "enabled": false is equivalent to reasoning_effort: "none". AISIX does not strip unrecognized top-level parameters, so either form reaches the upstream unchanged:

{
  "model": "deepinfra-deepseek-prod",
  "messages": [
    {
      "role": "user",
      "content": "Plan a three-step migration."
    }
  ],
  "reasoning_effort": "low"
}

Accepted values vary by model — openai/gpt-oss-120b documents low, medium, and high, while a toggle-style reasoning model accepts only enable and disable — so confirm the values for your model in the DeepInfra reasoning documentation. Using these parameters against a non-reasoning model has no effect.

On the response side, DeepInfra returns the model's thinking in reasoning_content, which is already the canonical field AISIX preserves and normalizes to. No response override is needed for the models listed above.

Because no response rewrite is registered for deepinfra, that behavior is a property of the models you configure rather than a guarantee AISIX enforces. If you add a model that streams reasoning on a different delta path, set response.reasoning_field on the provider key so streaming clients still find it at delta.reasoning_content. Verify with a streaming request rather than a non-streaming one, because the override applies to the streaming delta path.

Use the Anthropic-Shaped or Native DeepInfra Surface

DeepInfra publishes three request surfaces on the same host:

DeepInfra surface	Path	Reachable through a `deepinfra` provider key
OpenAI-compatible	`/v1/openai/...`	Yes. This is the surface configured by this guide.
Anthropic Messages	`/anthropic/v1/messages`	No.
Native inference	`/v1/inference/{model}`	No.

The community catalog default assigns the openai adapter to every provider it covers, and a catalog provider key cannot override the adapter — adapter is accepted only when provider is byo. To speak DeepInfra's Anthropic-shaped surface natively, create a separate bring-your-own endpoint provider key with "adapter": "anthropic" and "api_base": "https://api.deepinfra.com/anthropic", and treat it as a distinct upstream with its own credential and aliases.

For most deployments the OpenAI-compatible surface is sufficient, because AISIX already accepts Anthropic-shaped client requests on /v1/messages and translates them onto the openai adapter. Reach for the BYO path only when you need DeepInfra's own Anthropic implementation rather than the gateway's translation.

Endpoint Coverage

DeepInfra is an inference-only upstream, so only part of the proxy surface applies to a DeepInfra-backed alias.

Route	Behavior with a DeepInfra alias
`/v1/chat/completions`	Supported, including `stream: true`.
`/v1/responses`	Supported through the Responses bridge over the chat adapter path.
`/v1/messages`	Supported for Anthropic-shaped callers through translation. Token counting at `/v1/messages/count_tokens` requires an Anthropic-backed model.
`/v1/embeddings`	Supported. DeepInfra serves embedding models on the same OpenAI-compatible root, so one provider key covers both chat and embedding aliases. Create a separate model alias whose `model_name` is an embedding model ID. See Embeddings.
`/v1/images/generations`	Rejected. The route accepts only models whose provider is `openai`.
`/v1/rerank`	Rejected. The route accepts only the `openai`, `cohere`, and `jina` provider values.
`/v1/videos`	Rejected. The route accepts only its own provider allowlist, which does not include `deepinfra`.
`/passthrough/deepinfra/*`	Supported, with the path arithmetic described below.

note

A DeepInfra model ID that begins with openai/, such as openai/gpt-oss-120b, does not make the alias an OpenAI-provider model. The provider value is deepinfra, so routes that gate on provider identity rather than adapter — image generation, video generation, and rerank — reject the alias. See Provider Compatibility for the full route matrix.

On the passthrough route, AISIX appends the wildcard remainder to the provider key's api_base verbatim. It removes a duplicated leading path segment only when that segment is API-version shaped, such as v1. The DeepInfra base URL ends in openai, which is not version-shaped, so no deduplication applies and the arithmetic is a plain join:

/passthrough/deepinfra/models resolves to https://api.deepinfra.com/v1/openai/models.
/passthrough/deepinfra/v1/inference/deepseek-ai/DeepSeek-V3.2 resolves to https://api.deepinfra.com/v1/openai/v1/inference/..., which is not a DeepInfra route.

DeepInfra's native and Anthropic-shaped surfaces sit outside the configured root, so they are not reachable through a provider key whose api_base is the OpenAI-compatible root. Passthrough also requires the caller API key to be allowlisted for at least one model whose provider is deepinfra. See Provider Passthrough.

Next Steps

You have now connected AISIX to DeepInfra and verified the model alias. Continue with these guides:

Provider-Specific Overrides: register the parameter rename and any response mapping this community-catalog provider does not ship by default.
Model Aliases: add routing, cost metadata, and rate limits for this alias.
Routing and Failover: fail over between DeepInfra and a second provider that serves the same open-weight model.
Provider Compatibility: review supported proxy endpoints and provider-specific boundaries.

Prerequisites​

Configure the DeepInfra Upstream​

Create a Provider Key​

Why the Base URL Carries Both /v1 and /openai​

Create a Model​

Create a Caller API Key​

Verify the Provider Connection​

Set the Generated-Token Limit​

Control Reasoning Output​

Use the Anthropic-Shaped or Native DeepInfra Surface​

Endpoint Coverage​

Next Steps​