DeepInfra

DeepInfra provides hosted inference for open-weight models from many publishers. Applications call those models by stable AISIX aliases without receiving the upstream API token.

Prerequisites

Before starting, prepare the following:

One AISIX setup:
- For AISIX Cloud, an environment with an attached gateway and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
- For the open-source AISIX gateway, prepare either a local AISIX installation or the Docker setup from the Open-Source AISIX Gateway Quickstart. Configure the gateway to load a declarative resources file.
A DeepInfra API token from the DeepInfra dashboard.
curl and jq.

Configure with AISIX Cloud

Export the AISIX Cloud connection details:

# AISIX_CP is the Admin API base URL; include /api and omit a trailing slash
# The local On-Premises quickstart uses http://localhost:8080/api
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Create a provider key, model alias, and caller API key for the DeepInfra-backed chat-completions route.

DeepInfra is a community catalog provider with an OpenAI-compatible API. AISIX connects through the openai adapter and uses the DeepInfra API root as api_base. AISIX does not supply a curated base URL or provider-specific request and response rewrites.

The AISIX Cloud Admin API returns community_badge: true for DeepInfra. The dashboard groups it under All providers (community) and identifies the wire compatibility as assumed rather than verified.

Create a Provider Key

Create the provider key that stores the DeepInfra credential and API root:

# Replace with your value
export DEEPINFRA_API_KEY="YOUR_PROVIDER_API_KEY"

PROVIDER_KEY_ID=$(curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "deepinfra-prod",
    "provider": "deepinfra",
    "api_key": "'"${DEEPINFRA_API_KEY}"'",
    "api_base": "https://api.deepinfra.com/v1",
    "allowed_environments": ["'"${ENV_ID}"'"]
  }' | jq -r '.provider_key.id')

echo "$PROVIDER_KEY_ID"

❶ provider is deepinfra. The AISIX Cloud Admin API derives the adapter from the catalog entry; the adapter field is only accepted on BYO provider keys, and sending it on a catalog key returns a 400 error. For deepinfra the derived adapter is openai.

❷ api_key stores the DeepInfra API token. DeepInfra authenticates the OpenAI-compatible surface with HTTP bearer authentication, which is what the openai adapter already sends. The value follows the credential-handling behavior in Provider Keys.

❸ api_base is required for DeepInfra. models.dev publishes no api field for the deepinfra entry, so there is no cached default for the AISIX Cloud Admin API to fall back on. Omitting api_base returns a 400 error stating that models.dev does not publish a default base URL for this provider.

The command captures the returned provider key ID in PROVIDER_KEY_ID.

Why the Base URL Stops at `/v1`

DeepInfra's OpenAI SDK examples set base_url to https://api.deepinfra.com/v1/openai. DeepInfra also publishes the same API directly under /v1, including /v1/chat/completions, /v1/embeddings, /v1/images/generations, and /v1/audio/*.

AISIX appends the endpoint path to api_base, so the direct https://api.deepinfra.com/v1 root is the more useful configuration. One provider key can then serve chat, embeddings, and audio aliases, while a passthrough route can reach the compatible image endpoint and DeepInfra-native routes.

Two related behaviors are worth knowing:

If you paste the full direct endpoint URL, AISIX strips a known endpoint suffix such as /chat/completions along with any trailing slash before building the upstream URL. Setting api_base to https://api.deepinfra.com/v1/chat/completions therefore still works, but prefer the root form so the value stays readable.
AISIX does not synthesize a missing path segment for a non-OpenAI vendor. Setting api_base to the bare host https://api.deepinfra.com produces the upstream URL https://api.deepinfra.com/chat/completions, which DeepInfra does not serve. Use the root DeepInfra documents rather than relying on a shorter form to be repaired. Leaving api_base empty is not a workaround either: the gateway refuses to fall back to the OpenAI host for a non-OpenAI provider and returns an upstream configuration error instead of leaking the DeepInfra token to another vendor.

Create a Model

DeepInfra model IDs are namespaced by the weights publisher, in the form <publisher>/<Model-Name>, matching the Hugging Face repository ID for the same weights. The casing is not uniform, and the publisher segment is the Hub organization rather than the vendor's brand name — for example the GLM models are published under zai-org, not zhipuai. Copy the ID verbatim from the DeepInfra model catalog rather than retyping it.

Current IDs include:

Model ID	Notes
`deepseek-ai/DeepSeek-V3.2`	Reasoning model; returns reasoning in `reasoning_content`.
`openai/gpt-oss-120b`	Open-weight model; accepts `reasoning_effort` values `low`, `medium`, and `high`.
`meta-llama/Llama-3.3-70B-Instruct-Turbo`	Non-reasoning instruction model.

Create the model alias callers will send in requests:

MODEL_ID=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "deepinfra-deepseek-prod",
    "model_name": "deepseek-ai/DeepSeek-V3.2",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }' | jq -r '.model.id')

echo "$MODEL_ID"

❶ display_name is the alias callers send in model.

❷ model_name is the full DeepInfra model ID, including the publisher segment. A bare identifier such as DeepSeek-V3.2 is rejected by the upstream.

❸ provider_key_id attaches the alias to the DeepInfra provider key.

Create a Caller API Key

Create the caller API key that can access the model alias. The plaintext key is server-generated and returned once in the create response, so capture it now:

AISIX_API_KEY=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "deepinfra-caller",
    "allowed_models": ["'"${MODEL_ID}"'"]
  }' | jq -r '.plaintext')

echo "$AISIX_API_KEY"

The allowed_models value must reference the model ID captured in the previous step. After the write, the configuration projects to attached gateways automatically.

Configure with the Open-Source AISIX Gateway

Export the upstream credential and choose the caller API key that applications will send to the gateway:

export DEEPINFRA_API_KEY="YOUR_PROVIDER_API_KEY"
export CALLER_API_KEY="YOUR_CALLER_API_KEY"

Create a complete declarative resources file for this provider:

resources.yaml
_format_version: "1"

provider_keys:
  - display_name: "deepinfra-prod"
    provider: "deepinfra"
    adapter: "openai"
    api_key: ${DEEPINFRA_API_KEY}
    api_base: "https://api.deepinfra.com/v1"

models:
  - display_name: "deepinfra-deepseek-prod"
    provider: "deepinfra"
    model_name: "deepseek-ai/DeepSeek-V3.2"
    provider_key: "deepinfra-prod"

api_keys:
  - display_name: "deepinfra-caller"
    key_env: CALLER_API_KEY
    allowed_models:
      - "deepinfra-deepseek-prod"

If AISIX is installed locally, validate the file before loading it:

aisix validate --resources resources.yaml

After validation, start the gateway with the referenced environment variables in its process environment. Reload an existing gateway only if those variables are already available to the process; otherwise, restart it with the updated environment.

If you use Docker, adapt the validation and startup commands in the Open-Source AISIX Gateway Quickstart. Mount this resources.yaml file and pass every environment variable it references with -e in both commands. After the resources load, prepare the shared verification request below:

export AISIX_API_KEY="$CALLER_API_KEY"

Verify the Provider Connection

Export the AISIX gateway origin:

# The local quickstarts use http://127.0.0.1:3000
export AISIX_PROXY="YOUR_AISIX_GATEWAY_ORIGIN"

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "deepinfra-deepseek-prod",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from DeepInfra."
      }
    ]
  }'

The gateway returns an OpenAI-compatible response that echoes the caller-facing alias deepinfra-deepseek-prod.

If the request fails, isolate the cause by symptom:

Symptom	Likely cause
Upstream authentication error	The DeepInfra token in `api_key` is wrong or revoked.
Upstream 404	`api_base` does not point to a DeepInfra root that serves chat completions, or `model_name` omits the publisher prefix.
Upstream configuration error before any request is sent	`api_base` is empty on the provider key.

Set the Generated-Token Limit

This is the one request-shape difference most likely to affect a DeepInfra alias, and it is a direct consequence of the community-catalog path.

DeepInfra documents the generated-token cap as max_tokens in Chat Completions. Some OpenAI clients and integrations use the newer max_completion_tokens name instead. Several curated providers in the AISIX catalog carry a rename that converts the newer name to the legacy one before the request leaves the gateway. deepinfra has no AISIX-curated adapter mapping, so no rename is registered for it and AISIX forwards whichever name the caller sent, unchanged.

The practical result:

A caller that sends max_tokens reaches DeepInfra with the name DeepInfra documents, and the cap applies.
A caller that sends max_completion_tokens reaches DeepInfra with a name DeepInfra does not document. The upstream behavior is therefore not guaranteed: the request may fail or the cap may not apply.

If your clients send the current OpenAI name, register the rename yourself on the provider key:

{
  "request": {
    "param_renames": {
      "max_completion_tokens": "max_tokens"
    }
  }
}

The rename applies to every model that references the provider key. If a request carries both names, AISIX uses the value from the original caller-facing name. See Provider-Specific Overrides.

Control Reasoning Output

DeepInfra accepts the standard OpenAI reasoning_effort parameter at the top level of the chat-completions body, and also accepts a reasoning object with effort and enabled fields. Setting "enabled": false is equivalent to reasoning_effort: "none". AISIX does not strip unrecognized top-level parameters, so either form reaches the upstream unchanged:

{
  "model": "deepinfra-deepseek-prod",
  "messages": [
    {
      "role": "user",
      "content": "Plan a three-step migration."
    }
  ],
  "reasoning_effort": "low"
}

DeepInfra documents none, low, medium, and high for supported reasoning models. Model availability and behavior can still differ, so confirm the controls for your model in the DeepInfra reasoning documentation and its catalog page. Using these parameters against a non-reasoning model has no effect.

On the response side, DeepInfra returns the model's thinking in reasoning_content, which is already the canonical field AISIX preserves and normalizes to. No response override is needed for the models listed above.

Because no response rewrite is registered for deepinfra, that behavior is a property of the models you configure rather than a guarantee AISIX enforces. If you add a model that streams reasoning on a different delta path, set response.reasoning_field on the provider key so streaming clients still find it at delta.reasoning_content. Verify with a streaming request rather than a non-streaming one, because the override applies to the streaming delta path.

Use the Anthropic-Shaped or Native DeepInfra Surface

DeepInfra publishes several request surfaces on the same host:

DeepInfra surface	Path	Reachable through the provider key configured in this guide
Direct OpenAI-compatible	`/v1/chat/completions`, `/v1/embeddings`, `/v1/images/generations`, and `/v1/audio/...`	Yes. Normalized AISIX routes cover chat, embeddings, and audio; image generation requires a passthrough route.
OpenAI SDK compatibility root	`/v1/openai/...`	Not used by the normalized routes in this guide. DeepInfra publishes it as an equivalent base for OpenAI SDK clients.
Anthropic Messages	`/anthropic/v1/messages`	No.
Native inference	`/v1/inference/{model}`	Yes, through a passthrough route.

The community catalog default assigns the openai adapter to every provider it covers, and a catalog provider key cannot override the adapter — adapter is accepted only when provider is byo. To speak DeepInfra's Anthropic-shaped surface natively, create a separate bring-your-own endpoint provider key with "adapter": "anthropic" and "api_base": "https://api.deepinfra.com/anthropic", and treat it as a distinct upstream with its own credential and aliases.

For most deployments the OpenAI-compatible surface is sufficient, because AISIX already accepts Anthropic-shaped client requests on /v1/messages and translates them onto the openai adapter. Reach for the BYO path only when you need DeepInfra's own Anthropic implementation rather than the gateway's translation.

Endpoint Coverage

DeepInfra serves several inference modalities, but normalized AISIX route support depends on both the upstream base URL and the model's deepinfra provider value.

Route	Behavior with a DeepInfra alias
`/v1/chat/completions`	Supported, including `stream: true`.
`/v1/responses`	Supported through the Responses bridge over the chat adapter path. OpenAI-specific Responses fields without a chat equivalent are ignored.
`/v1/messages`	Supported for Anthropic-shaped callers through translation. Token counting at `/v1/messages/count_tokens` requires an Anthropic-backed model.
`/v1/embeddings`	Supported. DeepInfra serves embedding models on the same OpenAI-compatible root, so one provider key covers both chat and embedding aliases. Create a separate model alias whose `model_name` is an embedding model ID. See Embeddings.
`/v1/audio/transcriptions`, `/v1/audio/translations`, `/v1/audio/speech`	Supported with the provider key configured in this guide when the alias names a compatible DeepInfra audio model. See DeepInfra's audio API reference and Speech and Audio.
`/v1/images/generations`	Rejected. The route accepts only models whose provider is `openai`. DeepInfra's compatible image endpoint is reachable at `/passthrough/deepinfra/images/generations` through a passthrough route that injects the provider key created in this guide.
`/v1/rerank`	Rejected. The route accepts only the `openai`, `cohere`, and `jina` provider values.
`/v1/videos`	Rejected. The route accepts only its own provider allowlist, which does not include `deepinfra`.
`/passthrough/deepinfra/*`	Available through a configured passthrough route, with the path arithmetic described below.

A DeepInfra model ID that begins with openai/, such as openai/gpt-oss-120b, does not make the alias an OpenAI-provider model. The provider value is deepinfra, so routes that gate on provider identity rather than adapter — image generation, video generation, and rerank — reject the alias. See Provider Compatibility for the full route matrix.

The /passthrough/deepinfra paths on this page assume a passthrough route claiming that prefix with https://api.deepinfra.com/v1 as its target_url and the DeepInfra provider key attached; grant the route on the caller key's allowed_routes.

On a prefix-matched passthrough route, AISIX appends the wildcard remainder to the route's target_url. If both the target and the wildcard path contain the same API-version segment, AISIX removes the duplicate:

/passthrough/deepinfra/models resolves to https://api.deepinfra.com/v1/models.
/passthrough/deepinfra/images/generations resolves to DeepInfra's OpenAI-compatible image endpoint. The request body must use a DeepInfra model ID rather than an AISIX alias when it includes model.
/passthrough/deepinfra/v1/inference/deepseek-ai/DeepSeek-V3.2 resolves to the DeepInfra-native inference endpoint. AISIX removes the duplicated v1 segment because the route's target_url already ends in /v1.

A passthrough route forwards provider-native request and response bodies without rewriting an AISIX alias. AISIX detects OpenAI-compatible chat, completions, and Responses envelopes from each request and records supported usage fields. Requests without a recognized carrier field remain opaque: buffered responses record zero tokens, while opaque SSE can still record top-level supported usage fields. The Anthropic-shaped surface still needs the separate BYO provider key described above because it is rooted at /anthropic, outside the /v1 target used here. See Passthrough Routes.

Next Steps

You have now connected AISIX to DeepInfra and verified the model alias. Continue with these guides:

Provider-Specific Overrides: register the parameter rename and any response mapping this community-catalog provider does not ship by default.
Model Aliases: configure routing, retry behavior, or cost metadata for this alias.
Routing and Failover: fail over between DeepInfra and a second provider that serves the same open-weight model.
Provider Compatibility: review supported proxy endpoints and provider-specific boundaries.

Prerequisites​

Configure with AISIX Cloud​

Create a Provider Key​

Why the Base URL Stops at /v1​

Create a Model​

Create a Caller API Key​

Configure with the Open-Source AISIX Gateway​

Verify the Provider Connection​

Set the Generated-Token Limit​

Control Reasoning Output​

Use the Anthropic-Shaped or Native DeepInfra Surface​

Endpoint Coverage​

Next Steps​