Skip to main content
API7.ai Logo

Hugging Face

Connect AISIX AI Gateway to Hugging Face Inference Providers so applications can call open-weight models through the gateway's OpenAI-compatible API. AISIX keeps the Hugging Face credential on the gateway side. It authorizes model access with caller API keys and allowlists, and applies the same rate-limit and usage-accounting controls as other model aliases.

Hugging Face exposes an OpenAI-compatible API, so it uses the openai adapter with the Inference Providers router as the api_base. Unlike a single-vendor upstream, the router is a routing layer: the model ID in the request, not the URL, selects which inference provider actually serves the model.

Prerequisites​

The following examples configure the upstream through the AISIX Cloud Admin API. Before configuring the upstream, prepare the following:

  • A running AISIX stack with an environment and an attached gateway. Follow the AISIX On-Premises Quickstart to set up the stack, create an admin token with write scope, and capture the environment ID.
  • A Hugging Face access token with the Make calls to Inference Providers permission, created in Access Tokens.

A Hugging Face access token is an account-level credential rather than a per-vendor API key. One token authorizes calls to every inference provider the router can select, so a single provider key in AISIX covers the whole router catalog. Scope the token to Inference Providers only, so that the credential stored in AISIX cannot also read or write Hub repositories.

Export the connection details used throughout this page:

export AISIX_CP="http://localhost:8080/api"
export AISIX_TOKEN="aisix_pat_YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Configure the Hugging Face Upstream​

Create a provider key, model alias, and caller API key for the Hugging Face-backed chat-completions route.

Create a Provider Key​

Create the provider key that stores the Hugging Face token and the router root:

# Replace with your value
export HF_TOKEN="YOUR_PROVIDER_API_KEY"

PROVIDER_KEY_ID=$(curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "huggingface-prod",
    "provider": "huggingface",
    "api_key": "'"${HF_TOKEN}"'",
    "api_base": "https://router.huggingface.co/v1",
    "allowed_environments": ["'"${ENV_ID}"'"]
  }' | jq -r '.provider_key.id')

echo "$PROVIDER_KEY_ID"

❶ provider is huggingface. The Cloud Admin API derives the adapter from the catalog provider; the adapter field is only accepted on BYO provider keys.

❷ api_key stores the Hugging Face access token. It follows the credential-handling behavior in Provider Credentials.

❸ api_base is the Inference Providers router root. The host is router.huggingface.co rather than a per-model host, and /v1 is the OpenAI-compatible surface that sits in front of every routed inference provider. AISIX appends the endpoint path, so the chat route resolves to https://router.huggingface.co/v1/chat/completions. If you omit api_base, AISIX Cloud fills in the same router URL from the catalog; setting it explicitly keeps the resource self-describing.

The command captures the returned provider key ID in PROVIDER_KEY_ID.

note

This page covers the shared Inference Providers router. A dedicated Hugging Face Inference Endpoint has its own per-deployment hostname and is not reachable through the router URL. Configure that deployment as a private OpenAI-compatible endpoint instead.

Create a Model​

Hugging Face model IDs are Hub repository IDs in the form <org>/<model>, and the casing is not uniform across publishers: openai/gpt-oss-120b is entirely lowercase, while Qwen/Qwen3-235B-A22B-Thinking-2507 and deepseek-ai/DeepSeek-V4-Pro use mixed case. Copy the ID verbatim from the supported models list rather than retyping it, and confirm there that the model is currently served before creating a model alias.

Create the model alias callers will send in requests:

MODEL_ID=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "hf-gptoss-prod",
    "model_name": "openai/gpt-oss-120b",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }' | jq -r '.model.id')

echo "$MODEL_ID"

❶ display_name is the alias callers send in model.

❷ model_name is the Hugging Face model ID, for example openai/gpt-oss-120b or deepseek-ai/DeepSeek-V4-Pro. AISIX forwards this string to the router unchanged.

❸ provider_key_id attaches the alias to the Hugging Face provider key.

Pin the Serving Inference Provider​

Hugging Face accepts an optional suffix on the model ID that controls which inference provider serves the request. The suffix is part of the model string, so it belongs in model_name:

model_name valueRouting behavior
openai/gpt-oss-120bAutomatic routing, which by default selects the fastest available inference provider.
openai/gpt-oss-120b:groqPinned to the named inference provider.
openai/gpt-oss-120b:cheapestRouted to the most cost-efficient provider by price per output token.
openai/gpt-oss-120b:fastestRouted to the highest-throughput provider. This is the default policy.
openai/gpt-oss-120b:preferredRouted by the preference order configured in your Hugging Face Inference Providers settings.

See the Hugging Face chat completion documentation for the current suffix syntax.

The suffix changes latency, price, and which backend actually runs the model, so treat pinned and auto-routed variants as different upstreams. Create one model alias per variant rather than switching the suffix in place, so that per-alias cost metadata, rate limits, and usage records stay meaningful. See Model Aliases for the alias fields involved.

Create a Caller API Key​

Create the caller API key that can access the model alias. The plaintext key is server-generated and returned once in the response — store it securely:

AISIX_API_KEY=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "huggingface-caller",
    "allowed_models": ["'"${MODEL_ID}"'"]
  }' | jq -r '.plaintext')

echo "$AISIX_API_KEY"

The allowed_models value must reference the model ID captured in the previous step.

The gateway picks up the new resources automatically — no restart is needed.

Verify the Provider Connection​

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "http://127.0.0.1:3000/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hf-gptoss-prod",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from Hugging Face."
      }
    ]
  }'

The gateway returns an OpenAI-compatible response that echoes the caller-facing alias hf-gptoss-prod. If the request fails, check the provider key api_key, api_base, the exact capitalization of the Hub repository ID in model_name, and whether the token carries the Inference Providers permission. Router availability is per model rather than per account, so also confirm on the supported models list that the model is currently served.

Control Reasoning Output​

Reasoning models on the router accept a top-level reasoning_effort field in the chat-completions request body:

{
  "reasoning_effort": "low"
}

Common values are none, minimal, low, medium, high, and xhigh, but support, defaults, and which values are meaningful depend on both the model and the inference provider that serves it. The router documents no shared enable or disable toggle and no numeric reasoning-budget field, so confirm the values for your model in the Hugging Face chat completion documentation before relying on a specific setting. AISIX forwards the field to the router without interpreting it.

The AISIX catalog entry for huggingface carries no response.reasoning_field override, so AISIX reads streamed reasoning from the canonical delta.reasoning_content path and preserves it in the response. Because the same Hub repository ID can be served by different inference providers, a pinned provider may stream reasoning under a different delta path. In that case, set response.reasoning_field on a provider key dedicated to that pinned alias, rather than on the shared router provider key that every other alias inherits.

Supported Proxy Routes​

The Hugging Face provider value resolves to the openai adapter, which determines the routes a huggingface-backed alias can use:

  • Chat completions. /v1/chat/completions is the primary route, with streaming supported.
  • Responses. /v1/responses is served through the AISIX Responses bridge, because verbatim Responses forwarding applies only to models whose provider is openai. The bridged request reaches the router's chat-completions route.
  • Embeddings. Not available on the router. Hugging Face documents the OpenAI-compatible /v1 surface as serving chat completion only, and directs embedding workloads to the task-specific inference clients or to a dedicated deployment. AISIX would forward /v1/embeddings to https://router.huggingface.co/v1/embeddings, but the router does not serve that route. To put embeddings behind the gateway, deploy the model as a Hugging Face Inference Endpoint and configure it as a private OpenAI-compatible endpoint.
  • Provider passthrough. /passthrough/huggingface/<path> forwards the request verbatim with the Hugging Face token substituted for the caller key. Because api_base already ends in /v1, AISIX removes a duplicate leading v1 segment, so both /passthrough/huggingface/models and /passthrough/huggingface/v1/models reach https://router.huggingface.co/v1/models. See Provider Passthrough.
  • Not available. Image generation and video generation are restricted to other provider values, and the rerank route accepts only the openai, cohere, and jina provider values. A huggingface-backed alias is rejected on all three.

Next Steps​

You have now connected AISIX to Hugging Face and verified the model alias. Continue with these guides: