Skip to main content
API7.ai Logo

Hugging Face

Hugging Face Inference Providers routes requests to open-weight models served by multiple inference vendors. AISIX puts that catalog behind one OpenAI-compatible API with gateway-managed credentials, caller access, rate limits, and usage accounting.

Unlike a single-vendor upstream, Hugging Face uses a routing layer: the model ID in the request, rather than the URL, selects which inference provider serves the model.

This guide covers the shared Inference Providers router. A dedicated Hugging Face Inference Endpoint has its own per-deployment hostname and is not reachable through the router URL. Configure that deployment as a private OpenAI-compatible endpoint only when its serving engine exposes the OpenAI route your application needs, such as /v1/chat/completions or /v1/embeddings. A custom or task-native endpoint is not automatically OpenAI-compatible.

Prerequisites​

Before starting, prepare the following:

  • One AISIX setup:
    • For AISIX Cloud, an environment with an attached gateway and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
    • For the open-source AISIX gateway, prepare either a local AISIX installation or the Docker setup from the Open-Source AISIX Gateway Quickstart. Configure the gateway to load a declarative resources file.
  • A Hugging Face access token with the Make calls to Inference Providers permission, created in Access Tokens.
  • A Hugging Face account with remaining Inference Providers credits.
  • curl and jq.

A Hugging Face access token is an account-level credential rather than a per-vendor API key. One token authorizes calls to every inference provider the router can select, so a single provider key in AISIX covers the whole router catalog. Scope the token to Inference Providers only, so that the credential stored in AISIX cannot also read or write Hub repositories.

Configure with AISIX Cloud​

Export the AISIX Cloud connection details:

# AISIX_CP is the Admin API base URL; include /api and omit a trailing slash
# The local On-Premises quickstart uses http://localhost:8080/api
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Create a provider key, model alias, and caller API key for the Hugging Face-backed chat-completions route.

Because Hugging Face exposes an OpenAI-compatible API, AISIX connects through the openai adapter and uses the Inference Providers router as api_base.

Create a Provider Key​

Create the provider key that stores the Hugging Face token and the router root:

# Replace with your value
export HF_TOKEN="YOUR_PROVIDER_API_KEY"

PROVIDER_KEY_ID=$(curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "huggingface-prod",
    "provider": "huggingface",
    "api_key": "'"${HF_TOKEN}"'",
    "api_base": "https://router.huggingface.co/v1",
    "allowed_environments": ["'"${ENV_ID}"'"]
  }' | jq -r '.provider_key.id')

echo "$PROVIDER_KEY_ID"

❶ provider is huggingface. The AISIX Cloud Admin API derives the adapter from the catalog provider; the adapter field is only accepted on BYO provider keys.

❷ api_key stores the Hugging Face access token. It follows the credential-handling behavior in Provider Keys.

❸ api_base is the Inference Providers router root. The host is router.huggingface.co rather than a per-model host, and /v1 is the OpenAI-compatible surface that sits in front of every routed inference provider. AISIX appends the endpoint path, so the chat route resolves to https://router.huggingface.co/v1/chat/completions. If you omit api_base, AISIX Cloud fills in the same router URL from the catalog; setting it explicitly keeps the resource self-describing.

The command captures the returned provider key ID in PROVIDER_KEY_ID.

Create a Model​

Hugging Face model IDs are Hub repository IDs in the form <org>/<model>, and the casing is not uniform across publishers: openai/gpt-oss-120b is entirely lowercase, while Qwen/Qwen3-235B-A22B-Thinking-2507 and deepseek-ai/DeepSeek-V4-Pro use mixed case. Copy the ID verbatim from the supported models list rather than retyping it, and confirm there that the model is currently served before creating a model alias.

Create the model alias callers will send in requests:

MODEL_ID=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "hf-gptoss-prod",
    "model_name": "openai/gpt-oss-120b",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }' | jq -r '.model.id')

echo "$MODEL_ID"

❶ display_name is the alias callers send in model.

❷ model_name is the Hugging Face model ID, for example openai/gpt-oss-120b or deepseek-ai/DeepSeek-V4-Pro. AISIX forwards this string to the router unchanged.

❸ provider_key_id attaches the alias to the Hugging Face provider key.

Pin the Serving Inference Provider​

Hugging Face accepts an optional suffix on the model ID that controls which inference provider serves the request. The suffix is part of the model string, so it belongs in model_name:

model_name valueRouting behavior
openai/gpt-oss-120bAutomatic routing, which by default selects the fastest available inference provider.
openai/gpt-oss-120b:groqPinned to the named inference provider.
openai/gpt-oss-120b:cheapestRouted to the most cost-efficient provider by price per output token.
openai/gpt-oss-120b:fastestRouted to the highest-throughput provider. This is the default policy.
openai/gpt-oss-120b:preferredRouted by the preference order configured in your Hugging Face Inference Providers settings.

See the Hugging Face chat completion documentation for the current suffix syntax.

The suffix changes latency, price, and which backend actually runs the model, so treat pinned and policy-routed variants as different upstreams. Create one model alias per variant rather than switching the suffix in place, so rate limits and usage records remain attributable to that routing choice.

Pin the inference provider when accurate AISIX cost and budget calculations matter. With :fastest, :cheapest, or :preferred, the selected provider and price can change, so verify or override the alias's cost metadata. See Model Aliases for the alias fields involved.

Create a Caller API Key​

Create the caller API key that can access the model alias. The plaintext key is server-generated and returned once in the response — store it securely:

AISIX_API_KEY=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "huggingface-caller",
    "allowed_models": ["'"${MODEL_ID}"'"]
  }' | jq -r '.plaintext')

echo "$AISIX_API_KEY"

The allowed_models value must reference the model ID captured in the previous step.

The gateway picks up the new resources automatically — no restart is needed.

Configure with the Open-Source AISIX Gateway​

Export the upstream credential and choose the caller API key that applications will send to the gateway:

export HF_TOKEN="YOUR_PROVIDER_API_KEY"
export CALLER_API_KEY="YOUR_CALLER_API_KEY"

Create a complete declarative resources file for this provider:

resources.yaml
_format_version: "1"

provider_keys:
  - display_name: "huggingface-prod"
    provider: "huggingface"
    adapter: "openai"
    api_key: ${HF_TOKEN}
    api_base: "https://router.huggingface.co/v1"

models:
  - display_name: "hf-gptoss-prod"
    provider: "huggingface"
    model_name: "openai/gpt-oss-120b"
    provider_key: "huggingface-prod"

api_keys:
  - display_name: "huggingface-caller"
    key_env: CALLER_API_KEY
    allowed_models:
      - "hf-gptoss-prod"

If AISIX is installed locally, validate the file before loading it:

aisix validate --resources resources.yaml

After validation, start the gateway with the referenced environment variables in its process environment. Reload an existing gateway only if those variables are already available to the process; otherwise, restart it with the updated environment.

If you use Docker, adapt the validation and startup commands in the Open-Source AISIX Gateway Quickstart. Mount this resources.yaml file and pass every environment variable it references with -e in both commands. After the resources load, prepare the shared verification request below:

export AISIX_API_KEY="$CALLER_API_KEY"

Verify the Provider Connection​

Export the AISIX gateway origin:

# The local quickstarts use http://127.0.0.1:3000
export AISIX_PROXY="YOUR_AISIX_GATEWAY_ORIGIN"

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "hf-gptoss-prod",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from Hugging Face."
      }
    ]
  }'

The gateway returns an OpenAI-compatible response that echoes the caller-facing alias hf-gptoss-prod. If the request fails, check the provider key api_key and api_base. Verify the exact capitalization of the Hub repository ID in model_name. Ensure that the token carries the Inference Providers permission and that the account has remaining Inference Providers credits. Provider availability also varies by model, so confirm on the supported models list that the model is currently served.

Control Reasoning Output​

Reasoning models on the router accept a top-level reasoning_effort field in the chat-completions request body:

{
  "reasoning_effort": "low"
}

Common values are none, minimal, low, medium, high, and xhigh. Support, defaults, and which values are meaningful depend on both the model and its serving inference provider. The router documents no shared toggle or numeric reasoning budget, so confirm your model's values in the Hugging Face chat completion documentation. AISIX forwards the field without interpreting it.

The AISIX catalog entry for huggingface carries no response.reasoning_field override. AISIX recognizes both delta.reasoning_content and delta.reasoning, normalizing the latter to reasoning_content. Because the same Hub repository ID can be served by different inference providers, a pinned provider may stream reasoning under another delta path. In that case, set response.reasoning_field on a provider key dedicated to that pinned alias, rather than on the shared router provider key that every other alias inherits.

Supported Proxy Routes​

The Hugging Face provider value resolves to the openai adapter, but the shared router implements only part of the broader OpenAI API. Provider and model capabilities can also differ behind the router.

RouteBehavior with a huggingface provider key
/v1/chat/completionsSupported, including streaming. AISIX forwards function tools, response_format, reasoning_effort, and VLM image_url content. Actual tool use, structured output, reasoning, and vision support depend on both the Hub model and the inference provider selected for that request.
/v1/responsesSupported through the AISIX Responses bridge, not Hugging Face's native Responses API. The bridge carries text, local function calls and results, sampling fields, and streaming through chat completions. It drops non-text image and file input parts, hosted tools such as remote MCP, reasoning and structured-output text configuration, and state fields such as previous_response_id and store. It also omits reasoning output when re-encoding the chat result.
/v1/messagesSupported through translation to chat completions, not a native Hugging Face Messages API. /v1/messages/count_tokens is unavailable because the configured provider is not Anthropic.
/v1/embeddingsNot available through the router's OpenAI-compatible /v1 surface. Hugging Face provides embeddings through its task-specific feature-extraction interface, but AISIX does not translate the OpenAI embeddings body to that interface. A dedicated Inference Endpoint works only if its serving engine exposes /v1/embeddings, such as Text Embeddings Inference.
/v1/completions, /v1/audio/*, /v1/files, /v1/batches, and /v1/fine_tuning/jobsNot available on the configured shared router root. Hugging Face has separate task-native interfaces for capabilities such as text generation and speech, but these AISIX routes do not translate to or reach those interfaces.
/v1/images/generations, /v1/videos, and /v1/rerankNot supported for a huggingface provider value. Hugging Face may offer related native tasks, but these AISIX routes reject this provider before forwarding.

Normalized chat responses preserve content, local function calls, normalized reasoning, and basic token usage. They do not preserve every router or serving-provider field, such as created, system_fingerprint, choice indexes and log probabilities, or arbitrary provider-specific metadata.

To use Hugging Face's native Responses wire shape, send POST /passthrough/huggingface/responses with the exact Hugging Face model ID in model; passthrough does not rewrite an AISIX alias. AISIX relays the response body without normalization unless a guardrail blocks it. It detects the Responses envelope from input and records token usage when the response includes supported input_tokens and output_tokens fields.

The /passthrough/huggingface paths on this page assume a passthrough route claiming that prefix with the configured https://router.huggingface.co/v1 root as its target_url; grant the route on the caller key's allowed_routes. Such a route can reach /v1/responses and /v1/models, but it cannot escape that root to sibling task-native paths such as /hf-inference/models/<model>. Because the target already ends in /v1, AISIX removes a duplicate leading v1 segment, so both /passthrough/huggingface/models and /passthrough/huggingface/v1/models reach https://router.huggingface.co/v1/models.

Next Steps​

You have now connected AISIX to Hugging Face and verified the model alias. Continue with these guides: