Skip to main content
API7.ai Logo

Nebius Token Factory

Nebius Token Factory provides hosted inference for a catalog of models. AISIX gives applications stable aliases while the gateway holds the Token Factory API key.

Prerequisites​

Before starting, prepare the following:

  • One AISIX setup:
    • For AISIX Cloud, an environment with an attached gateway and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
    • For the open-source AISIX gateway, prepare either a local AISIX installation or the Docker setup from the Open-Source AISIX Gateway Quickstart. Configure the gateway to load a declarative resources file.
  • A Nebius Token Factory API key.
  • curl and jq.

Configure with AISIX Cloud​

Export the AISIX Cloud connection details:

# AISIX_CP is the Admin API base URL; include /api and omit a trailing slash
# The local On-Premises quickstart uses http://localhost:8080/api
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Nebius is a community catalog provider with an OpenAI-compatible API. AISIX connects through the openai adapter and authenticates upstream requests with a bearer token.

Create a Provider Key​

export NEBIUS_API_KEY="YOUR_NEBIUS_API_KEY"

PROVIDER_KEY_ID=$(
  curl -sS -X POST "$AISIX_CP/provider_keys" \
    -H "Authorization: Bearer $AISIX_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
      "display_name": "nebius-prod",
      "provider": "nebius",
      "api_key": "'"${NEBIUS_API_KEY}"'",
      "api_base": "https://api.tokenfactory.nebius.com/v1",
      "allowed_environments": ["'"${ENV_ID}"'"]
    }' | jq -er '.provider_key.id'
)

echo "$PROVIDER_KEY_ID"

Do not add adapter to a catalog provider key. AISIX derives the openai adapter and bearer auth scheme from the catalog. The explicit API base makes the target visible in configuration even when the same value is available from the synchronized catalog.

Create a Model​

Nebius model IDs include the publisher namespace. Create an alias with the complete ID:

MODEL_ID=$(
  curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
    -H "Authorization: Bearer $AISIX_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
      "display_name": "nebius-llama-prod",
      "model_name": "meta-llama/Llama-3.3-70B-Instruct",
      "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
    }' | jq -er '.model.id'
)

echo "$MODEL_ID"

Copy another ID from the current Nebius model catalog rather than removing or changing the publisher prefix.

Create a Caller API Key​

AISIX_API_KEY=$(
  curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
    -H "Authorization: Bearer $AISIX_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
      "display_name": "nebius-caller",
      "allowed_models": ["'"${MODEL_ID}"'"]
    }' | jq -er '.plaintext'
)

echo "$AISIX_API_KEY"

Configure with the Open-Source AISIX Gateway​

Export the upstream credential and choose the caller API key that applications will send to the gateway:

export NEBIUS_API_KEY="YOUR_PROVIDER_API_KEY"
export CALLER_API_KEY="YOUR_CALLER_API_KEY"

Create a complete declarative resources file for this provider:

resources.yaml
_format_version: "1"

provider_keys:
  - display_name: "nebius-prod"
    provider: "nebius"
    adapter: "openai"
    api_key: ${NEBIUS_API_KEY}
    api_base: "https://api.tokenfactory.nebius.com/v1"

models:
  - display_name: "nebius-llama-prod"
    provider: "nebius"
    model_name: "meta-llama/Llama-3.3-70B-Instruct"
    provider_key: "nebius-prod"

api_keys:
  - display_name: "nebius-caller"
    key_env: CALLER_API_KEY
    allowed_models:
      - "nebius-llama-prod"

If AISIX is installed locally, validate the file before loading it:

aisix validate --resources resources.yaml

After validation, start the gateway with the referenced environment variables in its process environment. Reload an existing gateway only if those variables are already available to the process; otherwise, restart it with the updated environment.

If you use Docker, adapt the validation and startup commands in the Open-Source AISIX Gateway Quickstart. Mount this resources.yaml file and pass every environment variable it references with -e in both commands. After the resources load, prepare the shared verification request below:

export AISIX_API_KEY="$CALLER_API_KEY"

Verify the Provider Connection​

Export the AISIX gateway origin:

# The local quickstarts use http://127.0.0.1:3000
export AISIX_PROXY="YOUR_AISIX_GATEWAY_ORIGIN"

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer $AISIX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nebius-llama-prod",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from Nebius Token Factory."
      }
    ]
  }'

The upstream request uses POST /v1/chat/completions, the exact Nebius model ID, and Authorization: Bearer <NEBIUS_API_KEY>.

Review Model Capabilities​

Nebius Token Factory serves chat, reasoning, vision, embedding, rerank, and image models from the same API root. Capabilities still depend on the selected model. The meta-llama/Llama-3.3-70B-Instruct model in this guide is a current text-only chat model that supports function tools and structured output; it is not a reasoning or vision model.

On normalized chat-completions requests, AISIX forwards OpenAI-style tools, structured-output controls, typed image or video content blocks, and top-level fields such as reasoning_effort. For a reasoning-capable Nebius model, AISIX normalizes upstream reasoning_content or reasoning to reasoning_content in the returned assistant message. Nebius can return multiple choices when n is greater than 1, but AISIX returns only the first choice on normalized chat routes. Use a passthrough route when the application requires every choice or another provider-native response shape.

Endpoint Coverage​

RouteBehavior with a Nebius alias
/v1/chat/completionsSupported, including streaming, tools, structured output, reasoning, and multimodal content supported by the selected model.
/v1/completionsSupported through the openai adapter for a Nebius model that accepts the legacy completions contract.
/v1/responsesSupported through the Responses bridge, which translates to chat completions instead of calling Nebius's native Responses API. Fields without a chat equivalent, including state and native tool semantics, are ignored; Responses reasoning controls and Nebius reasoning output are not preserved. Use a passthrough route when native Responses semantics are required.
/v1/messagesSupported through Anthropic-to-chat translation, not a Nebius-native Messages API. The bridge does not preserve Nebius reasoning as Anthropic thinking blocks. /v1/messages/count_tokens requires an Anthropic-backed model and rejects this configuration.
/v1/embeddingsSupported with a separate alias for a current Nebius embedding model, such as Qwen/Qwen3-Embedding-8B.
/v1/filesSupported for upload, list, retrieve, delete, and content download. AISIX rewrites returned file IDs so normalized follow-up calls route to the same alias.
/v1/fine_tuning/jobsSupported for create, list, retrieve, and cancel. In a create request, model must be the upstream Nebius base-model ID rather than an AISIX alias.
/v1/batches and /v1/audio/*Not supported. Nebius does not publish compatible OpenAI Batch or audio routes on this API root.
/v1/images/generationsRejected with 400 because the normalized route requires provider: openai. Nebius's native image-generation route is available through a passthrough route.
/v1/rerankRejected with 400 because the normalized route accepts only the openai, cohere, and jina provider values. Nebius's native rerank route is available through a passthrough route.
/v1/videosRejected with 501 not_implemented. Nebius does not publish a video-generation route; video input to a compatible chat model is a separate capability.
/passthrough/nebius/*Available through a configured passthrough route for Nebius-native routes beneath the /v1 target, including /responses, /images/generations, /rerank, and /models.

The /passthrough/nebius paths on this page assume a passthrough route claiming that prefix with https://api.tokenfactory.nebius.com/v1 as its target_url and the Nebius provider key attached; grant the route on the caller key's allowed_routes.

Normalized file and fine-tuning responses use AISIX-routed IDs for the normalized follow-up routes above. Nebius-only routes, such as /files/{id}/link and fine-tuning /events or /checkpoints, require raw Nebius IDs through a passthrough route. The route does not decode an AISIX-routed ID, so create and manage the resource through the passthrough route from the start when the workflow needs those native-only operations.

A passthrough route does not rewrite an AISIX model alias or resource ID. Send the exact Nebius model ID in a native request body. Because the route's target_url ends in /v1, both /passthrough/nebius/responses and /passthrough/nebius/v1/responses reach https://api.tokenfactory.nebius.com/v1/responses; AISIX removes one duplicated version segment.

A passthrough route binds a fixed target and credential rather than borrowing them from a caller-accessible model alias, and model allowlists do not gate it; give each Nebius account or API root its own route. AISIX detects chat, completions, and Responses envelopes and records supported usage fields. Requests without a recognized carrier field remain opaque: buffered responses record zero tokens, while opaque SSE can still record top-level supported usage fields. The route relays provider responses and SSE incrementally unless a hold-back output guardrail is attached. File and fine-tuning management calls are zero-token operations, and AISIX does not include Nebius training charges in inference cost accounting. See Passthrough Routes.

Troubleshooting​

SymptomCheck
Upstream 401 or 403Confirm the Nebius API key and project access.
Upstream 404Confirm /v1 is present in api_base, and verify that the requested route and model are compatible.
Model not foundPreserve the publisher namespace and model-name casing.
Provider key creation returns 400Use provider: "nebius" without adapter.

Next Steps​

You have now connected AISIX to Nebius Token Factory and verified the model alias. Continue with these guides: