Skip to main content
API7.ai Logo

ModelScope

Connect AISIX AI Gateway to ModelScope so applications can call ModelScope-hosted models through the gateway's OpenAI-compatible API. AISIX keeps the ModelScope credential on the gateway side. It authorizes model access with caller API keys and allowlists, and applies the same rate-limit and usage-accounting controls as other model aliases.

ModelScope is a model community platform whose API-Inference service serves open-weight models over an OpenAI-compatible endpoint, so it uses the openai adapter with HTTP bearer authentication and a ModelScope api_base.

ModelScope reaches AISIX through the community section of the provider catalog rather than through a curated catalog entry. The gateway accepts the modelscope provider ID because it is present in the models.dev catalog that the control plane caches, and it assumes an OpenAI-shaped wire unless you override that assumption on the provider key. Read Supply the Wire Details AISIX Does Not Curate before you put production traffic on this upstream.

Prerequisites​

The following examples configure the upstream through the AISIX Cloud Admin API. Before configuring the upstream, prepare the following:

  • Access to AISIX Cloud with an environment, an attached gateway, and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
  • A ModelScope API token from your ModelScope account. The ModelScope API-Inference documentation describes the account activation and quota rules that apply before a token can serve inference requests.

Export the connection details used by every request below:

# Replace with your values
export AISIX_CP="http://localhost:8080/api"
export AISIX_TOKEN="aisix_pat_YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Configure the ModelScope Upstream​

Create a provider key, model alias, and caller API key for the ModelScope-backed chat-completions route.

Create a Provider Key​

Create the provider key that stores the ModelScope credential and API root:

# Replace with your value
export MODELSCOPE_API_KEY="YOUR_PROVIDER_API_KEY"

PROVIDER_KEY_ID=$(curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "modelscope-prod",
    "provider": "modelscope",
    "api_key": "'"${MODELSCOPE_API_KEY}"'",
    "api_base": "https://api-inference.modelscope.cn/v1",
    "allowed_environments": ["'"${ENV_ID}"'"]
  }' | jq -r '.provider_key.id')

echo "$PROVIDER_KEY_ID"

❶ provider is modelscope. The Cloud Admin API accepts this value because the ID exists in the cached models.dev catalog, and it applies the catalog's default rule: the openai adapter and HTTP bearer authentication. Do not set the adapter field, which the Cloud Admin API accepts only on BYO provider keys.

❷ api_key stores the ModelScope API token. The openai adapter sends it in an Authorization: Bearer header. The value follows the credential-handling behavior in Provider Credentials.

❸ api_base is https://api-inference.modelscope.cn/v1, the API root that ModelScope's OpenAI-compatible chat-completions endpoint, https://api-inference.modelscope.cn/v1/chat/completions, hangs off. AISIX appends the endpoint path, such as /chat/completions, to the value you store. For modelscope this field is optional, because the control plane fills in this same root from the catalog when you omit it. The examples set it explicitly so the root each key targets stays visible in the configuration.

caution

Store the API root, not the full endpoint URL and not the bare host. AISIX strips a trailing /chat/completions if you paste one, but it never synthesizes the /v1 segment for a non-OpenAI host. An api_base of https://api-inference.modelscope.cn produces upstream requests to https://api-inference.modelscope.cn/chat/completions, a path ModelScope does not serve.

The command captures the returned provider key ID in PROVIDER_KEY_ID.

Create a Model​

ModelScope model IDs are organization-namespaced. Each ID repeats the organization/model path of the model page on ModelScope, including its capitalization. Current IDs in the AISIX provider catalog include the following:

Model IDContext windowNotes
Qwen/Qwen3-235B-A22B-Instruct-2507262,144 tokensInstruction model. Does not emit reasoning.
Qwen/Qwen3-235B-A22B-Thinking-2507262,144 tokensReasoning variant of the same weights.
Qwen/Qwen3-Coder-30B-A3B-Instruct262,144 tokensCoding and software-agent model.
ZhipuAI/GLM-4.6202,752 tokensGLM flagship served under the ZhipuAI organization.

Confirm the ID on the model's ModelScope page before you create an alias, because the set of models that API-Inference serves changes over time.

Create the model alias callers will send in requests:

MODEL_ID=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "modelscope-qwen-prod",
    "model_name": "Qwen/Qwen3-235B-A22B-Instruct-2507",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }' | jq -r '.model.id')

echo "$MODEL_ID"

❶ display_name is the alias callers send in model.

❷ model_name is the full ModelScope model ID, organization prefix included. Dropping the prefix, or reusing an unprefixed ID such as qwen3-235b-a22b-instruct-2507 from another host of the same weights, produces an upstream model-not-found error.

❸ provider_key_id attaches the alias to the ModelScope provider key.

Create a Caller API Key​

Create the caller API key that can access the model alias. The plaintext key is server-generated and returned once in the create response, so capture it now:

AISIX_API_KEY=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "modelscope-caller",
    "allowed_models": ["'"${MODEL_ID}"'"]
  }' | jq -r '.plaintext')

echo "$AISIX_API_KEY"

The allowed_models value must reference the model ID captured in the previous step. After the write, the configuration projects to attached gateways automatically.

Verify the Provider Connection​

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "http://127.0.0.1:3000/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "modelscope-qwen-prod",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from ModelScope."
      }
    ]
  }'

The gateway returns an OpenAI-compatible response that echoes the caller-facing alias modelscope-qwen-prod. If the request fails, check the provider key api_key, the api_base root, and the organization-prefixed model ID in model_name.

Choose Between ModelScope and Alibaba Cloud Model Studio​

ModelScope and Alibaba Cloud Model Studio both serve Qwen models, but they are separate upstreams with separate provider IDs in AISIX. Selecting the wrong one produces an upstream authentication or model-not-found error rather than a configuration error at create time, so confirm which service issued your credential first.

ModelScopeAlibaba Cloud Model Studio
Provider IDmodelscopealibaba
Catalog tierCommunityCurated
Setup guideThis pageQwen (Alibaba Cloud)
API roothttps://api-inference.modelscope.cn/v1Region-specific DashScope OpenAI-compatible root
CredentialModelScope API tokenDashScope API key, issued per region
Model ID formOrganization-namespaced repository path, such as Qwen/Qwen3-235B-A22B-Instruct-2507Model Studio model name, such as qwen-plus
Modeled /v1/videos routeRejectedSupported

The same distinction applies to GLM. ModelScope serves GLM weights under the ZhipuAI organization, but a GLM alias backed by the modelscope provider is not equivalent to one backed by the curated zhipuai provider described in Zhipu AI (GLM). Gateway surfaces that dispatch by provider label, such as the modeled video-generation route, key off the provider recorded on the model, not off which weights the upstream happens to serve.

Supply the Wire Details AISIX Does Not Curate​

For a curated provider, the AISIX provider catalog records the adapter, the authentication scheme, the default API root, and any request or response quirks, and the gateway applies them without operator input. ModelScope has no such entry. It resolves through the catalog's default rule, which grants the openai adapter and bearer authentication and marks the provider as community-sourced, with wire compatibility assumed to be OpenAI-shaped unless the operator overrides it per key. Everything beyond that assumption is yours to configure.

In practice:

  • No parameter renames are registered. AISIX forwards the top-level chat-completions parameters exactly as the caller sent them. Some curated providers carry a rename for a field such as max_completion_tokens; ModelScope carries none, so a body that works against a renamed provider is not automatically portable here, and the reverse is also true.
  • No response field remapping is registered. AISIX already normalizes reasoning that arrives at message.reasoning_content, delta.reasoning_content, or message.reasoning into the canonical reasoning_content field. Reasoning delivered on any other streaming delta path is not remapped unless you map it.
  • No provider-wide reasoning control applies. ModelScope does not publish a shared reasoning toggle, effort level, or thinking-token budget that spans the models it serves, so reasoning behavior follows the individual model. Validate it per model ID: Qwen/Qwen3-235B-A22B-Thinking-2507 reasons by design, while Qwen/Qwen3-235B-A22B-Instruct-2507 does not.

Both kinds of adjustment are provider-key fields. Add them to the provider key create request, or to the provider key entry in a declarative resources file:

{
  "request": {
    "param_renames": {
      "max_completion_tokens": "max_tokens"
    }
  },
  "response": {
    "reasoning_field": "delta.thinking"
  }
}

Use request.param_renames only when a specific ModelScope-served model rejects a parameter name your clients already send, and response.reasoning_field only when a streaming response carries reasoning on a path other than delta.reasoning_content. See Provider-Specific Overrides for the field catalog and the adapters each override applies to.

An override on the provider key applies to every model that references it, so validate a change against a non-production alias first. For the same reason, consider one provider key per organization namespace when the models you serve behave differently: Qwen/* and ZhipuAI/* share one endpoint but are different model families. Send both a buffered and a streaming request through each new alias, and confirm the fields your clients read are present before you route traffic to it.

Account for Cost and Shared Quota​

The managed control plane resolves per-request cost by the exact (provider, model name) pair, falling back to the models.dev catalog default when no organization override matches. The ModelScope catalog entries publish an input and output price of zero, so usage events for a modelscope model record their token counts and resolve to $0.00 spend. Budgets evaluated against that spend never advance, and a least_cost routing group treats the alias as free.

If cost reporting or budgets must reflect what ModelScope traffic actually costs your organization, add an organization pricing override for provider modelscope and the model name exactly as configured, organization prefix included. See Model Pricing.

One upstream ModelScope account backs every caller behind the gateway, so an upstream quota rejection reaches every caller at once. Set gateway-side caps so a single caller cannot exhaust the shared account: attach a rate limit to the model alias, to the caller API key, or to both. See Rate Limits.

Endpoint Coverage​

ModelScope API-Inference is an inference-only upstream, so only part of the proxy surface applies to a ModelScope-backed alias.

RouteBehavior with a ModelScope alias
/v1/chat/completionsSupported, including stream: true.
/v1/responsesSupported through the Responses bridge over the chat adapter path.
/v1/messagesSupported for Anthropic-shaped callers through translation. Token counting at /v1/messages/count_tokens requires an Anthropic-backed model.
/v1/embeddingsDispatches through the openai adapter, so it works only when the configured model is served at the upstream /v1/embeddings path. The ModelScope entries in the AISIX provider catalog are text chat models. See Embeddings.
/v1/images/generationsRejected with a 400 error. The route accepts only models whose provider is openai.
/v1/rerankRejected with a 400 error. The route accepts only the openai, cohere, and jina provider values.
/v1/videosRejected with 501 not_implemented. The route dispatches by provider label, and modelscope is not one of the labels it accepts, including for GLM weights that the zhipuai provider can drive on this route.
/passthrough/modelscope/*Supported for provider-native routes, with limited gateway normalization. See Provider Passthrough.

Next Steps​

You have now connected AISIX to ModelScope and verified the model alias. Continue with these guides: