Models

In this guide, you will create the model aliases callers send to AISIX and learn how single-target and multi-target models fit into the request path.

A model connects the caller-facing model name to the upstream target AISIX should call. A single-target model points to one upstream model through one provider key. A multi-target model points to multiple target model aliases and lets AISIX choose among them at request time.

Start with a single-target model. Create a multi-target model when callers need one stable name in front of failover, round-robin, or weighted target selection.

Prerequisites

Before starting, prepare the following:

• A self-hosted gateway with the admin listener available.
• The admin key from the gateway config.yaml.
• A provider key ID for the upstream credential. If you do not have one yet, configure Provider Keys first.

Create a Single-Target Model

A single-target model maps one AISIX model alias to one upstream model.

Create a single-target model with the provider key ID you prepared:

export AISIX_ADMIN_KEY="admin-local-only-change-me"
export PROVIDER_KEY_ID="YOUR_PROVIDER_KEY_ID"

curl -sS -X POST "http://127.0.0.1:3001/admin/v1/models" \
  -H "Authorization: Bearer ${AISIX_ADMIN_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "gpt-4o-prod",
    "provider": "openai",
    "model_name": "gpt-4o",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }'

You should see a response similar to the following:

{
  "id": "677c847f-d92d-4f0e-b445-8b449764f06a",
  "value": {
    "display_name": "gpt-4o-prod",
    "provider": "openai",
    "model_name": "gpt-4o",
    "provider_key_id": "YOUR_PROVIDER_KEY_ID"
  },
  "revision": 1
}

Copy the highlighted id if you plan to update, inspect, or delete this model later.

The display_name is the name callers send in model. The model_name is the upstream model ID or deployment name AISIX sends to the provider. These values can be the same, but they do not have to be.

Create a Multi-Target Model

A multi-target model is useful when one caller-facing alias should select from multiple target models. It uses a routing block instead of storing its own provider key or upstream model name.

Create the target models first. Then create the multi-target model and reference those target aliases by display_name:

curl -sS -X POST "http://127.0.0.1:3001/admin/v1/models" \
  -H "Authorization: Bearer ${AISIX_ADMIN_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "chat-prod",
    "routing": {
      "strategy": "failover",
      "targets": [
        {"model": "gpt-4o-primary"},
        {"model": "gpt-4o-secondary"}
      ],
      "retries": 1,
      "max_fallbacks": 1,
      "retry_on_429": true
    }
  }'

You should see a response similar to the following:

{
  "id": "134c4b01-09e6-41b7-97c7-f4e9a608f4c2",
  "value": {
    "display_name": "chat-prod",
    "routing": {
      "strategy": "failover",
      "targets": [
        {
          "model": "gpt-4o-primary"
        },
        {
          "model": "gpt-4o-secondary"
        }
      ],
      "retries": 1,
      "max_fallbacks": 1,
      "retry_on_429": true
    }
  },
  "revision": 1
}

Copy the highlighted id if you plan to update, inspect, or delete this multi-target model later.

Choose the strategy based on how the gateway should select the first target:

Strategy	Use When
failover	One target is primary and later targets are backups.
round_robin	Similar targets should receive simple request distribution.
weighted	Targets should receive unequal traffic shares.

Fallback still follows the target order after the first target is selected. For the detailed routing behavior, see Routing and Failover.

Configure Optional Model Behavior

Most models only need a caller-facing alias, provider label, upstream model name, and provider key ID. Add optional fields only when the behavior is part of your traffic plan.

Common optional fields include:

• timeout, when a provider request should have a stricter per-request timeout.
• stream_timeout, when a streaming request should have a separate per-chunk read timeout.
• allowed_cidrs, when only callers from specific client IP ranges should use the model alias.
• background_model_check, when AISIX should probe a single-target model outside the request path and mark it unhealthy after failed probes.
• cooldown, when real request failures should temporarily exclude a single-target model from routing.
• rate_limit, when the limit should follow one model alias. For details, see Rate Limits.

Multi-target models use the selected target model's provider settings, timeout, health, and cooldown behavior. Configure those fields on the target models, not on the multi-target alias.

allowed_cidrs applies to the model alias the caller names. AISIX resolves the client IP from the immediate peer unless proxy.real_ip is configured to trust forwarded headers from your load balancer or ingress.

Cost Metadata

Use cost when usage reporting or budget checks need pricing metadata for the model alias. The field records input and output cost in USD per 1,000 tokens; it does not affect provider routing or access control.

Add the cost object when you create or update the model alias that represents the priced upstream model:

{
  "display_name": "gpt-4o-prod",
  "provider": "openai",
  "model_name": "gpt-4o",
  "provider_key_id": "PROVIDER_KEY_ID",
  "cost": {
    "input_per_1k": 0.0025,
    "output_per_1k": 0.01
  }
}

Verify Caller Visibility

After creating or changing a model, verify both the admin resource and the caller-visible proxy behavior.

List models through the proxy with a caller API key that is allowed to use the alias:

export AISIX_API_KEY="YOUR_CALLER_API_KEY"

curl -sS "http://127.0.0.1:3000/v1/models" \
  -H "Authorization: Bearer ${AISIX_API_KEY}"

GET /v1/models lists single-target models that the caller API key is allowed to access. Multi-target aliases are intentionally hidden from this discovery response, even though callers can target them directly if they know the alias.

Next Steps

You have now configured the caller-facing model alias. Continue with Caller API Keys to allow callers to use the alias.

For complete model request fields, response shapes, and status routes, see the Admin API Reference.