Skip to main content
API7.ai Logo
Version: Dev

Decision Models

Decision models answer structured questions about an input instead of generating text. Each question is one of three kinds: a yes/no judgement answered with a probability, a choice among named options, or a score on an ordered scale. The answers come back as probabilities and confidences that an application can threshold, route on, or log.

AISIX serves decision models on two endpoints, one for each request format in use:

EndpointRequest formatQuestion types
POST /v1/systemoneTypeSafe's System One format: model, state, and a questions object keyed by question name. Answers are keyed by the same names, and usage reports input_tokens and output_tokens.noul (yes/no), choice, score
POST /v1/decisionsOpenAI's Decisions format: model, input, and a questions array. answers is an array in question order and can contain a refusal; usage reports input_tokens, output_tokens, total_tokens, and their details.predicate (yes/no), choice, score

Either endpoint works with any of the supported upstreams. When the caller's format differs from the upstream's, AISIX converts the request and the answers, as described in Format Conversion.

Both endpoints are non-streaming. They use the same caller API keys, model aliases, access controls, rate limits, and budgets as the other proxy endpoints, with the exception that guardrails do not run on them.

Supported Upstreams​

The provider of the model's provider key decides which upstream serves a request and which format it speaks:

Provider key providerUpstreamFormatUpstream URL
typesafeTypeSafeSystem One{api_base}/v1/systemone, with api_base defaulting to https://api.typesafe.ai
azureMicrosoft FoundrySystem One{resource root}/providers/microsoft/v1/systemone
openrouterOpenRouterSystem One{api_base}/systemone
openaiOpenAIDecisions{api_base}/decisions, with api_base defaulting to https://api.openai.com/v1

A target whose provider key has any other provider is refused with 400, and the message names the model and its provider, for example model `my-model` uses provider `anthropic`; /v1/decisions requires a typesafe, azure, openai, or openrouter provider.

The model's model_name is the upstream model ID AISIX sends as model:

UpstreamExample model_name
TypeSafejev-latest
Microsoft FoundryThe name of the deployment that serves Microsoft-Decision-1 in your Foundry resource
OpenAIgpt-6-luna
OpenRoutertypesafe/jev-1.13

TypeSafe​

Create a provider key whose provider is typesafe and whose secret is your TypeSafe API key. AISIX sends the key as a bearer token. api_base is optional. Leave it empty to use https://api.typesafe.ai, or set it to the host of another TypeSafe endpoint. AISIX appends /v1/systemone to the host and accepts a value that already ends in /v1 or /v1/systemone.

A typesafe provider key can back both decision models and the classifier of a TypeSafe jev semantic router. AISIX Cloud refuses it on an embedding model.

Microsoft Foundry​

Use a provider key whose provider is azure and set its api_base to the endpoint of the Foundry resource that hosts your Microsoft-Decision-1 deployment, such as https://acme-foundry.services.ai.azure.com. You can paste the project endpoint (…/api/projects/<project>) or an endpoint that ends in /openai/v1, /openai, /models, or /v1 instead; AISIX reduces each of them to the resource root before appending /providers/microsoft/v1/systemone. A Foundry key without api_base is refused with 400.

Set the model's model_name to the deployment name. Authenticate the same way as Azure OpenAI: a resource API key is sent in the api-key header, and an Entra ID client credential is exchanged for a token that is sent as Authorization: Bearer <token>.

OpenAI​

Use a provider key whose provider is openai. api_base defaults to https://api.openai.com/v1, and AISIX calls {api_base}/decisions. Set model_name to a model that serves the Decisions API, such as gpt-6-luna.

OpenRouter​

Use a provider key whose provider is openrouter and whose api_base is https://openrouter.ai/api/v1, as described in the OpenRouter guide. AISIX calls {api_base}/systemone. Set model_name to an OpenRouter model that its System One endpoint serves, such as typesafe/jev-1.13.

Configure a Decision Model​

A decision model is an ordinary direct model whose provider key is one of the four providers above. The following example creates a TypeSafe-backed model.

AISIX Cloud​

Export the AISIX Cloud connection details and your TypeSafe API key:

# AISIX_CP is the Admin API base URL; include /api and omit a trailing slash.
# The local On-Premises quickstart uses http://localhost:8080/api.
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"
export TYPESAFE_API_KEY="YOUR_TYPESAFE_API_KEY"

Create the provider key and allow it in the environment:

TYPESAFE_KEY_ID=$(curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "typesafe",
    "provider": "typesafe",
    "api_key": "'"$TYPESAFE_API_KEY"'",
    "allowed_environments": ["'"$ENV_ID"'"]
  }' | jq -r '.provider_key.id')

Create a direct model on that key:

curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "jev",
    "model_name": "jev-latest",
    "provider_key_id": "'"$TYPESAFE_KEY_ID"'"
  }'

On the dashboard, create the key on the Provider Keys page with the typesafe provider, then create a direct model on the Models page and pick that key. The form suggests jev-latest as the upstream model ID.

Grant the model in a caller API key's allowed models, then set its price.

Open-Source AISIX Gateway​

Add the provider key and the model to the resources file:

resources.yaml (decision model)
provider_keys:
  - display_name: typesafe
    provider: typesafe
    api_key: ${TYPESAFE_API_KEY}

models:
  - display_name: jev
    provider: typesafe
    model_name: jev-latest
    provider_key: typesafe

Add jev to the caller key's allowed_models, then validate and reload the complete resources file.

Model Kinds​

The decision endpoints serve the following model kinds:

  • Direct models whose provider key is typesafe, azure, openai, or openrouter.
  • Routing models, which fail over across their targets the same way other single-shot endpoints do. See Routing and Failover. Every target must be a decision model on one of the four providers; a target on another provider ends the request with 400 instead of failing over.
  • Wildcard aliases, which resolve to a direct model like any other request.

Ensemble, semantic, and embedding models are refused with 400, for example model `my-ensemble` is a `ensemble` model; /v1/systemone serves direct and routing models only.

Send a Request​

Export the gateway connection and the model alias:

# AISIX_PROXY has no trailing slash or endpoint path such as /v1.
# The local quickstarts use http://127.0.0.1:3000.
export AISIX_PROXY="YOUR_AISIX_GATEWAY_URL"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="jev"

System One Format​

Send a request with a named choice question to /v1/systemone:

curl -sS -X POST "${AISIX_PROXY}/v1/systemone" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"${AISIX_MODEL}"'",
    "state": "The API returns 500 on every call; engineering should look.",
    "questions": {
      "team": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {
          "billing": "Charges and invoices",
          "engineering": "Bugs and outages"
        }
      }
    }
  }'

The answers are keyed by question name, and model carries the alias the caller sent:

{
  "model": "jev",
  "answers": {
    "team": {
      "type": "choice",
      "choice": "engineering",
      "confidence": 0.97,
      "probabilities": {
        "billing": 0.03,
        "engineering": 0.97
      }
    }
  },
  "usage": {
    "input_tokens": 381,
    "output_tokens": 62
  }
}

A noul question is answered with a probability in its noul field. A score question is answered with score, confidence, a legend that maps each level index to its label, and probabilities keyed by level index.

Decisions Format​

Send the same question to /v1/decisions as an array entry:

curl -sS -X POST "${AISIX_PROXY}/v1/decisions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"${AISIX_MODEL}"'",
    "input": "The API returns 500 on every call; engineering should look.",
    "questions": [
      {
        "type": "choice",
        "name": "team",
        "instructions": "Which team should handle this?",
        "choices": [
          {"value": "billing", "description": "Charges and invoices"},
          {"value": "engineering", "description": "Bugs and outages"}
        ]
      }
    ]
  }'

The answers come back in question order:

{
  "model": "jev",
  "answers": [
    {
      "type": "choice",
      "name": "team",
      "choice": "engineering",
      "probabilities": [
        {"value": "billing", "probability": 0.03},
        {"value": "engineering", "probability": 0.97}
      ],
      "confidence": 0.97
    }
  ],
  "usage": {
    "input_tokens": 381,
    "input_tokens_details": {"cached_tokens": 0, "cache_write_tokens": 0},
    "output_tokens": 62,
    "output_tokens_details": {"reasoning_tokens": 0},
    "total_tokens": 443
  }
}

A predicate question is answered with probability. A score question is answered with score, confidence, and probabilities entries that carry each level's index as value, its label, and its probability. An answer of type refusal means the model declined to answer that question.

Use an SDK​

Both vendors' SDKs reach the gateway by changing only their base URL.

The TypeSafe Python SDK appends /v1/systemone to its base URL, so pass the gateway origin without /v1:

import os

from typesafe_sdk import Choice, TypeSafeClient

with TypeSafeClient(
    api_key=os.environ["AISIX_API_KEY"],
    base_url=os.environ["AISIX_PROXY"],
    model="jev",
) as client:
    response = client.system_one(
        state="The API returns 500 on every call; engineering should look.",
        questions={
            "team": Choice(
                instructions="Which team should handle this?",
                criteria={"billing": "Charges and invoices", "engineering": "Bugs and outages"},
            ),
        },
    )

print(response.choices["team"].choice)

The OpenAI Python SDK appends /decisions to its base URL, so pass the gateway origin with /v1:

import os

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AISIX_API_KEY"],
    base_url=os.environ["AISIX_PROXY"] + "/v1",
)

response = client.decisions.create(
    model="jev",
    input="The API returns 500 on every call; engineering should look.",
    questions=[
        {
            "type": "choice",
            "name": "team",
            "instructions": "Which team should handle this?",
            "choices": [
                {"value": "billing", "description": "Charges and invoices"},
                {"value": "engineering", "description": "Bugs and outages"},
            ],
        }
    ],
)

print(response.answers[0])

Format Conversion​

Same Format​

When the caller's format is the upstream's, AISIX forwards the request body unchanged except for model, which it rewrites to the upstream model ID. The response is returned unchanged except for model, which carries the caller's alias again. Fields AISIX does not know about pass through in both directions, so the upstream decides whether to accept them. A provider key's request body overrides still apply.

Converted Requests​

When the formats differ, AISIX converts the request and the answers. The conversion loses or reshapes information in a few places:

  • Message input on /v1/decisions becomes the System One state string: each message's text parts are joined with blank lines, and the messages are joined the same way.
  • System One keys are the question names when every Decisions question has a distinct name. When any name is missing or repeated, every question is keyed by its position instead, "0" through "n-1", and the answers are mapped back by position.
  • A noul question's true and false criteria are folded into the predicate's instructions as Answer true when: … and Answer false when: …. A predicate becomes a noul question with instructions only.
  • Boolean choice values travel to a System One upstream as the keys "true" and "false", and are restored to boolean values in the answer.
  • Score levels with a description travel to a System One upstream as "label: description" criteria. Answers are labelled from the request, and a level the upstream gave no probability is reported with probability 0.
  • A System One choice or score question without instructions reaches OpenAI with a default instruction, such as Which choice best fits the input?, because the Decisions format requires one.
  • An answer that is missing, or whose type does not match its question, becomes a refusal on /v1/decisions and is omitted from answers on /v1/systemone, which has no refusal type.
  • Fields one format has no place for are dropped.

Some requests cannot be converted and are refused with 400 before AISIX contacts the upstream:

RequestUpstreamError message
/v1/decisions with an input_image content partTypeSafe, Foundry, or OpenRouterinput_image content parts are not supported because this model's upstream accepts text input only
/v1/decisions with safety_identifierTypeSafe, Foundry, or OpenRouter`safety_identifier` is supported only when the model is served by OpenAI
/v1/systemone with a choice or score question that has fewer than 2 options or levelsOpenAIOpenAI needs at least 2 choices or levels on every choice or score question
/v1/systemone with audio or videosOpenAIaudio and video input are not supported when the model is served by OpenAI

A body that does not match its endpoint's format, such as a /v1/systemone request without state or a question of an unknown type, is also refused with 400 when it has to be converted.

Upstream Errors and Malformed Answers​

An upstream 4xx is returned with the upstream's status and its error message. An upstream 5xx becomes 502. Retryable failures fail over to the next target of a routing model.

On a converted request, AISIX reads the upstream answers before replying. A response that is not JSON, that lacks its answers collection, or that contains an answer missing a field its own type requires fails the whole request with 502, and the message names the field, for example the decision upstream returned an unexpected decision response: `answers.team.confidence` is missing or not a number. The failure counts against the target like any other upstream fault.

Governance and Telemetry​

The decision endpoints apply the same request-side controls as the other model endpoints:

  • Caller authentication, model access, and the model's allowed_cidrs.
  • API key and model rate limits on requests and tokens. The token count is the upstream's input plus output tokens.
  • Budgets in AISIX Cloud.
  • Model cooldown, retries, and failover.

Guardrails attached to the model or API key do not run on /v1/systemone or /v1/decisions, and nothing is redacted. Do not rely on a guardrail to screen decision traffic.

Each request emits usage events whose operation is systemone for /v1/systemone and decisions for /v1/decisions, whichever upstream served the request. The events carry the upstream's input and output tokens, plus the cached, cache-write, and reasoning counts that an OpenAI upstream reports. Access logs and request metrics are recorded as on other endpoints.

Pricing​

Price a decision model with an organization pricing override that matches its provider and model name. The models.dev catalog has no prices for TypeSafe jev models, for OpenRouter's typesafe/jev-1.13, or for Foundry deployment names, so without an override their usage records $0.00 spend.

An override can also carry an optional set of decision-endpoint rates. In the Add pricing dialog, expand Decisions pricing (optional), or send a decisions object in the Admin API request. These rates apply only to requests on /v1/systemone and /v1/decisions, whichever format the caller used:

  • Each rate you set replaces the matching base rate for those requests.
  • A rate you leave blank uses the matching base rate.
  • The usual fallbacks then apply to the rates in effect: cache read and cache write bill at the prompt rate when they have no rate of their own, and reasoning bills at the completion rate.

Use this when the same model is billed differently on the decision endpoints than on its other endpoints. For example, gpt-6-luna has a catalog price for its general use. To bill its decision calls at different rates, add an override for openai / gpt-6-luna with the base rates and the decision-endpoint rates.

Decision-endpoint rates affect recorded cost and budgets only. least_cost routing ranks targets on the base rates. Saving an override replaces the whole price, so an Admin API request that saves a price without decisions clears the decision-endpoint rates.

Limitations​

  • Background health checks. The background check probes a model with a chat completions request. AISIX Cloud refuses to enable it on a model whose provider key is a typesafe key, and refuses to move a model that has it enabled onto such a key. Do not enable it on a decision-only model behind an azure, openai, or openrouter key either: the probe fails and marks the model unhealthy.
  • Ensembles and semantic routers. A model whose provider serves no chat completions, such as a model on a jina, runwayml, or typesafe key, cannot be an ensemble member or judge, or a semantic router target. AISIX Cloud rejects the reference, because those positions send chat requests.
  • Playground. The Playground sends chat completions requests and does not support decision models. It does not list models on a typesafe key.
  • Streaming. Both endpoints are non-streaming.

Next Steps​