Weights & Biases Inference

Connect AISIX AI Gateway to Weights & Biases Inference so applications can call W&B-hosted models through a stable AISIX model alias.

W&B Inference exposes an OpenAI-compatible API with bearer authentication. AISIX accepts the community catalog provider ID wandb and derives the openai adapter.

Setting	Value used by AISIX
Catalog provider ID	wandb
API base	https://api.inference.wandb.ai/v1
Example upstream model	meta-llama/Llama-3.1-8B-Instruct
Authentication	Bearer API key

Prerequisites

Prepare:

• An AISIX Cloud environment with an attached gateway.
• A W&B API key authorized for Inference.
• curl and jq.

Export the AISIX connection details:

export AISIX_CP="http://localhost:8080/api"
export AISIX_TOKEN="aisix_pat_YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"
export AISIX_PROXY="http://127.0.0.1:3000"

Configure the W&B Inference Upstream

Create a Provider Key

export WANDB_API_KEY="YOUR_WANDB_API_KEY"

PROVIDER_KEY_ID=$(
  curl -sS -X POST "$AISIX_CP/provider_keys" \
    -H "Authorization: Bearer $AISIX_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
      "display_name": "wandb-prod",
      "provider": "wandb",
      "api_key": "'"${WANDB_API_KEY}"'",
      "api_base": "https://api.inference.wandb.ai/v1",
      "allowed_environments": ["'"${ENV_ID}"'"]
    }' | jq -er '.provider_key.id'
)

echo "$PROVIDER_KEY_ID"

The AISIX provider ID is wandb, not weights-and-biases. Do not add adapter; AISIX derives the openai adapter for catalog providers.

The explicit API base includes /v1. AISIX appends /chat/completions when it sends the upstream request.

Create a Model

Create an alias with the complete W&B model ID:

MODEL_ID=$(
  curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
    -H "Authorization: Bearer $AISIX_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
      "display_name": "wandb-llama-prod",
      "model_name": "meta-llama/Llama-3.1-8B-Instruct",
      "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
    }' | jq -er '.model.id'
)

echo "$MODEL_ID"

Preserve the publisher namespace and casing when substituting another model from the W&B Inference catalog.

Create a Caller API Key

AISIX_API_KEY=$(
  curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
    -H "Authorization: Bearer $AISIX_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
      "display_name": "wandb-inference-caller",
      "allowed_models": ["'"${MODEL_ID}"'"]
    }' | jq -er '.plaintext'
)

echo "$AISIX_API_KEY"

Verify the Provider Connection

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer $AISIX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "wandb-llama-prod",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from W&B Inference."
      }
    ]
  }'

AISIX forwards the upstream model ID to POST /v1/chat/completions with the W&B key in the bearer header.

Endpoint Coverage

Chat completions is supported. AISIX can bridge compatible Responses and Anthropic Messages requests through the chat adapter. Embeddings require an embedding model that W&B serves on the same API root.

Image generation, video generation, and rerank do not accept the wandb provider value. See Provider Compatibility.

Troubleshooting

Symptom	Check
Upstream 401 or 403	Confirm the W&B key has Inference access.
Upstream 404	Keep /v1 in the API base and verify the model ID.
Model not found	Preserve the publisher namespace and casing.
Provider key creation returns 400	Use provider: "wandb" without adapter.

For provider-key rotation and environment scoping, see Provider Credentials.