Ollama

Connect AISIX AI Gateway to a local or private Ollama server. Applications keep using AISIX caller keys and model aliases while Ollama runs the model on infrastructure you control.

Ollama is a private endpoint rather than an AISIX catalog provider. Configure it with the byo provider sentinel and select the openai adapter explicitly.

Setting	Value used by AISIX
Provider ID	`byo`
Adapter	`openai`
Default Ollama API base	`http://localhost:11434/v1/`
Example upstream model	`gpt-oss:20b`
Local authentication	No authentication; AISIX stores a required placeholder

Prerequisites

Prepare:

An AISIX Cloud environment with an attached gateway.
An Ollama server reachable from the AISIX data plane.
curl and jq.

Pull the model used in this guide:

ollama pull gpt-oss:20b

Ollama binds to 127.0.0.1:11434 by default. If AISIX runs in another container or host, configure a reachable bind address by following the Ollama server configuration. For example, a foreground server can listen on all interfaces:

OLLAMA_HOST="0.0.0.0:11434" ollama serve

caution

The local Ollama API does not require authentication. Binding it to 0.0.0.0 makes it reachable from other network peers. Restrict the listening network with firewall, container-network, or Kubernetes policy controls, and do not expose the port directly to the public Internet.

Export the AISIX connection details and an API base reachable from the AISIX data plane:

export AISIX_CP="http://localhost:8080/api"
export AISIX_TOKEN="aisix_pat_YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"
export AISIX_PROXY="http://127.0.0.1:3000"

# Docker Desktop data plane to Ollama on the host
export OLLAMA_API_BASE="http://host.docker.internal:11434/v1"

Choose the address for your topology:

AISIX data plane and Ollama topology	Example API base
Both processes on one host	`http://127.0.0.1:11434/v1`
AISIX in Docker Desktop, Ollama on the host	`http://host.docker.internal:11434/v1`
Both containers on one Docker network	`http://ollama:11434/v1`
Kubernetes	`http://ollama.<namespace>.svc.cluster.local:11434/v1`

On Linux, host.docker.internal may require an explicit host-gateway mapping. A successful request from your laptop does not prove that the AISIX data plane container can reach the same address.

Configure the Ollama Upstream

Create a Provider Key

PROVIDER_KEY_ID=$(
  curl -sS -X POST "$AISIX_CP/provider_keys" \
    -H "Authorization: Bearer $AISIX_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
      "display_name": "ollama-local",
      "provider": "byo",
      "adapter": "openai",
      "api_key": "ollama",
      "api_base": "'"${OLLAMA_API_BASE}"'",
      "allowed_environments": ["'"${ENV_ID}"'"]
    }' | jq -er '.provider_key.id'
)

echo "$PROVIDER_KEY_ID"

AISIX requires a non-empty api_key in the provider-key schema. Ollama's OpenAI client examples use ollama because the client requires a value, but the local Ollama server ignores it. AISIX sends the placeholder as a bearer token; it is not a security control.

A BYO key requires provider: "byo", a non-empty api_key, and api_base. This guide sets adapter: "openai" explicitly; when omitted, AISIX defaults a BYO key to the OpenAI-compatible adapter.

Create a Model

Use the exact local Ollama model tag:

MODEL_ID=$(
  curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
    -H "Authorization: Bearer $AISIX_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
      "display_name": "ollama-gpt-oss-prod",
      "model_name": "gpt-oss:20b",
      "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
    }' | jq -er '.model.id'
)

echo "$MODEL_ID"

Run ollama ls to see installed model tags. The tag, including a suffix such as :20b, is sent upstream unchanged.

Create a Caller API Key

AISIX_API_KEY=$(
  curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
    -H "Authorization: Bearer $AISIX_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
      "display_name": "ollama-caller",
      "allowed_models": ["'"${MODEL_ID}"'"]
    }' | jq -er '.plaintext'
)

echo "$AISIX_API_KEY"

Verify the Provider Connection

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer $AISIX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ollama-gpt-oss-prod",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from Ollama."
      }
    ]
  }'

Ollama should log POST /v1/chat/completions, and AISIX should return an OpenAI-compatible response.

Endpoint Coverage

Ollama documents OpenAI-compatible chat completions, completions, Responses, models, and embeddings routes. Through AISIX, chat completions is the primary route; bridged Responses and Messages requests use the same chat adapter. Embeddings require an installed embedding model.

The normalized AISIX image, video, and rerank routes do not accept the byo provider value even when Ollama implements a similarly named endpoint. See Provider Compatibility.

Troubleshooting

Symptom	Check
Connection refused or timeout	Test `OLLAMA_API_BASE` from the AISIX data plane container, not only from the host.
Ollama listens only on `127.0.0.1`	Set `OLLAMA_HOST` through the supported service configuration and restart Ollama.
Model not found	Run `ollama pull gpt-oss:20b` and verify the tag with `ollama ls`.
Provider key creation returns `400`	Include `provider: "byo"`, a non-empty `api_key`, and `api_base`. `adapter` is optional; if supplied, use a supported value such as `openai`.

For a generic private-server setup, see Bring Your Own Endpoint.

Prerequisites​

Configure the Ollama Upstream​

Create a Provider Key​

Create a Model​

Create a Caller API Key​

Verify the Provider Connection​

Endpoint Coverage​

Troubleshooting​