Skip to main content
API7.ai Logo

OVHcloud AI Endpoints

OVHcloud AI Endpoints provides managed inference endpoints for hosted models. AISIX gives applications caller keys and stable model aliases for accessing those endpoints.

Prerequisites​

Before starting, prepare the following:

  • One AISIX setup:
    • For AISIX Cloud, an environment with an attached gateway and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
    • For the open-source AISIX gateway, prepare either a local AISIX installation or the Docker setup from the Open-Source AISIX Gateway Quickstart. Configure the gateway to load a declarative resources file.
  • An OVHcloud AI Endpoints access token.
  • Access to the selected model.
  • curl and jq.

Configure with AISIX Cloud​

Export the AISIX Cloud connection details:

# AISIX_CP is the Admin API base URL; include /api and omit a trailing slash
# The local On-Premises quickstart uses http://localhost:8080/api
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

OVHcloud is a community catalog provider with an OpenAI-compatible endpoint. AISIX connects through the openai adapter and authenticates upstream requests with a bearer token.

Create a Provider Key​

export OVHCLOUD_API_KEY="YOUR_OVHCLOUD_ACCESS_TOKEN"

PROVIDER_KEY_ID=$(
  curl -sS -X POST "$AISIX_CP/provider_keys" \
    -H "Authorization: Bearer $AISIX_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
      "display_name": "ovhcloud-prod",
      "provider": "ovhcloud",
      "api_key": "'"${OVHCLOUD_API_KEY}"'",
      "api_base": "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1",
      "allowed_environments": ["'"${ENV_ID}"'"]
    }' | jq -er '.provider_key.id'
)

echo "$PROVIDER_KEY_ID"

Use the catalog ID ovhcloud and omit adapter. AISIX derives the OpenAI wire format and bearer authentication.

The API base is the shared OpenAI-compatible root documented by OVHcloud. It includes /v1; AISIX adds /chat/completions.

Create a Model​

Create an alias with the exact OVHcloud model ID:

MODEL_ID=$(
  curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
    -H "Authorization: Bearer $AISIX_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
      "display_name": "ovhcloud-llama-prod",
      "model_name": "Meta-Llama-3_3-70B-Instruct",
      "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
    }' | jq -er '.model.id'
)

echo "$MODEL_ID"

OVHcloud model IDs do not necessarily match the publisher's original repository ID. Copy the value shown by OVHcloud, including underscores, hyphens, and casing.

Create a Caller API Key​

AISIX_API_KEY=$(
  curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
    -H "Authorization: Bearer $AISIX_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
      "display_name": "ovhcloud-caller",
      "allowed_models": ["'"${MODEL_ID}"'"]
    }' | jq -er '.plaintext'
)

echo "$AISIX_API_KEY"

Configure with the Open-Source AISIX Gateway​

Export the upstream credential and choose the caller API key that applications will send to the gateway:

export OVHCLOUD_API_KEY="YOUR_PROVIDER_API_KEY"
export CALLER_API_KEY="YOUR_CALLER_API_KEY"

Create a complete declarative resources file for this provider:

resources.yaml
_format_version: "1"

provider_keys:
  - display_name: "ovhcloud-prod"
    provider: "ovhcloud"
    adapter: "openai"
    api_key: ${OVHCLOUD_API_KEY}
    api_base: "https://oai.endpoints.kepler.ai.cloud.ovh.net/v1"

models:
  - display_name: "ovhcloud-llama-prod"
    provider: "ovhcloud"
    model_name: "Meta-Llama-3_3-70B-Instruct"
    provider_key: "ovhcloud-prod"

api_keys:
  - display_name: "ovhcloud-caller"
    key_env: CALLER_API_KEY
    allowed_models:
      - "ovhcloud-llama-prod"

If AISIX is installed locally, validate the file before loading it:

aisix validate --resources resources.yaml

After validation, start the gateway with the referenced environment variables in its process environment. Reload an existing gateway only if those variables are already available to the process; otherwise, restart it with the updated environment.

If you use Docker, adapt the validation and startup commands in the Open-Source AISIX Gateway Quickstart. Mount this resources.yaml file and pass every environment variable it references with -e in both commands. After the resources load, prepare the shared verification request below:

export AISIX_API_KEY="$CALLER_API_KEY"

Verify the Provider Connection​

Export the AISIX gateway origin:

# The local quickstarts use http://127.0.0.1:3000
export AISIX_PROXY="YOUR_AISIX_GATEWAY_ORIGIN"

Send a chat-completions request through the AISIX proxy:

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer $AISIX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ovhcloud-llama-prod",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from OVHcloud AI Endpoints."
      }
    ]
  }'

AISIX sends the OVHcloud model ID to the OpenAI-compatible chat endpoint with bearer authentication.

Endpoint Coverage​

An OVHcloud model alias does not reach every proxy route. AISIX uses the openai adapter for the normalized inference routes, but some routes also restrict the configured provider value.

RouteOVHcloud support
/v1/chat/completions, including stream: trueSupported. Tool calling, structured output, vision, and other capabilities depend on the selected model.
/v1/responsesSupported through the chat-based Responses bridge, not OVHcloud's native Responses API. Fields without a chat equivalent are ignored. Use /passthrough/ovhcloud/responses when the native contract is required.
/v1/messagesSupported for Anthropic-shaped callers through translation to chat completions. /v1/messages/count_tokens is unavailable because it requires an Anthropic-protocol provider key.
/v1/embeddingsSupported when the alias names an OVHcloud embedding model, such as bge-m3.
/v1/audio/transcriptionsSupported when the alias names an OVHcloud speech-to-text model, such as whisper-large-v3. AISIX rewrites the model alias and forwards the OpenAI-compatible multipart form to OVHcloud's transcription endpoint.
/v1/audio/translations and /v1/audio/speechNot available on the shared OVHcloud OpenAI-compatible base. OVHcloud handles translation through the transcription prompt and publishes text-to-speech models on separate native endpoints.
/v1/images/generations, /v1/videos, and /v1/rerankNot available for an alias whose provider value is ovhcloud; these AISIX routes use provider allowlists that exclude it.
/v1/modelsReturns caller-accessible AISIX aliases, not the OVHcloud model catalog. Use /passthrough/ovhcloud/models for OVHcloud's current list.
/passthrough/ovhcloud/*restAvailable through a configured passthrough route for OVHcloud-native routes, with limited gateway normalization.

The /passthrough/ovhcloud paths on this page assume a passthrough route claiming that prefix with the OVHcloud API base as its target_url; grant the route name on the caller key's allowed_routes. The normalized /v1/responses route is usually preferable when its translated feature set is sufficient: AISIX rewrites the model alias, streams incrementally, and parses usage. For native OVHcloud Responses behavior, passthrough forwards the body unchanged, so send the upstream model ID rather than the AISIX alias. OVHcloud currently requires store: false and client-managed conversation history on that API. A passthrough route relays the upstream response incrementally, including SSE. AISIX detects the Responses envelope from input and records token usage when the response includes supported input_tokens and output_tokens fields.

See Provider Compatibility for the gateway-wide endpoint rules.

Troubleshooting​

SymptomCheck
Upstream 401 or 403Confirm the access token and project entitlement.
Upstream 404Keep /v1 in api_base and copy the OVHcloud model ID exactly.
Model not foundCheck underscores, hyphens, and version segments in model_name.
Provider key creation returns 400Use provider: "ovhcloud" and omit adapter.

Next Steps​

You have now connected AISIX to OVHcloud AI Endpoints and verified the model alias. Continue with these guides: