Google Vertex AI

AISIX AI Gateway can route requests to Google Vertex AI. Callers can reach Vertex-hosted Gemini and partner models through the gateway's OpenAI-compatible API.

This configuration is for Vertex-hosted models that should use AISIX authentication, model allowlists, rate limits, and usage accounting. AISIX authenticates to Vertex with a GCP OAuth2 bearer token and can mint that token from a service-account key.

Prerequisites

The following examples use a self-hosted AISIX gateway. Before configuring the upstream, prepare the following:

• A running AISIX gateway with the Admin API and proxy API available. Commands use the Quickstart listener addresses; substitute your configured addresses if needed.
• The admin key from the gateway config.yaml.
• The Vertex AI API enabled in the target GCP project.
• A Vertex region, such as us-central1, and a service account that can invoke the target model.
• The GCP project ID and Vertex model ID.

Configure the Vertex Upstream

Create a Vertex provider key, model alias, and caller API key. The provider key stores the GCP project, region, and credential mode; the model selects the Vertex publisher model ID.

Create a Vertex Provider Key

Create the Vertex provider key with the GCP credential settings:

# Replace with your values
export AISIX_ADMIN_KEY="YOUR_ADMIN_KEY"

curl -sS -X POST "http://127.0.0.1:3001/admin/v1/provider_keys" \
  -H "Authorization: Bearer ${AISIX_ADMIN_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "vertex-prod",
    "provider": "google-vertex",
    "adapter": "vertex",
    "secret": "{\"project\":\"my-gcp-project\",\"region\":\"us-central1\",\"service_account_json\":{\"type\":\"service_account\",\"private_key\":\"-----BEGIN PRIVATE KEY-----\\nYOUR_SERVICE_ACCOUNT_PRIVATE_KEY\\n-----END PRIVATE KEY-----\\n\",\"client_email\":\"vertex-sa@my-gcp-project.iam.gserviceaccount.com\",\"token_uri\":\"https://oauth2.googleapis.com/token\"}}"
  }'

❶ provider labels the upstream.

❷ adapter selects Vertex.

❸ secret is a JSON string with project, region, and exactly one credential mode. The example uses service_account_json as a nested object inside the secret. The region drives the <region>-aiplatform.googleapis.com host unless you override api_base.

Set api_base when Vertex traffic should use a proxy or private endpoint instead of the regional host.

Use service_account_json unless you already manage short-lived GCP access tokens yourself. If you use access_token, you are responsible for refreshing it.

Provider key secrets follow the credential-handling behavior described in Provider Credentials.

Save the returned provider key ID for the model resource.

Create a Model

Map a caller-facing alias to the Vertex model ID:

# Replace with your values
export PROVIDER_KEY_ID="YOUR_PROVIDER_KEY_ID"

curl -sS -X POST "http://127.0.0.1:3001/admin/v1/models" \
  -H "Authorization: Bearer ${AISIX_ADMIN_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "gemini-prod",
    "provider": "google-vertex",
    "model_name": "gemini-2.5-flash",
    "provider_key_id": "'"${PROVIDER_KEY_ID}"'"
  }'

❶ provider uses the same label as the provider key.

❷ model_name is the Vertex publisher model ID.

❸ provider_key_id attaches the model to the Vertex credential.

Other supported examples include Claude on Vertex, OpenAI-compatible partner models such as Llama, Mistral, and AI21.

AISIX chooses the Vertex route from model_name:

Model ID Family	How AISIX Sends It to Vertex
Gemini models, such as gemini-*	Uses the Google Gemini publisher route.
Claude models, such as claude-*	Uses the Anthropic publisher route with an Anthropic Messages body.
OpenAI-compatible MaaS models, such as Llama, DeepSeek, Qwen, GPT-OSS, MiniMax, Moonshot, or Z.ai	Uses Vertex's OpenAI-compatible chat-completions route.
Mistral and AI21 models	Uses the partner publisher route with an OpenAI-compatible body.

Create a Caller API Key

Choose the caller API key value that the application will send to AISIX, then hash it for the admin resource:

# Replace with your values
export AISIX_API_KEY="YOUR_CALLER_API_KEY"

CALLER_KEY_HASH=$(printf '%s' "${AISIX_API_KEY}" | shasum -a 256 | awk '{print $1}')

Create the API key resource with access to the Vertex-backed model alias:

curl -sS -X POST "http://127.0.0.1:3001/admin/v1/apikeys" \
  -H "Authorization: Bearer ${AISIX_ADMIN_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "key_hash": "'"${CALLER_KEY_HASH}"'",
    "allowed_models": ["gemini-prod"]
  }'

The allowed_models value must match the model alias you created.

Verify the Provider Connection

Send a chat-completions request through the AISIX proxy. The example uses Gemini, which requires at least one user or assistant turn.

curl -sS -X POST "http://127.0.0.1:3000/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini-prod",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from Vertex."
      }
    ]
  }'

The gateway returns an OpenAI-compatible response with the caller-facing alias:

{
  "object": "chat.completion",
  "model": "gemini-prod",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello from Vertex!"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 4,
    "completion_tokens": 4,
    "total_tokens": 8
  }
}

Check Vertex logs, metrics, quota usage, or provider-side request records for the test request. If AISIX returns a token-minting or upstream authentication error, check the service-account key, region, Vertex API enablement, IAM role, and model access.

Vertex Publisher Routing

Publisher selection is prefix-based. If model_name does not match a supported prefix, the gateway rejects the request before the provider request with an unsupported-publisher configuration error.

The example uses Gemini because it is the main Google publisher path. For partner models, validate the exact model ID, quota, and regional availability in your Vertex project before exposing the alias to callers.

Provider-key request and response overrides can apply on Vertex routes, but they are most directly useful on OpenAI-compatible routes. Gemini's native contents format does not match every OpenAI-style override target.

Next Steps

You have now connected AISIX to Google Vertex AI and verified the model alias. Continue with these guides:

• Gemini (Google AI Studio): configure Gemini with an AI Studio API key instead of a Google Cloud project.
• Model Aliases: configure routing, health behavior, or cost metadata for the alias.
• Provider Compatibility: review supported proxy endpoints and provider-specific boundaries.