Skip to main content
API7.ai Logo

Google Vertex AI

Google Vertex AI is Google Cloud's managed platform for Gemini and partner models. AISIX gives applications one OpenAI-compatible API for these Vertex-hosted models.

This configuration is for Vertex-hosted models that should use AISIX authentication, model allowlists, rate limits, and usage accounting. AISIX authenticates to Vertex with a GCP OAuth2 bearer token and can mint that token from a service-account key.

Prerequisites​

Before starting, prepare the following:

  • One AISIX setup:
    • For AISIX Cloud, an environment with an attached gateway and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
    • For the open-source AISIX gateway, prepare either a local AISIX installation or the Docker setup from the Open-Source AISIX Gateway Quickstart. Configure the gateway to load a declarative resources file.
  • The Vertex AI API enabled in the target GCP project.
  • A Vertex location supported by the target model and a service account that can invoke it. The current Gemini example uses global.
  • The GCP project ID and Vertex model ID.
  • curl and jq.

Configure with AISIX Cloud​

Export the AISIX Cloud connection details:

# AISIX_CP is the Admin API base URL; include /api and omit a trailing slash
# The local On-Premises quickstart uses http://localhost:8080/api
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Create a Vertex provider key, model alias, and caller API key. The provider key stores the GCP project, region, and credential mode; the model selects the Vertex publisher model ID.

Create a Vertex Provider Key​

Create the Vertex provider key with the GCP credential settings:

PROVIDER_KEY_ID=$(curl -sS -X POST "$AISIX_CP/provider_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "vertex-prod",
    "provider": "google-vertex",
    "api_base": "https://aiplatform.googleapis.com",
    "config": {
      "project": "my-gcp-project",
      "region": "global",
      "service_account_json": {
        "type": "service_account",
        "private_key": "-----BEGIN PRIVATE KEY-----\nYOUR_SERVICE_ACCOUNT_PRIVATE_KEY\n-----END PRIVATE KEY-----\n",
        "client_email": "vertex-sa@my-gcp-project.iam.gserviceaccount.com",
        "token_uri": "https://oauth2.googleapis.com/token"
      }
    },
    "allowed_environments": ["'"$ENV_ID"'"]
  }' | jq -r '.provider_key.id')

❶ provider selects the Google Vertex catalog entry, which routes traffic through the Vertex protocol adapter.

❷ AISIX Cloud requires api_base for this platform provider. For the global location, use https://aiplatform.googleapis.com; https://global-aiplatform.googleapis.com is not the global endpoint. For a regional location, use https://<region>-aiplatform.googleapis.com, matching config.region. A proxy or private endpoint can replace either origin.

❸ config is the structured credential with project, region, and exactly one credential mode. The example uses service_account_json as a nested object. Omit api_key when the credential is supplied through config; a non-empty api_key is rejected for this provider.

Use service_account_json unless you already manage short-lived GCP access tokens yourself. AISIX signs a JWT, mints an OAuth token, caches it, and refreshes it before expiry. If you use access_token, you are responsible for refreshing it. The current adapter does not discover Application Default Credentials, metadata-server credentials, or Workload Identity Federation credentials.

Provider key secrets follow the credential-handling behavior described in Provider Keys.

The command captures the returned provider key ID for the model resource. allowed_environments lets the environment reference this key.

Create a Model​

Map a caller-facing alias to the Vertex model ID:

MODEL_ID=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/models" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "gemini-prod",
    "model_name": "gemini-3.6-flash",
    "provider_key_id": "'"$PROVIDER_KEY_ID"'"
  }' | jq -r '.model.id')

❶ model_name is the Vertex publisher model ID. The current Gemini 3.6 Flash example is generally available in global. Google documents custom temperature, top-P, and top-K values as ignored for this model, so omit temperature and top_p from calls through AISIX. Requests to this model must also end with a user message. Google rejects a GenerateContent request whose final turn has the model role, and AISIX maps a final OpenAI assistant message to that role.

❷ provider_key_id attaches the model to the Vertex credential captured in the previous step.

Other supported examples include Claude on Vertex, OpenAI-compatible MaaS models such as Llama, and partner publisher models from Mistral and AI21.

AISIX chooses the Vertex route from model_name:

Model ID FamilyHow AISIX Sends It to Vertex
Gemini models, such as gemini-*Uses the Google Gemini publisher route.
Claude models, such as claude-*Uses the Anthropic publisher route with an Anthropic Messages body.
OpenAI-compatible MaaS models, such as Llama, DeepSeek, Qwen, GPT-OSS, MiniMax, Moonshot, or Z.aiUses Vertex's OpenAI-compatible chat-completions route.
Mistral and AI21 modelsUses the partner publisher route with an OpenAI-compatible body.

Create a Caller API Key​

Create the API key resource with access to the Vertex-backed model alias. The server generates the key and returns the plaintext once:

AISIX_API_KEY=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/api_keys" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "vertex-caller",
    "allowed_models": ["'"$MODEL_ID"'"]
  }' | jq -r '.plaintext')

allowed_models references the model by ID, so the caller can only use the Vertex-backed alias. Store the plaintext key securely; read endpoints do not return it again.

Configure with the Open-Source AISIX Gateway​

For the open-source gateway, set adapter: vertex and place the project, region, and credential mode in the provider key's api_key value. The example below reads the structured credential from an environment variable:

export VERTEX_CREDENTIAL='{"project":"my-gcp-project","region":"global","service_account_json":{"type":"service_account","private_key":"YOUR_PRIVATE_KEY","client_email":"vertex-sa@my-gcp-project.iam.gserviceaccount.com","token_uri":"https://oauth2.googleapis.com/token"}}'

Use the exported credential in the declarative resources file:

resources.yaml
_format_version: "1"

provider_keys:
  - display_name: vertex-prod
    provider: google-vertex
    adapter: vertex
    api_key: ${VERTEX_CREDENTIAL}
    api_base: https://aiplatform.googleapis.com

models:
  - display_name: gemini-prod
    provider: google-vertex
    model_name: gemini-3.6-flash
    provider_key: vertex-prod

api_keys:
  - display_name: vertex-caller
    key_env: VERTEX_CALLER_KEY
    allowed_models:
      - gemini-prod

The credential must contain project, region, and exactly one of access_token or service_account_json. Use the same model_name values described above for other Vertex publishers.

For a regional OSS configuration, api_base can be omitted and AISIX derives https://<region>-aiplatform.googleapis.com. Keep it explicit for region: global, because the correct origin is https://aiplatform.googleapis.com. Also set it explicitly for a proxy or private endpoint.

Export the caller key:

export VERTEX_CALLER_KEY="YOUR_CALLER_API_KEY"

If AISIX is installed locally, validate the file before loading it:

aisix validate --resources resources.yaml

After validation, start the gateway with the referenced environment variables in its process environment. Reload an existing gateway only if those variables are already available to the process; otherwise, restart it with the updated environment.

If you use Docker, adapt the validation and startup commands in the Open-Source AISIX Gateway Quickstart. Mount this resources.yaml file and pass every environment variable it references with -e in both commands. After the resources load, set AISIX_API_KEY for the shared verification request:

export AISIX_API_KEY="$VERTEX_CALLER_KEY"

Verify the Provider Connection​

Export the AISIX gateway origin:

# The local quickstarts use http://127.0.0.1:3000
export AISIX_PROXY="YOUR_AISIX_GATEWAY_ORIGIN"

Send a chat-completions request through the AISIX proxy. The example uses Gemini, which requires at least one user or assistant turn.

curl -sS -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini-prod",
    "messages": [
      {
        "role": "user",
        "content": "Say hello from Vertex."
      }
    ]
  }'

The gateway returns an OpenAI-compatible response with the caller-facing alias:

{
  "object": "chat.completion",
  "model": "gemini-prod",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello from Vertex!"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 4,
    "completion_tokens": 4,
    "total_tokens": 8
  }
}

Check Vertex logs, metrics, quota usage, or provider-side request records for the test request. If AISIX returns a token-minting or upstream authentication error, check the service-account key, region, Vertex API enablement, IAM role, and model access.

Endpoint and Content Support​

Vertex AI exposes more capabilities than the current AISIX vertex adapter translates. The gateway behavior depends on both the caller route and the publisher family selected by model_name.

RouteBehavior with a google-vertex provider key
/v1/chat/completionsSupported, including streaming. The current Gemini publisher translation is text-only; partner publisher behavior depends on its rail.
/v1/embeddingsSupported for Google-publisher text embedding models, such as gemini-embedding-001. AISIX sends text inputs to publishers/google/models/<model>:predict; partner and multimodal embedding formats are not supported. With gemini-embedding-001, send one input string per request. AISIX forwards an input array as one upstream request, but this model accepts only one input and can reject an array with multiple items.
/v1/responsesSupported through the chat-based Responses bridge, not a native Vertex Responses endpoint. Fields without a chat equivalent are ignored.
/v1/messagesSupported through translation to chat. /v1/messages/count_tokens is unavailable because the configured provider is google-vertex, including for Claude models hosted on Vertex.
/v1/completionsNot supported by the Vertex adapter.
/v1/images/generations, /v1/audio/*, /v1/videos, and /v1/rerankNot supported by the Vertex adapter or these routes' provider rules, even though Vertex offers separate native media services.
/v1/files, /v1/batches, and /v1/fine_tuning/jobsNot supported because the AISIX jobs surface does not accept the vertex adapter.
/passthrough/google-vertex/*Not a workaround for native Vertex APIs. Passthrough bypasses the adapter, so it neither mints OAuth tokens from the service-account JSON nor extracts an access token from the structured credential.

For Gemini publisher models, AISIX currently serializes text parts, system instructions, and basic generation settings. It does not serialize image, audio, or video parts; function declarations and tool results; Gemini-specific thinking controls; or thought signatures. Tool messages are reduced to user text, and non-text response parts are not returned.

Do not use the current Vertex adapter for Gemini function-calling loops. Gemini 3 requires applications to replay thought signatures during tool use, but this adapter neither returns nor accepts those signatures. This limitation also applies when Gemini is reached through the Responses or Messages bridge.

AISIX maps Vertex aggregate token totals into the normalized response. It does not expose Vertex's detailed thought-token or cached-content-token counters separately. Use Vertex billing and monitoring when that breakdown is required.

Vertex Publisher Routing​

Publisher selection is prefix-based. If model_name does not match a supported prefix, the gateway rejects the request before the provider request with an unsupported-publisher configuration error.

The example uses Gemini because it is the main Google publisher path. For partner models, validate the exact model ID, quota, and regional availability in your Vertex project before exposing the alias to callers.

A provider key has one project and location. Create another provider key when a partner model is available only in a different location; changing model_name does not change the location used in the Vertex resource path.

Provider-key request and response overrides can apply on Vertex routes, but they are most directly useful on OpenAI-compatible routes. Gemini's native contents format does not match every OpenAI-style override target.

Next Steps​

You have now connected AISIX to Google Vertex AI and verified the model alias. Continue with these guides: