Skip to main content
API7.ai Logo
Version: 3.19.0

Proxy Vertex AI Requests

Vertex AI provides access to Google's Gemini models through an OpenAI-compatible API.

This guide shows how to integrate APISIX with Vertex AI using the ai-proxy plugin. With provider set to vertex-ai, you can configure your project and region through provider_conf without specifying a custom endpoint.

Prerequisite(s)​

  • Install Docker.
  • Install cURL to send requests to the services for validation.
  • Follow the Getting Started Tutorial to start a new APISIX instance in Docker or on Kubernetes.
  • Have a Google Cloud project with Vertex AI API enabled.
  • If you use the Admin API examples and authentication is enabled, export an admin-role key as ADMIN_API_KEY. Configure the key under deployment.admin.admin_key. The Docker quickstart disables authentication for local evaluation; keep it enabled in production.

Obtain a Vertex AI Service Account Key​

Create a service account and JSON key by following the Google Cloud service account documentation. Ensure the service account has permissions to call Vertex AI (for example, Vertex AI User).

Optionally save the service account JSON to an environment variable:

export GCP_SERVICE_ACCOUNT_JSON="$(cat /path/to/service-account.json)"

Create a Route to Vertex AI​

Create a route with the ai-proxy plugin as such:

curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
  -H "X-API-KEY: ${ADMIN_API_KEY}" \
  --data-binary @- <<EOF
{
  "id": "vertex-ai-chat",
  "uri": "/anything",
  "plugins": {
    "ai-proxy": {
      "provider": "vertex-ai",
      "provider_conf": {
        "project_id": "evident-xxx",
        "region": "us-central1"
      },
      "auth": {
        "gcp": {
          "service_account_json": "$GCP_SERVICE_ACCOUNT_JSON"
        }
      },
      "options": {
        "model": "google/gemini-2.5-flash"
      }
    }
  }
}
EOF

❶ Set the provider to vertex-ai and configure project_id and region.

❷ Replace with your service account JSON.

❸ Set a model supported by Vertex AI, for example google/gemini-2.5-flash.

Verify​

Send a request with the following prompts to the route:

curl "http://127.0.0.1:9080/anything" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      { "role": "system", "content": "You are a mathematician" },
      { "role": "user", "content": "What is 1+1?" }
    ]
  }'

You should receive a response similar to the following:

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "1 + 1 = 2\n"
      },
      "index": 0,
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "completion_tokens": 8,
    "extra_properties": {
      "google": {
        "traffic_type": "ON_DEMAND"
      }
    },
    "total_tokens": 19,
    "prompt_tokens": 11
  },
  "object": "chat.completion",
  "model": "google/gemini-2.5-flash",
  ...
}

Next Steps​

You have learned how to integrate APISIX with Vertex AI. See the Vertex AI documentation and Gemini models pages for more details.