Skip to main content
API7.ai Logo
Version: 3.9.x

Route AI Traffic to Gemini Enterprise Agent Platform

Gemini Enterprise Agent Platform is the current name for the Google Cloud model platform previously documented as Vertex AI. It provides Gemini and partner models through Google Cloud authentication and regional endpoints. This guide routes Agent Platform traffic through API7 Gateway with the ai-proxy plugin. The released provider identifier remains vertex-ai.

Prerequisites​

  • Install Docker.

  • Install cURL to send requests to the services for validation.

  • Install jq to encode the service-account key for the Admin API request.

  • Have a running API7 Gateway instance.

  • Have a Google Cloud project with billing and the Agent Platform API enabled.

  • Create a token from the Dashboard and save it to an environment variable:

    export API_KEY=your-dashboard-token   # replace with your Dashboard token

  • Replace {gateway_group_id} with your gateway group ID. Use default if you are following the quickstart.

  • If you are following the Admin API examples, create or reuse a service in API7 Gateway. If you do not have one yet, follow Create or Reuse a Service, then save its ID to an environment variable:

    export SERVICE_ID=your-service-id         # replace with your service ID

Configure GCP Authentication​

Create a service account, grant it the Agent Platform User (roles/aiplatform.user) role, and create a JSON service-account key. API7 Gateway requires the downloaded key because it does not discover Application Default Credentials or Workload Identity Federation credentials.

Save the raw key for ADC and a JSON-encoded string for the Admin API request:

export GCP_SERVICE_ACCOUNT_JSON="$(cat /path/to/service-account.json)"
export GCP_SERVICE_ACCOUNT_JSON_STRING="$(jq -Rs . < /path/to/service-account.json)"

The gateway automatically handles OAuth2 token generation and caching from the service-account credentials.

Configure the AI Proxy for Agent Platform​

Create a route with the ai-proxy plugin:

curl -k "https://localhost:7443/apisix/admin/routes?gateway_group_id={gateway_group_id}" -X PUT \
  -H "X-API-KEY: ${API_KEY}" \
  --data-binary @- <<EOF
  {
    "id": "vertex-ai-route",
    "service_id": "$SERVICE_ID",
    "paths": ["/vertex-ai"],
    "plugins": {
      "ai-proxy": {
        "provider": "vertex-ai",
        "override": {
          "endpoint": "https://aiplatform.googleapis.com/v1beta1/projects/<GCP_PROJECT_ID>/locations/global/endpoints/openapi/chat/completions"
        },
        "auth": {
          "gcp": {
            "service_account_json": $GCP_SERVICE_ACCOUNT_JSON_STRING
          }
        },
        "options": {
          "model": "google/gemini-3.5-flash"
        }
      }
    }
  }
EOF

❶ Set the provider to vertex-ai and override its endpoint with the documented global Chat Completions URL. Replace <GCP_PROJECT_ID> with your project ID.

❷ Provide the GCP service account JSON. The gateway generates and caches OAuth2 tokens automatically.

❸ Set an Agent Platform model that supports Chat Completions on the global endpoint. The example uses google/gemini-3.5-flash, which supports the default Standard PayGo consumption mode on global.

The global endpoint is explicit because the released vertex-ai driver otherwise derives a regional hostname from provider_conf.region.

Validate the Configuration​

Send a chat completion request:

curl "http://127.0.0.1:9080/vertex-ai" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      { "role": "system", "content": "You are a mathematician." },
      { "role": "user", "content": "What is 1+1?" }
    ]
  }'

You should receive a response similar to the following:

{
  "object": "chat.completion",
  "model": "google/gemini-3.5-flash",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "1 + 1 = 2\n"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 11,
    "completion_tokens": 8,
    "total_tokens": 19
  }
}

To enable streaming responses, set "stream": true in the request body. Use the proxy-buffering plugin to disable NGINX proxy_buffering to avoid server-sent events (SSE) being buffered.

Next Steps​

You have connected API7 Gateway to Gemini Enterprise Agent Platform. See the Agent Platform model catalog and OpenAI-compatible Chat Completions documentation for supported models and request fields.