AI Request Rewrite | APISIX & API7 API Gateway Docs

Parameters

See plugin common configurations for configuration options available to all plugins.

provider
string
required
vaild vaule:
openai, deepseek, azure-openai, aimlapi, gemini, vertex-ai, anthropic, openrouter, openai-compatible
LLM service provider.
When set to openai, the plugin will proxy requests to https://api.openai.com/chat/completions.
When set to deepseek, the plugin will proxy requests to https://api.deepseek.com/chat/completions.
When set to gemini (API7 Enterprise 3.9.3 and later), the plugin will proxy requests to https://generativelanguage.googleapis.com/v1beta/openai/chat/completions. If you are proxying requests to an embedding model, you should configure the embedding model endpoint in the override.
When set to vertex-ai (API7 Enterprise 3.9.3 and later), the plugin proxies requests to Google Cloud Vertex AI. For chat completions, the plugin will proxy requests to https://{region}-aiplatform.googleapis.com/v1beta1/projects/{project_id}/locations/{region}/endpoints/openapi/chat/completions. For embeddings, the plugin will proxy requests to https://{region}-aiplatform.googleapis.com/v1/projects/{project_id}/locations/{region}/publishers/google/models/{model}:predict. These require configuring provider_conf with project_id and region. Alternatively, you can configure override for a custom endpoint.
When set to anthropic (API7 Enterprise 3.9.2 and later), the plugin will proxy requests to https://api.anthropic.com/v1/chat/completions.
When set to openrouter (API7 Enterprise 3.9.2 and later), the plugin will proxy requests to https://openrouter.ai/api/v1/chat/completions.
When set to aimlapi, the plugin uses the OpenAI-compatible driver and proxies the request to https://api.aimlapi.com/v1/chat/completions by default. The aimlapi option is currently available in APISIX and will be supported in API7 Enterprise soon.
When set to openai-compatible, the plugin proxies requests to the custom endpoint configured in override.
When set to azure-openai, the plugin also proxies requests to the custom endpoint configured in override and additionally removes the model parameter from user requests.
auth
object
required
Authentication configurations.
- header
  object
  Authentication headers. At least one of the header and query should be configured.
- query
  object
  Authentication query parameters. At least one of the header and query should be configured.
- gcp
  object
  GCP service account authentication for Vertex AI. Available in API7 Enterprise from 3.9.3 and not in APISIX.
  service_account_json
  string
  GCP service account JSON content used for authentication. This can be configured using this parameter or by setting the GCP_SERVICE_ACCOUNT environment variable.
  max_ttl
  integer
  Maximum TTL for GCP access token caching, in seconds.
  expire_early_secs
  integer
  default: 60
  Number of seconds to expire the access token before its actual expiration time. This prevents edge cases where tokens expire during active requests.
options
object
Model configurations.
In addition to model, you can configure additional parameters and they will be forwarded to the upstream LLM service in the request body. For instance, if you are working with OpenAI, you can configure additional parameters such as temperature, top_p, and stream. See your LLM provider's API documentation for more available options.
- model
  string
  Name of the LLM model, such as gpt-4 or gpt-3.5. See your LLM provider's API documentation for more available models.
provider_conf
object
Provider-specific configuration. When provider is vertex-ai, one of provider_conf or override should be configured.
Available in API7 Enterprise from 3.9.3 and not in APISIX.
- project_id
  string
  required
  Google Cloud Project ID for Vertex AI.
- region
  string
  required
  Google Cloud Region for Vertex AI.
override
object
Override setting.
- endpoint
  string
  LLM provider endpoint. Required when provider is openai-compatible.
timeout
integer
default: 30000
vaild vaule:
greater than or equal to 1
Request timeout in milliseconds when requesting the LLM service.
keepalive
boolean
default: true
If true, keep the conneciton alive when requesting the LLM service.
keepalive_timeout
integer
default: 60000
vaild vaule:
greater than or equal to 1000
Request timeout in milliseconds when requesting the LLM service.
keepalive_pool
integer
default: 30
Keepalive pool size for when connecting with the LLM service.
ssl_verify
boolean
default: true
If true, verify the LLM service's certificate.

Parameters​

Parameters