Skip to main content
API7.ai Logo

Parameters​

See plugin common configurations for configuration options available to all plugins.

  • prompt

    string

    required

    The prompt to instruct the LLM on how to rewrite the client request.

  • provider

    string

    required

    vaild vaule:

    openai, deepseek, azure-openai, aimlapi, gemini, vertex-ai, anthropic, openrouter, openai-compatible

    LLM service provider.

    When set to openai, the plugin will proxy requests to https://api.openai.com/v1/chat/completions.

    When set to deepseek, the plugin will proxy requests to https://api.deepseek.com/chat/completions.

    When set to gemini (available from APISIX 3.15.0 and Enterprise 3.9.3), the plugin will proxy requests to https://generativelanguage.googleapis.com/v1beta/openai/chat/completions. If you are proxying requests to an embedding model, you should configure the embedding model endpoint in the override.

    When set to vertex-ai (available from APISIX 3.15.0 and Enterprise 3.9.3), the plugin proxies requests to Google Cloud Vertex AI. For chat completions, the plugin will proxy requests to https://{region}-aiplatform.googleapis.com/v1beta1/projects/{project_id}/locations/{region}/endpoints/openapi/chat/completions. For embeddings, the plugin will proxy requests to https://{region}-aiplatform.googleapis.com/v1/projects/{project_id}/locations/{region}/publishers/google/models/{model}:predict. These require configuring provider_conf with project_id and region. Alternatively, you can configure override for a custom endpoint.

    When set to anthropic (available from APISIX 3.15.0 and Enterprise 3.9.2), the plugin will proxy requests to https://api.anthropic.com/v1/chat/completions.

    When set to openrouter (available from APISIX 3.15.0 and Enterprise 3.9.2), the plugin will proxy requests to https://openrouter.ai/api/v1/chat/completions.

    When set to aimlapi (available from APISIX 3.14.0 and Enterprise 3.8.17), the plugin uses the OpenAI-compatible driver and proxies the request to https://api.aimlapi.com/v1/chat/completions.

    When set to openai-compatible, the plugin proxies requests to the custom endpoint configured in override.

    When set to azure-openai, the plugin also proxies requests to the custom endpoint configured in override and additionally removes the model parameter from user requests.

  • auth

    object

    required

    Authentication configurations.

    • header

      object

      Authentication headers. At least one of the header and query should be configured.

    • query

      object

      Authentication query parameters. At least one of the header and query should be configured.

    • gcp

      object

      GCP service account authentication for Vertex AI. Available in API7 Enterprise from 3.9.3 and not in APISIX.

      • service_account_json

        string

        GCP service account JSON content used for authentication. This can be configured using this parameter or by setting the GCP_SERVICE_ACCOUNT environment variable.

      • max_ttl

        integer

        Maximum TTL for GCP access token caching, in seconds.

      • expire_early_secs

        integer

        default: 60

        Number of seconds to expire the access token before its actual expiration time. This prevents edge cases where tokens expire during active requests.

  • options

    object

    Model configurations.

    In addition to model, you can configure non-streaming generation parameters, such as temperature and top_p, which are forwarded to the upstream LLM service. The plugin must receive a complete JSON response before it can replace the original request body.

    • model

      string

      Name of the LLM model, such as gpt-4 or gpt-3.5. See your LLM provider's API documentation for more available models.

  • provider_conf

    object

    Provider-specific configuration. When provider is vertex-ai, one of provider_conf or override should be configured.

    Available in API7 Enterprise from 3.9.3 and not in APISIX.

    • project_id

      string

      required

      Google Cloud Project ID for Vertex AI.

    • region

      string

      required

      Google Cloud Region for Vertex AI.

  • override

    object

    Override setting.

    • endpoint

      string

      LLM provider endpoint. Required when provider is openai-compatible.

  • timeout

    integer

    default: 30000

    vaild vaule:

    between 1 and 60000 inclusive

    Request timeout in milliseconds when requesting the LLM service.

  • keepalive

    boolean

    default: true

    If true, keep the connection alive when requesting the LLM service.

  • keepalive_timeout

    integer

    default: 60000

    vaild vaule:

    greater than or equal to 1000

    Keepalive timeout in milliseconds for idle connections to the LLM service.

  • keepalive_pool

    integer

    default: 30

    Keepalive pool size for when connecting with the LLM service.

  • ssl_verify

    boolean

    default: true

    If true, verify the LLM service's certificate.

  • max_req_body_size

    integer

    default: 67108864

    Maximum request body size in bytes buffered into memory. Larger request bodies are rejected. Available in API7 Enterprise from version 3.9.17 on the 3.9 line and from version 3.10.4 on the 3.10 line.