Skip to main content
API7.ai Logo

Parameters​

See plugin common configurations for configuration options available to all plugins.

  • prompt

    string

    required

    The prompt to instruct the LLM on how to rewrite the client request.

  • provider

    string

    required

    vaild vaule:

    openai, deepseek, azure-openai, aimlapi, gemini, vertex-ai, anthropic, openrouter, openai-compatible

    LLM service provider.

    When set to openai, the plugin will proxy requests to https://api.openai.com/v1/chat/completions.

    When set to deepseek, the plugin will proxy requests to https://api.deepseek.com/chat/completions.

    When set to gemini (available from APISIX 3.15.0 and Enterprise 3.9.3), the plugin will proxy requests to https://generativelanguage.googleapis.com/v1beta/openai/chat/completions. If you are proxying requests to an embedding model, you should configure the embedding model endpoint in the override.

    When set to vertex-ai (introduced in API7 Enterprise 3.9.3 and APISIX 3.15.0), configure override.endpoint with the complete Vertex AI chat completions endpoint, including its path. Set auth.header.Authorization to Bearer ACCESS_TOKEN, using a Google Cloud OAuth 2.0 access token. The plugin does not support auth.gcp or use provider_conf to construct the endpoint.

    When set to anthropic (available from APISIX 3.15.0 and Enterprise 3.9.2), the plugin will proxy requests to https://api.anthropic.com/v1/chat/completions.

    When set to openrouter (available from APISIX 3.15.0 and Enterprise 3.9.2), the plugin will proxy requests to https://openrouter.ai/api/v1/chat/completions.

    When set to aimlapi (available from APISIX 3.14.0 and Enterprise 3.8.17), the plugin uses the OpenAI-compatible driver and proxies the request to https://api.aimlapi.com/v1/chat/completions.

    When set to openai-compatible, the plugin proxies requests to the custom endpoint configured in override.

    When set to azure-openai, the plugin also proxies requests to the custom endpoint configured in override and additionally removes the model parameter from user requests.

  • auth

    object

    required

    Authentication configurations.

    • header

      object

      Authentication headers. At least one of the header and query should be configured.

    • query

      object

      Authentication query parameters. At least one of the header and query should be configured.

  • options

    object

    Model configurations.

    In addition to model, you can configure non-streaming generation parameters, such as temperature and top_p, which are forwarded to the upstream LLM service. The plugin must receive a complete JSON response before it can replace the original request body.

    • model

      string

      Name of the LLM model, such as gpt-4 or gpt-3.5. See your LLM provider's API documentation for more available models.

  • override

    object

    Override setting.

    • endpoint

      string

      Complete LLM provider endpoint, including the path. Required when provider is openai-compatible. Configure it for vertex-ai because AI Request Rewrite does not use project and region fields to construct the endpoint.

  • timeout

    integer

    default: 30000

    vaild vaule:

    between 1 and 60000 inclusive

    Request timeout in milliseconds when requesting the LLM service.

  • keepalive

    boolean

    default: true

    If true, keep the connection alive when requesting the LLM service.

  • keepalive_pool

    integer

    default: 30

    Keepalive pool size for when connecting with the LLM service.

  • ssl_verify

    boolean

    default: true

    If true, verify the LLM service's certificate.

  • max_req_body_size

    integer

    default: 67108864

    vaild vaule:

    greater than or equal to 1

    Maximum request body size in bytes buffered into memory. Larger request bodies are rejected with HTTP 400 before the rewrite LLM is called. Introduced in API7 Enterprise 3.9.17 and 3.10.4, and APISIX 3.18.0.