AI Proxy | API7 Docs

Parameters

See plugin common configurations for configuration options available to all plugins.

auth
object
required
Authentication configurations.
- header
  object
  Authentication headers. At least one of the header and query should be configured.
- query
  object
  Authentication query parameters. At least one of the header and query should be configured.
model
object
required
Model specifications.
- provider
  string
  required
  vaild vaule:
  openai
  LLM service provider. Currently only openai is supported.
- name
  string
  required
  Name of the LLM model, such as gpt-4 or gpt-3.5. See your LLM provider API documentation for more available models.
- options
  object
  Model settings.
  max_tokens
  integer
  default: 256
  Maximum number of tokens (words, punctuation, or parts of words) in the generated response when working with completions endpoints. It determines the length of the output from the model.
  temperature
  number
  vaild vaule:
  between 0.0 and 5.0 inclusive
  Temperature used for sampling, which controls the randomness of the output. Higher values will make the model’s responses more creative and diverse, and lower values will make the responses more focused and deterministic.
  While the parameter accepts any value between 0 and 5, not every model has the same upper limit. Most OpenAI models have an upper limit of 2. please see your LLM provider API documentation to confirm the valid range.
  top_p
  number
  vaild vaule:
  between 0 and 1 inclusive
  Top-p probability mass. An alternative to sampling with temperature, where the model considers the results of the tokens with top-p probability mass. For example, setting top_p to 0.9 means only the top 90% probable tokens are considered.
  stream
  boolean
  default: false
  If true, stream response by SSE.
override
object
Override setting.
- endpoint
  string
  LLM provider endpoint to replace the default endpoint with. If not configured, the plugin uses the default OpenAI endpoint https://api.openai.com/v1/chat/completions.
passthrough
boolean
default: false
If true, relay the response from LLM to the upstream service. By default it is set to false, so the upstream node can be set to any arbitrary value.
timeout
integer
default: 3000
vaild vaule:
between 1 and 60000 inclusive
Request timeout in milliseconds when requesting the LLM service.
keepalive
boolean
default: true
If true, keep the conneciton alive when requesting the LLM service.
keepalive_timeout
integer
default: 60000
vaild vaule:
greater than or equal to 1000
Request timeout in milliseconds when requesting the LLM service.
keepalive_pool
integer
default: 30
Keepalive pool size for when connecting with the LLM service.
ssl_verify
boolean
default: true
If true, verify the LLM service's certificate.

Parameters​

Parameters