Kafka Logger | API7 Docs

Parameters

See plugin common configurations for configuration options available to all plugins.

brokers
array
required
vaild vaule:
greater than 0
List of Kafka broker nodes.
- host
  string
  required
  The host of Kafka broker.
- port
  string
  required
  The port of Kafka broker.
- sasl_config
  string
  The SASL configuration of Kafka broker
  mechanism
  string
  default: PLAIN
  vaild vaule:
  PLAIN
  The mechanism of SASL configuration.
  user
  string
  required
  The user of SASL configuration.
  password
  string
  required
  The password of SASL configuration.
kafka_topic
string
required
Target topic to push the logs for organization.
producer_type
string
default: async
vaild vaule:
async or sync
Target topic to push the logs for organization.
required_acks
integer
vaild vaule:
-1 or 1
Number of acknowledgements the leader needs to receive for the producer to consider the request complete. This controls the durability of the sent records. See Kafka documentation for more information. acks=0 is not yet supported.
key
string
Key used for allocating partitions for messages.
timeout
integer
default: 3
vaild vaule:
greater than 0
Timeout for the upstream to send data.
meta_format
string
vaild vaule:
default or origin
Format to collect the request information. Setting to default collects the information in JSON format and origin collects the information with the original HTTP request. See the example for more details.
log_format
object
Custom log format in key-value pairs in JSON format. Support built-in variables in values.
You can also configure log format on a global scale using the plugin metadata, which configures the log format for all kafka-logger plugin instances. If the log format configured on the individual plugin instance differs from the log format configured on plugin metadata, the log format configured on the individual plugin instance takes precedence. See the example for more details.
include_req_body
boolean
default: false
If true, include the request body in the log. Note that if the request body is too big to be kept in the memory, it can not be logged due to NGINX's limitations.
include_req_body_expr
array
An array of one or more conditions in the form of APISIX expressions. Used when the include_req_body is true. Request body would only be logged when the expressions configured here evaluate to true.
max_req_body_bytes
integer
default: 524288
vaild vaule:

=1

Maximum request body allowed in bytes. Request bodies falling within this limit will be pushed to Kafka. If the size exceeds the configured value, the body will be truncated before being pushed to Kafka.
include_resp_body
boolean
default: false
If true, include the response body in the log.
include_resp_body_expr
array
An array of one or more conditions in the form of APISIX expressions. Used when the include_resp_body is true. Response body would only be logged when the expressions configured here evaluate to true.
max_resp_body_bytes
integer
default: 524288
vaild vaule:

=1

Maximum response body allowed in bytes. Response bodies falling within this limit will be pushed to Kafka. If the size exceeds the configured value, the body will be truncated before being pushed to Kafka.
cluster_name
integer
default: 1
vaild vaule:
greater or equal to 0
Name of the cluster. Used when there are two or more Kafka clusters. Only works if producer_type is set to async.
producer_batch_num
integer
default: 200
vaild vaule:
greater or equal to 1
The number of messages to send in one batch. Same as the batch_num parameter in lua-resty-kafka.
producer_batch_size
integer
default: 1048576
vaild vaule:
greater or equal to 0
The size of the TCP send buffer to use when sending data. Same as the batch_size parameter in lua-resty-kafka, but in bytes.
producer_max_buffering
integer
default: 50000
vaild vaule:
greater or equal to 1
Maximum buffer size. Same as the max_buffering parameter in lua-resty-kafka.
producer_time_linger
integer
default: 1
vaild vaule:
greater or equal to 1
Flush time. Same as the flush_time parameter in lua-resty-kafka, but in seconds.
producer_time_linger
integer
default: 1
vaild vaule:
greater or equal to 1
Flush time. Same as the flush_time parameter in lua-resty-kafka, but in seconds.
meta_refresh_interval
integer
default: 30
vaild vaule:
greater or equal to 1
Time interval to auto refresh the metadata. Same as the refresh_interval parameter in lua-resty-kafka, but in seconds.
name
string
default: kafka logger
Unique identifier of the batch processor. If you use Prometheus to monitor APISIX metrics, the name is exported in apisix_batch_process_entries.
batch_max_size
integer
default: 1000
vaild vaule:
greater than 0
The number of log entries allowed in one batch. Once reached, the batch will be sent to the logging service. Setting this parameter to 1 means immediate processing.
inactive_timeout
integer
default: 5
vaild vaule:
greater than 0
The maximum time in seconds to wait for new logs before sending the batch to the logging service. The value should be smaller than buffer_duration.
buffer_duration
integer
default: 60
vaild vaule:
greater than 0
The maximum time in seconds from the earliest entry allowed before sending the batch to the logging service.
retry_delay
integer
default: 1
vaild vaule:
greater or equal to 0
The time interval in seconds to retry sending the batch to the logging service if the batch was not successfully sent.
max_retry_count
integer
default: 60
vaild vaule:
greater or equal to 0
The maximum number of unsuccessful retries allowed before dropping the log entries.

Parameters​

Parameters