Upstreams and Load Balancing

An Upstream defines the backend target(s) for a Service. It specifies the destination nodes, how the gateway distributes traffic among them (load balancing), and how it monitors their availability (health checks).

In API7 Enterprise, Upstreams are typically configured within a Service's upstream field, or can be managed as standalone objects referenced by multiple Services.

Load balancing algorithms

API7 Gateway supports multiple algorithms to distribute traffic across backend nodes.

Algorithm	Description	Best for...
roundrobin	Distributes requests sequentially across nodes based on their weights.	Most general-purpose applications.
chash	Uses consistent hashing (by URI, cookie, etc.) to map requests to the same node.	Stateful applications requiring session affinity.
least_conn	Routes requests to the node with the fewest active connections.	Workloads with varying request processing times.
ewma	Exponentially Weighted Moving Average (Peak EWMA). Selects nodes with the lowest predicted latency.	Dynamically responding to backend performance fluctuations.

Consistent Hashing Keys

When type is set to chash, the gateway hashes an attribute of each request and uses that hash to pick a backend node. Requests that produce the same value are consistently routed to the same node. Two fields control this:

• hash_on: where the value is read from. Defaults to vars.
• key: which field to read from that source.

The format that key accepts depends on hash_on. The gateway validates the combination when the upstream is saved and rejects an invalid key with a 400 error.

hash_on	key refers to	Accepted key values	Example
vars	An NGINX variable	Only one of: uri, server_name, server_addr, request_uri, remote_port, remote_addr, query_string, host, hostname, mqtt_client_id, or a query argument in the form arg_<name>. Write the name without the leading $.	remote_addr, arg_device_id
header	A request header	The header name only, without the http_ prefix (letters, digits, -, and _; case-insensitive).	X-Api-Key
cookie	A request cookie	The cookie name only, without the cookie_ prefix (letters, digits, -, and _).	session_id
consumer	The authenticated consumer	Not applicable — omit key. The gateway hashes on the consumer's name.	—
vars_combinations	One or more variables	Any string of $-prefixed variable references, resolved per request.	$request_uri$remote_addr

note

hash_on: vars accepts only the fixed set of variables listed above — not the full list of built-in variables. To hash on a variable outside that set, or on a combination of values, use hash_on: vars_combinations. Reference each variable with a $ prefix, for example $request_uri$remote_addr.

If key cannot be resolved for a request (for example, the named header or argument is absent), the gateway falls back to hashing on remote_addr.

Key fields

Field	Type	Description
type	string	Load balancing algorithm: roundrobin (default), chash, least_conn, or ewma.
nodes	array	List of {host, port, weight, priority} objects for static backend targets.
scheme	string	Protocol for communicating with the backend: http, https, grpc, or grpcs.
retries	integer	Number of retries when a request fails. Range: 0-65535.
timeout	object	Timeout settings: {connect, send, read} in seconds. Default: 60 for each.
retry_timeout	number	Timeout to continue retries. Default: 0 (disabled).
pass_host	string	pass (service host), node (node host), or rewrite (custom host).
keepalive_pool	object	Connection pool settings: {size, idle_timeout, requests}.
hash_on	string	Used with chash to select the hash source: vars (default), header, cookie, consumer, or vars_combinations. See Consistent Hashing Keys.
key	string	The value to hash on when type is chash. Required unless hash_on is consumer; its accepted format depends on hash_on. See Consistent Hashing Keys.

Service discovery

Instead of defining static nodes, you can use Service Discovery to dynamically fetch backend targets from a registry.

Field	Description
service_name	The service identifier in the registry (e.g., <registry_id>/public/group/my-service).
discovery_type	The discovery mechanism: kubernetes, nacos, or consul.

Health checks

The gateway performs health checks to identify and remove unhealthy nodes from the load balancing rotation.

• Active Health Checks: The gateway periodically sends heartbeat requests (HTTP or TCP) to the backend.
• Passive Health Checks: The gateway monitors real-time traffic to identify failing nodes (e.g., observing consecutive 5xx errors).

TLS to upstream

If your backend requires secure communication, you can configure the following fields:

• client_certificate: ID of the certificate used for client authentication (mTLS).
• ca_certificates: List of CA certificate IDs to verify the backend's certificate.
• tls_verify: If true, the gateway verifies the backend's certificate against the ca_certificates.

Next steps

• Configure Health Checks — monitor upstream availability with active and passive health checks.
• Service Discovery — dynamically resolve backend endpoints from service registries.
• Consumers and Credentials — manage API caller identities and authentication.