Upstreams and Load Balancing

An Upstream defines the backend target(s) for a Service. It specifies the destination nodes, how the gateway distributes traffic among them (load balancing), and how it monitors their availability (health checks).

In API7 Enterprise, Upstreams are typically configured within a Service's upstream field, or can be managed as standalone objects referenced by multiple Services.

Load balancing algorithms

API7 Gateway supports multiple algorithms to distribute traffic across backend nodes.

Algorithm	Description	Best for...
roundrobin	Distributes requests sequentially across nodes based on their weights.	Most general-purpose applications.
chash	Uses consistent hashing (by URI, cookie, etc.) to map requests to the same node.	Stateful applications requiring session affinity.
least_conn	Routes requests to the node with the fewest active connections.	Workloads with varying request processing times.
ewma	Exponentially Weighted Moving Average (Peak EWMA). Selects nodes with the lowest predicted latency.	Dynamically responding to backend performance fluctuations.

Key fields

Field	Type	Description
type	string	Load balancing algorithm: roundrobin (default), chash, least_conn, or ewma.
nodes	array	List of {host, port, weight, priority} objects for static backend targets.
scheme	string	Protocol for communicating with the backend: http, https, grpc, or grpcs.
retries	integer	Number of retries when a request fails. Range: 0-65535.
timeout	object	Timeout settings: {connect, send, read} in seconds. Default: 60 for each.
retry_timeout	number	Timeout to continue retries. Default: 0 (disabled).
pass_host	string	pass (service host), node (node host), or rewrite (custom host).
keepalive_pool	object	Connection pool settings: {size, idle_timeout, requests}.
hash_on	string	Used with chash: vars, header, cookie, consumer, or vars_combinations.
key	string	Required for chash. The hash key (e.g., $remote_addr or a header name).

Service discovery

Instead of defining static nodes, you can use Service Discovery to dynamically fetch backend targets from a registry.

Field	Description
service_name	The service identifier in the registry (e.g., <registry_id>/public/group/my-service).
discovery_type	The discovery mechanism: kubernetes, nacos, or consul.

Health checks

The gateway performs health checks to identify and remove unhealthy nodes from the load balancing rotation.

• Active Health Checks: The gateway periodically sends heartbeat requests (HTTP or TCP) to the backend.
• Passive Health Checks: The gateway monitors real-time traffic to identify failing nodes (e.g., observing consecutive 5xx errors).

TLS to upstream

If your backend requires secure communication, you can configure the following fields:

• client_certificate: ID of the certificate used for client authentication (mTLS).
• ca_certificates: List of CA certificate IDs to verify the backend's certificate.
• tls_verify: If true, the gateway verifies the backend's certificate against the ca_certificates.

Next steps

• Configure Health Checks — monitor upstream availability with active and passive health checks.
• Service Discovery — dynamically resolve backend endpoints from service registries.
• Consumers and Credentials — manage API caller identities and authentication.