# Upstreams and Load Balancing An **Upstream** defines a Service's backend nodes, how the gateway distributes requests among them, and how it checks their health. In API7 Gateway, each upstream belongs to a service. If several services use the same backend, configure an upstream for each service. The Admin API manages a service's upstreams at `/apisix/admin/services/{service_id}/upstreams` in the owning gateway group. ## Load Balancing Algorithms[​](#load-balancing-algorithms "Direct link to Load Balancing Algorithms") API7 Gateway supports multiple algorithms to distribute traffic across backend nodes. | Algorithm | Description | Best for... | | ------------ | --------------------------------------------------------------------------------------------------- | ----------------------------------------------------------- | | `roundrobin` | Distributes requests sequentially across nodes based on their weights. | Most general-purpose applications. | | `chash` | Uses consistent hashing (by URI, cookie, etc.) to map requests to the same node. | Stateful applications requiring session affinity. | | `least_conn` | Routes requests to the node with the fewest active connections. | Workloads with varying request processing times. | | `ewma` | Exponentially Weighted Moving Average (Peak EWMA). Selects nodes with the lowest predicted latency. | Dynamically responding to backend performance fluctuations. | ### Consistent Hashing Keys[​](#consistent-hashing-keys "Direct link to Consistent Hashing Keys") When `type` is set to `chash`, the gateway hashes an attribute of each request and uses that hash to pick a backend node. Requests that produce the same value are consistently routed to the same node. Two fields control this: * `hash_on`: where the value is read from. Defaults to `vars`. * `key`: which field to read from that source. The format that `key` accepts depends on `hash_on`. The gateway validates the combination when the upstream is saved and rejects an invalid `key` with a `400` error. | `hash_on` | `key` refers to | Accepted `key` values | Example | | ------------------- | -------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------ | | `vars` | An NGINX variable | Only one of: `uri`, `server_name`, `server_addr`, `request_uri`, `remote_port`, `remote_addr`, `query_string`, `host`, `hostname`, `mqtt_client_id`, or a query argument in the form `arg_`. Write the name **without** the leading `$`. | `remote_addr`, `arg_device_id` | | `header` | A request header | The header name only, without the `http_` prefix (letters, digits, `-`, and `_`; case-insensitive). | `X-Api-Key` | | `cookie` | A request cookie | The cookie name only, without the `cookie_` prefix (letters, digits, `-`, and `_`). | `session_id` | | `consumer` | The authenticated consumer | Not applicable — omit `key`. The gateway hashes on the consumer's name. | — | | `vars_combinations` | One or more variables | Any string of `$`-prefixed variable references, resolved per request. | `$request_uri$remote_addr` | note `hash_on: vars` accepts only the fixed set of variables listed above — **not** the full list of [built-in variables](https://docs.api7.ai/api7-gateway/reference/built-in-variables.md). To hash on a variable outside that set, or on a combination of values, use `hash_on: vars_combinations`. Reference each variable with a `$` prefix, for example `$request_uri$remote_addr`. If `key` cannot be resolved for a request (for example, the named header or argument is absent), the gateway falls back to hashing on `remote_addr`. ## Key Fields[​](#key-fields "Direct link to Key Fields") | Field | Type | Description | | ---------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `type` | string | Load balancing algorithm: `roundrobin` (default), `chash`, `least_conn`, or `ewma`. | | `nodes` | array | List of `{host, port, weight, priority}` objects for static backend targets. | | `scheme` | string | Protocol for communicating with the backend: `http`, `https`, `grpc`, or `grpcs`. | | `retries` | integer | Number of retries when a request fails. Range: 0-65535. | | `timeout` | object | Timeout settings: `{connect, send, read}` in seconds. Default: 60 for each. | | `retry_timeout` | number | Timeout to continue retries. Default: 0 (disabled). | | `pass_host` | string | `pass` (service host), `node` (node host), or `rewrite` (custom host). | | `keepalive_pool` | object | Connection pool settings: `{size, idle_timeout, requests}`. | | `hash_on` | string | Used with `chash` to select the hash source: `vars` (default), `header`, `cookie`, `consumer`, or `vars_combinations`. See [Consistent Hashing Keys](#consistent-hashing-keys). | | `key` | string | The value to hash on when `type` is `chash`. Required unless `hash_on` is `consumer`; its accepted format depends on `hash_on`. See [Consistent Hashing Keys](#consistent-hashing-keys). | Each timeout must be positive. API7 Gateway 3.10.6 and later accept fractional seconds; earlier patches require positive whole seconds. The schema neither enforces a `0.001`-second floor nor rounds smaller positive values up to it. A [route's `timeout`](https://docs.api7.ai/api7-gateway/key-concepts/services-and-routes.md#route) overrides the upstream timeout for that route. ## Service Discovery[​](#service-discovery "Direct link to Service Discovery") Instead of defining static `nodes`, you can use Service Discovery to dynamically fetch backend targets from a registry. | Field | Description | | ---------------- | --------------------------------------------------------------------------------------- | | `service_name` | The service identifier in the registry (e.g., `/public/group/my-service`). | | `discovery_type` | The discovery mechanism: `kubernetes`, `nacos`, or `consul`. | ## Health Checks[​](#health-checks "Direct link to Health Checks") The gateway performs health checks to identify and remove unhealthy nodes from the load balancing rotation. * **Active Health Checks**: The gateway periodically sends heartbeat requests (HTTP or TCP) to the backend. * **Passive Health Checks**: The gateway monitors real-time traffic to identify failing nodes (e.g., observing consecutive 5xx errors). ## TLS to Upstream[​](#tls-to-upstream "Direct link to TLS to Upstream") If your backend requires secure communication, you can configure the following fields: * `client_certificate`: ID of the certificate used for client authentication (mTLS). * `ca_certificates`: List of CA certificate IDs to verify the certificate presented by the backend. * `tls_verify`: If true, the gateway verifies the certificate presented by the backend against the `ca_certificates`. From API7 Enterprise version 3.10.9, when no `ca_certificates` are referenced, the gateway verifies it against its `apisix.ssl.ssl_trusted_certificate` setting instead. ## Next Steps[​](#next-steps "Direct link to Next Steps") * [Configure Health Checks](https://docs.api7.ai/api7-gateway/how-to-guides/traffic-management/health-check.md) — monitor upstream availability with active and passive health checks. * [Service Discovery](https://docs.api7.ai/api7-gateway/key-concepts/service-discovery.md) — dynamically resolve backend endpoints from service registries. * [Consumers and Credentials](https://docs.api7.ai/api7-gateway/key-concepts/consumers-and-credentials.md) — manage API caller identities and authentication.