Thread-per-Core Workers
The AISIX gateway serves traffic from a pool of worker threads. Two startup settings control that pool: proxy.workers sets how many worker threads serve traffic, and proxy.thread_per_core selects how those workers share the proxy listener and the upstream connections. This page explains both settings and their defaults, shows how to verify which mode a running gateway uses, and covers the behaviors that follow from serving with independent workers.
Understand the Serving Modes
In thread-per-core serving, each worker is an independent thread with its own runtime, its own listener on proxy.addr, and its own pool of upstream connections. The workers share the proxy address through the SO_REUSEPORT socket option, and the kernel assigns each new client connection to one worker by hashing the connection's addresses and ports. A request is then accepted, processed, and answered on a single thread, and the upstream call it makes stays on that thread as well.
With proxy.thread_per_core: false, the gateway serves from one shared runtime instead: a single listener accepts connections, and the worker threads balance tasks among themselves through work stealing. A request can then migrate between threads during processing, for example when an upstream response arrives on a different thread from the one that started the request. Each migration costs a thread wake-up and a context switch. Thread-per-core serving keeps a request on one thread end to end, so those handoffs do not occur.
Thread-per-core serving is on by default on Linux and off by default elsewhere, because the connection spreading it relies on is Linux kernel behavior. In the validation benchmarks for this mode, it sustained 54-88% more requests per second than the shared runtime on a 4-worker x86 virtual machine, and 28-42% more on Arm (AWS Graviton) hardware. The p99 latency under load roughly halved in both cases. The difference is gateway throughput capacity; provider latency dominates end-to-end request time in either mode.
Configure the Worker Pool
Both settings live in the proxy block of the startup configuration:
proxy:
addr: "0.0.0.0:3000"
thread_per_core: true
workers: 4
| Field | Default | Description |
|---|---|---|
proxy.thread_per_core | true on Linux, false elsewhere | Serve from independent workers with per-worker listeners and upstream pools. Set false to serve from one shared runtime on any platform. |
proxy.workers | Parallelism available to the process | Number of worker threads that serve traffic, in either mode. Must be at least 1. |
Both settings are read once at process start. Changing either one takes effect at the next gateway restart.
Leave proxy.workers unset in most deployments. The default follows the parallelism actually available to the process, so a container CPU limit, a cgroup quota, or a taskset affinity mask sizes the pool without restating the count in configuration. A gateway limited to 4 vCPUs starts 4 workers even on a 16-core host. Set an explicit count only when the gateway should use fewer threads than it may run on, for example when it shares its CPU allowance with a sidecar.
AISIX rejects proxy.workers: 0 at startup, because zero workers would bind no listener at all. Omit the field to use the default instead.
Both fields follow the standard environment-override form, with double underscores between nested field names:
export AISIX_PROXY__THREAD_PER_CORE=false
export AISIX_PROXY__WORKERS=8
Deployments that inject all startup configuration through environment variables, such as Kubernetes installs, set the fields this way. See Environment Variables for the override mechanism.
Verify the Active Mode
The serving mode is visible without any dedicated endpoint.
At startup, a gateway in thread-per-core mode logs one aisix listening (http, thread-per-core) line per worker, each carrying its worker index, or (https, thread-per-core) when the proxy listener terminates TLS. The shared runtime logs a single aisix listening (http) line.
On a running process, list its threads:
ps -T -p "$(pgrep -x aisix)"
In thread-per-core mode, the worker threads are named tpc-0 through tpc-<N-1>, one per worker:
PID SPID TTY TIME CMD
23110 23110 ? 00:00:00 aisix
23110 23111 ? 00:00:00 tokio-runtime-w
23110 23112 ? 00:00:00 tokio-runtime-w
23110 23115 ? 00:00:41 tpc-0
23110 23116 ? 00:00:40 tpc-1
23110 23117 ? 00:00:41 tpc-2
23110 23118 ? 00:00:39 tpc-3
In this mode, a small control runtime carries the metrics listener, signal handling, and background export work. Its tokio-runtime-w threads appear alongside the workers and do not count toward proxy.workers. With thread_per_core: false, no tpc- threads exist, and all serving threads carry the default runtime thread name tokio-runtime-w.
Account for Per-Worker Connection Pools
In thread-per-core serving, each worker keeps its own pool of connections to upstream hosts. Pooled connections are never handed between workers: a worker that holds an idle connection to a provider reuses it, and a worker that does not opens its own.
Two sizing consequences follow:
upstream.pool_max_idle_per_hostapplies per worker. A gateway with 8 workers andpool_max_idle_per_host: 32can hold up to 256 idle connections to one provider host. To bound the process as a whole, divide the intended total by the worker count.- The idle upstream connections a process holds scale with the worker count. Account for that when a provider, a NAT gateway, or a corporate egress proxy limits connections per client.
The pool timeouts keep their meaning unchanged; see Tune the Upstream Connection Layer for the pool settings themselves.
Plan for Low-Concurrency Traffic
The kernel spreads client connections across workers, not individual requests. That spreading is even across many connections and uneven across few. Below about four client connections per worker, some workers can sit idle while their siblings carry several connections each. In that range, throughput can fall below the shared runtime, which balances individual requests instead of connections.
What counts is the number of connections the gateway itself accepts, not the number of end clients. Direct traffic from many clients sits far above the threshold. However, an L7 load balancer or ingress in front of the gateway, as Network and Security recommends for the caller-facing port, can pool requests from many clients onto a small number of gateway-side connections. Benchmarks that drive the gateway from a handful of persistent connections sit in the same range: a load test with 8 connections against 8 workers measures connection placement, not gateway capacity.
If the gateway receives its traffic over only a few long-lived connections, raise the number of connections the tier in front of it keeps toward the gateway, lower proxy.workers so that each worker still receives several connections, or set proxy.thread_per_core: false.
Understand Listener Sharing
Thread-per-core workers share one proxy address through the SO_REUSEPORT socket option. Two properties of that option are worth knowing when operating or auditing a gateway host.
Port conflicts still fail loudly at startup. Before binding its workers' listeners, the gateway probes the address with a regular bind. Starting a gateway while another process holds the address, including a second gateway, therefore fails at startup, exactly as a single-listener process does. The probe leaves one theoretical gap: two gateways started in the same instant can both pass their probe and bind the same address. Sequential restarts, as orchestrators perform them, never hit this window; avoid deliberately starting two gateways on one address at the same moment.
A same-user process can join the listener while the gateway runs. While the gateway is serving, any process running under the same effective user ID that itself sets SO_REUSEPORT can bind the proxy address and receive a share of new connections. The kernel restricts joining to the same effective user ID, so this is not a cross-user risk. It is a property of the mode worth remembering when auditing what runs on a gateway host: if some connections appear to bypass the gateway, check for another process bound to the proxy port, for example with ss -tlpn.
Next Steps
Continue with Network and Security to protect the proxy and metrics listeners, or return to Performance and Sizing to size CPU for your request volume.