Graceful Shutdown and Draining

AISIX shuts down in stages so a load balancer can withdraw an instance without interrupting work the gateway has already accepted. The gateway becomes unready first, continues serving requests and long-lived sessions during the drain, then closes its listeners and finishes any snapshot-cache write. The same sequence applies to AISIX OSS and to gateways connected to AISIX Enterprise or AISIX Cloud.

Size both the gateway's drain window and the deployment platform's termination deadline. The drain window gives the traffic layer time to stop routing. The platform deadline gives the gateway time to finish requests, streams, Realtime sessions, idle connections, and snapshot persistence.

Understand the Shutdown Sequence

On SIGTERM or SIGINT, the gateway:

1. Answers /readyz with 503 Service Unavailable immediately, so the next health check withdraws it. /livez stays 200: the process is healthy and must not be restarted while it drains.
2. Keeps accepting new connections for at least shutdown.min_drain_secs, which defaults to 30 seconds.
3. Adds Connection: close to HTTP/1.1 responses and sends HTTP/2 clients a GOAWAY frame, so clients stop reusing the connection without interrupting in-flight work. A WebSocket upgrade, such as a Realtime handshake, still completes normally.
4. After the minimum window, waits without a deadline of its own for the in-flight count to reach zero, then stops accepting new connections. A streaming response counts as in flight until it ends, and so does an open Realtime WebSocket session (/v1/realtime), from its handshake until it closes and its usage is recorded.
5. Waits for the connections it has already accepted to finish. A request that arrives on one of them is served normally and answered with Connection: close, and a Realtime session opened on one keeps the gateway up until that session closes. An idle pooled connection is closed. A connection that has not sent a request yet is held until it sends one, downstream.idle_timeout_secs expires, or the client closes it.
6. On an etcd-backed or control-plane-connected gateway that keeps a snapshot cache, waits up to a further five seconds for a cache write still in flight to reach disk, then exits.

Point the health check for an external load balancer at /readyz, not at a bare TCP connect. A TCP check cannot observe readiness, so it sees only the listener closing, which is the event the drain window exists to delay. See Health Checks for the complete liveness and readiness contract.

Configure the Drain Window

Set min_drain_secs above the time the load balancer needs to stop routing after /readyz changes. For Kubernetes, this is periodSeconds x failureThreshold; for an external load balancer, use its check interval multiplied by its retry count.

config.yaml

shutdown:
  min_drain_secs: 30

The window is a minimum, not a shutdown deadline. After it elapses, the gateway still waits for the in-flight count to reach zero. Setting it to 0 drops the minimum window and is appropriate only when no health-check-driven system routes traffic to the instance.

Size the Platform Deadline

The gateway's wait for in-flight work has no deadline of its own. The deployment platform sets the real limit on the sequence: terminationGracePeriodSeconds in Kubernetes or TimeoutStopSec under systemd.

Size that limit above shutdown.min_drain_secs, plus the longest request, stream, or Realtime session you intend to preserve, plus the five-second snapshot-cache drain when persistence is enabled. Add operational margin and the duration of any preStop hook, because the hook consumes the same Kubernetes termination budget.

A Realtime session that remains open when the platform's grace period ends is closed with the process, the same as a stream.

Account for Idle and TLS Connections

With the default downstream.idle_timeout_secs: 0, a connection that is open but never sends a request keeps a draining gateway up until the client closes it or the platform stops the process. A connection that a load balancer opens in advance is one example.

Set downstream.idle_timeout_secs to bound that wait. On a TLS listener, the timeout applies after the TLS handshake completes. A connection that never completes its handshake is not bounded by it.

Finish the Snapshot-Cache Write

The snapshot-cache allowance is additional time because it starts only after in-flight requests finish. The five-second limit is fixed. If it expires, the gateway logs a warning and exits. Atomic file replacement prevents a partial snapshot, but the next process can restore an older snapshot or no snapshot.

Watch a Drain in the Logs

Use the drain log fields to distinguish unfinished requests from connections that remain open or continue arriving:

Message	Fields	Written
draining — /readyz now reports 503, still accepting new connections	min_drain_secs, in_flight, open_connections	Once, when the signal arrives.
still draining in-flight requests	in_flight, open_connections	Periodically, while the drain runs.
request arrived while draining	method, path, plus the request's own request_id, peer, and downstream_request_id	Per request that reaches the gateway after the signal.
accepted a new downstream connection while draining	peer, open_connections	Per connection accepted after the signal.

in_flight counts active requests, including streaming responses and open Realtime sessions. open_connections also includes idle pooled connections and health-probe connections. If in_flight remains above zero, AISIX is still finishing accepted work. If open_connections grows, another component is still opening connections to the draining instance.

For a failed request during a rollout, match request arrived while draining with accepted a new downstream connection while draining by peer. A matching accept record means the connection arrived after draining began; without one, the client reused an older pooled connection. Health checks can produce accept records but are excluded from the request-arrival record.

peer and downstream_request_id provide the arrival context described in Access Logs and Request Correlation. Match peer to the fronting proxy's connection record and downstream_request_id to its request record.

Next Steps

Use Production Deployment to prepare the rest of the gateway runtime for production. Use Troubleshooting when an instance does not become ready, finish draining, or reconnect after a rollout.