Improve Rate Limiting Accuracy Across Gateway Instances

When multiple gateway instances enforce one shared rate limit, the limit-count-advanced plugin keeps the counter in Redis. You can synchronize with Redis on every request (sync_interval: -1) or batch the synchronization (sync_interval: 0.1 or larger). Batching removes a Redis round trip from every request and is the practical choice at high traffic volumes. The cost is that each instance makes admission decisions on counter data that can be up to one sync interval old, which produces a bounded counting error.

This guide explains where the error comes from, how large it can get, and why the sliding window algorithm keeps the long-term error near zero. If your rate limits must stay accurate while delayed synchronization is enabled, use window_type: sliding.

How Delayed Synchronization Works

With a positive sync_interval, each gateway instance:

1. Serves requests from a local counter in shared memory, admitting a request when cached global remaining − local increments since the last synchronization > 0.
2. Every sync_interval seconds, flushes its local increments to Redis in one atomic operation and caches the returned global remaining quota.

Between synchronizations, instances cannot see each other's local increments. Every admitted request is eventually written to Redis, so the shared counter always converges to the true admitted total. However, each instance discovers other instances' consumption only at its next synchronization.

             ◀── sync_interval ──▶
Instance A ──●━━━━━━━━━━━━━━━━━━━●━━━━━━━━━━━━━━━━━━━●──▶
             ┆ admits against    ┆                   ┆
             ┆ its cached view   ┆                   ┆
Redis      ──●───────────────────●───────────────────●──▶
             ┆   A and B cannot see each other's     ┆
             ┆   admissions between sync points      ┆
Instance B ──●━━━━━━━━━━━━━━━━━━━●━━━━━━━━━━━━━━━━━━━●──▶

● = flush local increments to Redis, refresh the cached remaining

Where the Error Comes From

The staleness window produces a deterministic, bounded overshoot:

• Sustained traffic: each instance always sees its own admissions, so it is blind only to what the other instances admit within one sync interval. With N balanced instances, each receiving r requests per second, and a sync interval of s seconds, the total overshoot per window is about (N − 1) × r × s. The two-instance tests below match this: 1 × 400 × 0.2 = 80 extra requests per window.
• Worst case: a burst that reaches all instances within one sync interval of a fresh window can pass up to instances × count requests, because every instance starts from a full cached budget.

The error is a designed trade-off, not a defect: batching turns one Redis round trip per request into one per instance per interval. You can shrink the error by lowering sync_interval (minimum 0.1, and it must stay smaller than time_window), or eliminate the delayed-sync error entirely with sync_interval: -1 at the cost of per-request Redis latency.

Fixed Windows Repeat the Error Every Window

A fixed window resets the counter to zero at every window boundary. After each reset, every instance again starts from a full cached budget, so the overshoot repeats every window and is never repaid.

The following results come from load tests against two gateway instances behind a round-robin load balancer with a shared Redis, with traffic split evenly across the instances. Demand is a sustained 800 requests per second and sync_interval is 0.2; the 1-second-window test ran for 30 seconds and the 10-second-window test for 90 seconds. The numbers are observations under this workload and illustrate how each window type behaves:

Limit	Admitted with fixed window	Error
500 requests per 1 second	~579 requests per second, every window	+16% repeated every window
1000 requests per 10 seconds	1085–1087 requests, every window	+8.5% repeated every window

The traffic shape inside each window is also poor: the whole budget is consumed in a burst at the window start, then everything is rejected until the next boundary. In the 10-second test, the backend received about 1085 requests in the first 2 seconds of each window and nothing for the remaining 8 seconds.

admitted per second — fixed window, limit 1000 per 10 s

600 ┤ ▐█            ▐█            ▐█
400 ┤ ▐█            ▐█            ▐█
200 ┤ ▐█            ▐█            ▐█
  0 ┼ ▐█▁▁▁▁▁▁▁▁▁▁▁ ▐█▁▁▁▁▁▁▁▁▁▁▁ ▐█▁▁▁▁▁▁▁▁▁▁▁ ──▶
    └── window 1 ───┴── window 2 ───┴── window 3 ──
       1085 (+8.5%)   1086 (+8.6%)   1085 (+8.5%)
       burst, then 0/s for 8 s — repeated every window

Sliding Windows Absorb and Repay the Error

A sliding window weighs the previous window's admitted count into the current budget, so consumption history survives window boundaries. Any short-term overshoot is on the books: it reduces the next window's budget, and the long-run average converges to the configured limit.

The same tests with window_type: sliding:

Limit	Admitted with sliding window	Error
500 requests per 1 second	~445 requests per second	−10%, stays at or under the limit
1000 requests per 10 seconds	first window +8.2%, later windows within ±3.5%	+0.2% cumulative over the whole run

The traffic shape improves as well: instead of burst-and-starve cycles, the gateway admits a steady count / time_window rate — about 100 requests every second in the 10-second test. Sliding windows also avoid the classic fixed-window boundary problem, where up to twice the limit can pass in a short span straddling two windows.

admitted per second — sliding window, limit 1000 per 10 s

600 ┤
400 ┤
200 ┤
  0 ┼ ▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄ ──▶
    └── window 1 ───┴── window 2 ───┴── window 3 ──
       1082 (+8.2%)   1004 (+0.4%)    980 (−2.0%)
       a steady ~100/s; the first window's overshoot
       is repaid by the following windows

Why the 1-Second Test Sits 10% Under the Limit

In a sliding window, budget is released continuously as the previous window's requests age out. An instance only discovers newly released budget at its next synchronization, so budget released in the middle of an interval waits up to one full sync_interval before any instance can spend it. An instance that finds the window full at synchronization time also admits nothing until its next synchronization, even though budget keeps releasing during that pause.

Both effects scale with the ratio of sync_interval to time_window — the fraction of the window that a stale view covers:

sync_interval : time_window	Measured conservative bias
0.2 / 1 (20%)	about −10%
0.2 / 10 (2%)	about −1% per window, +0.2% cumulative

To keep a sliding window close to its limit, keep sync_interval a small fraction of time_window — around 5% or less. This bias errs on the safe side: in steady state, the backend receives slightly less than the limit rather than a repeated overshoot. A fresh window can still overshoot once, up to the instances × count bound, and sliding windows repay it in the following windows.

What Sliding Windows Do Not Change

Within a single fresh window — for example, a burst in the first seconds after a counter starts — both window types share the same bound. Each instance can admit up to the full count before its first synchronization, so a burst can pass up to instances × count requests. The difference appears afterward: a sliding window repays that overshoot in the following windows, while a fixed window repeats it every window.

For capacity planning with delayed synchronization enabled, size your backend for the worst case instances × count per window regardless of the window type.

Recommendations

Goal	Configuration
Keep limits accurate while delayed synchronization is enabled	window_type: sliding
Smooth, predictable backend load	window_type: sliding
Tighter short-term error	Lower sync_interval (minimum 0.1, smaller than time_window)
Keep sliding windows close to the limit	Keep sync_interval at or below about 5% of time_window
Exact enforcement, latency budget allows one Redis round trip per request	sync_interval: -1

Example configuration:

{
  "limit-count-advanced": {
    "count": 1000,
    "time_window": 10,
    "window_type": "sliding",
    "key": "consumer_name",
    "key_type": "var",
    "policy": "redis-cluster",
    "redis_cluster_name": "my-cluster",
    "redis_cluster_nodes": ["redis-node-1:6379", "redis-node-2:6379"],
    "sync_interval": 0.2,
    "rejected_code": 429
  }
}

Next Steps

• Configure Rate Limiting — set up basic and advanced rate limiting.
• limit-count-advanced plugin reference — all configuration options.