Traffic Controls
After the request path is working, traffic controls let you decide how that path should behave under real application traffic. They protect shared model capacity, enforce spend policy, block unsafe content, and cache repeat responses.
Some controls run before AISIX calls the provider. Others run after the provider returns a response. The request path looks like this:
Request-side controls can stop a request before AISIX calls the provider. Rate limits, budgets, and input guardrails decide whether the request is allowed to continue. Cache lookup can return a stored response on a hit; on a miss, AISIX calls the provider.
After the provider returns, output guardrails can inspect the response before AISIX returns it to the caller.
Choose a Control
Start with the control that represents the resource or policy you need to protect:
| Goal | Control | Where It Acts |
|---|---|---|
| Limit request volume, token usage, or concurrent work | Rate Limits | Before the provider call; token usage is recorded after the response reports it. |
| Cap managed AI spend for an organization, environment, caller API key, provider key, team, or member | Budgets | Before the provider call in managed deployments. |
| Detect, block, or mask sensitive or unsafe content | Guardrails | On request content, response content, or both. |
| Reuse eligible Chat Completions responses | Caching | Before the provider call; a cache hit returns without contacting the provider. |
Controls compose on the same request. For example, a Chat Completions request can authenticate a caller, resolve its model, evaluate input guardrails, check budgets, reserve rate-limit capacity, and then look for a cached response. Configure each control independently, then verify the combined path with the same caller API key and model alias the application will use.