Guardrail Behavior

Guardrails add content-policy checks before AISIX forwards supported gateway traffic, after an upstream responds, or at both points. Their shared runtime controls determine which routes and content are checked, whether a match blocks traffic, and what callers and operators observe.

Built-in keyword and PII guardrails evaluate content inside the gateway. Semantic screening calls a configured embedding model, while custom scripts run operator-defined logic. Provider integrations send extracted content to a moderation service and apply its decision. Choosing a Guardrail Provider compares these options.

All categories use the behavior described on this page. The failure settings normally decide what happens when an applicable check cannot complete; some stream and decode paths have stricter or fixed handling.

Guardrail Hook Point

The hook point controls where AISIX runs a guardrail and what happens when it blocks content:

Hook Point	What AISIX Checks	Effect When Blocked
input	The caller request before AISIX sends it upstream.	The upstream is not called.
output	Response content before AISIX returns it to the caller.	The caller does not receive the blocked response.
both	Both request and response content.	AISIX applies the same guardrail on both sides where the route supports it.

Input guardrails cover the text-bearing parts of standard inference routes. These include chat completions, completions, responses, messages, embeddings, image generation and editing, video generation, audio speech, and rerank requests. AISIX also scans optional audio transcription and translation prompts, Realtime text frames, MCP tools/call arguments, and A2A message text. The same input checks cover Batch and Fine-tuning request bodies and passthrough route bodies.

Output guardrails scan returned text from chat completions, completions, responses, messages, and audio transcription and translation. They also scan Realtime text frames, MCP tools/call results, Batch and Fine-tuning response bodies, and passthrough route bodies. A2A, embeddings, image generation and editing, video generation, audio speech, rerank, and Count Tokens run only the input hook. Realtime binary frames are relayed without a guardrail content check.

On native /v1/responses, the output scan covers message text and each tool call's name, arguments, and input. Hosted-tool results, such as mcp_call.output, File Search results, and Code Interpreter logs, are not scanned or masked, and neither are refusal parts or a hosted tool's code or action text.

Guardrails never scan tool-call IDs, in either direction, on /v1/chat/completions, /v1/messages, /v1/messages/count_tokens, /v1/responses, or a passthrough route carrying a recognized envelope. An ID is the id, call_id, tool_use_id, or tool_call_id that ties a tool call to its result, including a Gemini thought signature carried in it. Function names, arguments, and tool results are still scanned. A rule that matches only text inside a tool-call ID therefore does not fire.

On passthrough routes, guardrails attach through the passthrough_route attachment scope, alongside the caller's API key, team, and environment scopes. The scanned text follows the detected body envelope. Requests carrying a recognized LLM envelope (chat, completions, Responses) scan the extracted message or prompt text and ordinary JSON strings forwarded alongside those fields. Tool-call IDs and the other opaque handles that Guardrail Scan Coverage lists are left out. The extraction includes tool-call arguments and the content of Anthropic Messages traffic; Guardrail Scan Coverage lists what is read for each envelope.

If AISIX recognizes an envelope but cannot decode it, AISIX does not fall back to scanning the raw body. This prevents typed opaque media from being reinterpreted as text. For a passthrough route carrying a recognized Responses envelope, output scanning covers only recognized visible text, refusal, and tool fields. Known media, reasoning, and unknown output items remain opaque. Malformed recognized buffered output cannot be evaluated and follows the configured failure policy. Unknown Responses stream events remain opaque.

Opaque buffered traffic scans decoded JSON string values when the body is JSON and otherwise scans the textual body. An opaque SSE stream can scan only a bare JSON string. Other content-bearing payload shapes cannot be evaluated because AISIX cannot reliably associate their text with the preceding stream content.

The request is scanned before dispatch. A blocking verdict returns 422 without contacting the upstream. The built-in pii guardrail cannot write a mask back to a provider-native body, so a mask-action-only match is forwarded unchanged and recorded as described in Surfaces That Cannot Apply a Mask. Use a block action when that content must not reach the upstream. Guardrail kinds such as Presidio and Lakera instead return a blocking verdict when a maskable result has no write-back channel. One case overrides all this. A mask hit inside an Anthropic thinking block is forwarded unchanged for every guardrail kind, because the block is signed. See Reasoning and Thinking Content.

A non-streaming response is buffered in full before relay, and AISIX scans its eligible content. A blocking verdict returns 422 with the upstream body withheld. Streaming responses relay incrementally under the chain's streamed-output policy. Hold-back guardrails buffer frames, either windowed or in full, and release them only after the scanned text clears. A blocking verdict ends the stream with a terminal SSE error frame while held frames are dropped. Each surface uses its own protocol's error shape rather than one canonical frame. Chat completions carry content_filter. /v1/messages uses the nested Anthropic shape, with an Anthropic-legal error type and an error.code of content_filter or guardrail_unavailable. A passthrough route follows the protocol of the stream it relays: the Anthropic shape for an Anthropic Messages stream, and the chat completions shape otherwise. /v1/responses splits by upstream. Where the provider serves the Responses API natively, the held-back stream is refused with a 422 rather than a frame. Where AISIX bridges a provider that does not, the refusal is the flat Responses error event. See Guardrail Refusals.

Surfaces That Cannot Apply a Mask

Some surfaces relay content that AISIX has no channel to rewrite:

• Passthrough route requests and responses.
• /a2a messages.
• Batch and Fine-tuning request bodies and the upstream bodies returned for them.
• /v1/realtime text frames, in both directions.
• The /v1/videos prompt.

When a pii mask rule matches on one of these surfaces, the content passes unmodified and AISIX records the match on the usage record:

Enforcement mode	What AISIX records
block	An enforced hit with action mask_unsupported and the detector counts. guardrail_blocked stays false, and redacted_entity_counts does not count the match because nothing was redacted.
monitor	A monitor hit with action would_mask_unsupported in place of would_mask, because enforcing the rule would not mask anything either.

Use a block action where the matched content must not pass. Presidio, Lakera, AWS Bedrock, Alibaba Cloud AI Guardrails, and custom scripts do not forward a mask outcome on these surfaces; they refuse the content instead. The other surfaces that run guardrails rewrite masked content, with the exceptions described elsewhere on this page.

The Decision Endpoints Are Not Screened

Guardrails do not run on the decision endpoints, /v1/systemone and /v1/decisions. Neither input nor output checks apply, whatever is attached in the request's scope, and nothing is redacted. The usage events of these requests carry no guardrail evidence.

The Files Routes Are Not Screened

Guardrails do not apply to the Files API at all. POST /v1/files, GET /v1/files, GET /v1/files/{id}, DELETE /v1/files/{id}, and GET /v1/files/{id}/content run no input and no output guardrail check, whatever is attached in the request's scope and whatever fail_open says. An uploaded file leaves the gateway exactly as the caller wrote it, and a downloaded one reaches the caller exactly as the provider stored it. A keyword, PII, or remote guardrail that would refuse the same text in a chat message does not see it here.

Telemetry does not report the gap. A Files usage event carries no marker for the missing guardrail pass: the guardrail evidence fields remain empty. Attaching a fail-closed guardrail to the environment therefore produces no signal that these routes are uncovered. An empty guardrail evidence section on a Files event does not confirm guardrail coverage. Where uploaded content must be screened, screen it before it reaches the gateway.

/v1/batches and /v1/fine_tuning/jobs are not affected. Their request bodies are structured JSON the caller submits rather than an uploaded blob, and both those bodies and the responses are still scanned.

Reasoning and Thinking Content

Reasoning is split by direction, and the two halves are not symmetrical. Plan a PII or blocklist policy around the boundary rather than assuming a guardrail covers everything a reasoning-capable model touches.

Reasoning the caller sends is in scope. Replayed reasoning is caller-supplied text entering the model like any other, so it is scanned, and masked at the same slots:

Route	Scanned and masked
/v1/responses	A reasoning item's content[].text and its summary[].text
/v1/chat/completions	An assistant turn's reasoning_content
/v1/messages	A thinking block's text — scanned, but see the exception below

Anthropic thinking blocks are never rewritten. A mask-action hit inside a thinking block is forwarded unchanged. This holds for every guardrail kind, not only the built-in pii one, because rewriting the block would invalidate the signature the provider issued for it.

Block-action rules still fire on a thinking block, so use a block action where such content must not reach the upstream. A redacted_thinking block is a separate case. It carries only encrypted provider data and contributes no text to the scan, so no rule can match inside it. It is likewise relayed untouched.

Reasoning the model generates is out of scope. It is neither scanned nor masked on any route — not on /v1/chat/completions, /v1/messages, or /v1/responses, streaming or buffered.

This Narrowed the Enforcement Boundary

Generated reasoning used to be scanned incidentally on three paths: buffered /v1/messages, /v1/responses, and passthrough routes carrying one of those envelopes. It no longer is. A block rule whose term appears only inside model-generated reasoning, and nowhere in the answer the model returned, blocked those responses before and does not block them now. The visible answer is scanned exactly as before.

If you relied on that, move the rule to the input hook. Otherwise, accept that generated reasoning is outside the output guardrail's scope.

One exception is not worth relying on either. When the request does not match a recognized envelope, a buffered JSON response is handled as raw traffic, so its decoded string values can include reasoning. That is raw-content scanning, not reasoning-aware handling. A malformed recognized response does not fall back to this path. In a raw SSE stream, only a bare JSON string can be scanned.

Encrypted Reasoning Content Is Not Masked

On /v1/responses, a reasoning item can carry an encrypted_content field. That field holds encrypted provider data. AISIX forwards it verbatim, never scans it, and never rewrites it.

So masking the readable summary does not remove the sensitive text from the encrypted copy the provider receives. The plaintext the caller can see is masked; the provider still gets the item's encrypted payload intact. Treat a reasoning item's encrypted_content as content that leaves your boundary unmasked, and use a block action rather than a mask action where that is unacceptable.

How Keyword and PII Guardrails Read Structured Content

On /v1/chat/completions, /v1/completions, /v1/messages, /v1/messages/count_tokens, /v1/responses, and MCP tools/call, the built-in keyword and pii guardrails judge the same pieces of content that masking rewrites, one segment at a time. They do not judge one text joined from the whole request or response. A block rule, a monitor-mode would_mask, and an enforced mask therefore agree about what matched.

Each of the following is a segment of its own:

• The text of one message or one content block.
• Each string value inside tool-call arguments, tool input, the tool results the caller sends, and MCP tools/call results, including MCP structuredContent.
• Each object key and each number inside those structured payloads. AISIX judges them but cannot rewrite them without changing the payload's contract, so a pii mask hit on one blocks the request or response instead of masking it.
• A tool name. AISIX judges it but never rewrites it: a block rule refuses, and a mask hit is forwarded unchanged.

This decides what a pattern can match. An anchored pattern such as ^\d{4}$ matches one whole segment, so it matches the number in the arguments {"pin":1234}. A pattern cannot match across segments: across two messages, across a key and its value, or across a tool name and its arguments. Review rules written to match across those boundaries. Passthrough routes are outside this model and scan their extracted text as one piece.

On native /v1/messages, a non-streamed response's server-tool blocks (server_tool_use, mcp_tool_use, and *_tool_result) are not read by keyword and pii. A streamed response masks the input_json_delta of every block that emits one, while its tool-result blocks are not scanned or masked.

Two text blocks of one message are two segments, on /v1/chat/completions as on /v1/messages. A banned word or a phone number split across two text blocks, such as ["call 138001", "38000"], is neither blocked nor masked. When such a chat request goes to a provider that receives only plain message text, AISIX masks each block on its own and then joins the blocks, so that provider receives the same text as a provider that receives the blocks. redacted_entity_counts on the usage event counts each masked entity once.

Choose the Input Message Window

A guardrail's input_messages setting selects the part of a request available to its input check. It does not decide which roles the guardrail reads inside that window. Some kinds apply their own role selector afterwards, such as text_source on semantic screening and Azure Text Moderation.

input_messages	Window available to the input check
all	The whole request: system, developer, user, assistant, and tool messages. This is the default.
latest_turn	The messages after the last assistant message, with system and developer messages excluded: the current turn's user messages and the tool results answering them.

IDE and agent clients resend the whole conversation on every request. Under all, an old message remains available to the guardrail and can keep matching on later requests. latest_turn removes history the model has already answered before the guardrail applies its kind-specific text selection.

Under latest_turn, the tool results inside the window are the ones the current turn produced: tool messages on /v1/chat/completions, tool_result blocks on /v1/messages, and function_call_output items on /v1/responses.

On the Responses API, AISIX treats assistant messages, function_call, custom_tool_call, and reasoning items as assistant turns. It treats function_call_output and custom_tool_call_output items as tool results. When a tool-call item is inside the selected window, its name and arguments or input are available to the input check. A masking guardrail that reads the item can rewrite the arguments or custom-tool input, but not the structural tool name.

"Last assistant message" is measured over the messages other than system and developer messages. A trailing assistant message is therefore a prefill — text the caller wrote for the model to continue — and belongs to the current turn rather than closing it. For a request with no assistant message, the window is the whole request apart from its system and developer messages.

A developer message is OpenAI's newer name for a system message, and every guardrail treats it as one. Under latest_turn it stays outside the window wherever it appears, so it is neither scanned nor masked. Under all it is available to the input check; whether the guardrail reads it depends on the kind's text selection. The same applies to developer input items on the Responses API.

The setting applies to every guardrail kind, and to the input check only: hook_point: input, and the input side of both. It has no effect on output checks, which always read the whole response. In AISIX Cloud, setting latest_turn on a guardrail whose hook_point is output is refused with 400 INVALID_REQUEST rather than stored as a setting that could never take effect.

A Guardrail Rewrites Only Its Own Window

A guardrail can read and rewrite only messages in its window, and its kind may narrow that set further. Under latest_turn, a masking guardrail leaves the conversation history exactly as the client sent it. Keep all on a guardrail whose job is to mask PII across the whole conversation.

Either way, the skipped history is still forwarded to the model unchanged. latest_turn changes what AISIX makes available to the input check, not what AISIX sends.

One boundary is worth stating plainly. If a client keeps a blocked message in its history and resends it with no assistant reply between the two, that message remains in the current-turn window and can still be inspected by a guardrail that reads its role.

Configure it on the dashboard's guardrail form, where the Input scan choice — Whole request or Latest turn only — appears for the input and both hook points. Through the Admin API, set input_messages on the guardrail. In resources.yaml, update the corresponding entry in guardrails:

resources.yaml (guardrail entry)

guardrails:
  - name: block-secrets
    kind: keyword
    hook_point: input
    input_messages: latest_turn
    patterns:
      - kind: literal
        value: internal-project-codename

Guardrail Matching

After selecting a hook point, determine where the guardrail applies:

• In AISIX Cloud, create a guardrail and attach it to the environment or to selected model aliases, MCP servers, caller API keys, or teams. Attachments let one definition apply at several scopes with explicit priority.
• For the open-source AISIX gateway, declare the guardrail in resources.yaml and give it a guardrail_attachments entry. Both sources scope a guardrail the same way, so a guardrail the file declares but nothing attaches inspects no traffic.

Attach Before Enabling

A guardrail's attachments are the whole of its scope. One with no attachment is enforced nowhere, and that is a valid state rather than an error — the model, API key or team it was scoped to may since have been deleted, which removes the attachment naming it. The dashboard shows such a guardrail as Not attached, and the gateway logs a warning naming it once.

Two consequences worth planning for. Deleting the last attachment does not widen the guardrail; it silences it, so use that to take a rule out of service and keep enabled for whether the rule exists at all. And a guardrail created through the Admin API is attached to nothing until you POST an attachment — the dashboard does this for you when you pick a scope on the create form, but a direct API call does not.

When more than one guardrail applies to the same request, AISIX evaluates them as one chain. A blocking verdict from any guardrail blocks the request or response.

A model-alias scope and an MCP-server scope select things a single request never has both of: an MCP tool call resolves no model, and a model request routes to no MCP server. A model-scoped guardrail therefore never inspects MCP tool calls, and an MCP-server-scoped one never inspects model traffic. To guard both, attach the guardrail at the environment scope or add one attachment of each kind.

An A2A call resolves neither a model nor an MCP server. Only environment, caller API key, and team attachments can cover its message text; model and MCP-server attachments cannot.

If the same guardrail is attached through multiple matching scopes configured in AISIX Cloud, AISIX keeps one copy of that guardrail in the request chain. The highest-priority attachment wins. When priorities are equal, the more specific scope wins in this order: caller API key, team, then model alias or MCP server, then environment. Model alias and MCP server share a rank because no single request can match both.

Guardrail Scope Follows the Addressed Model

A model attachment matches the model the caller addressed. AISIX resolves that model once, before it picks a target, and uses it for the whole request. When the addressed model is a routing, semantic routing, or ensemble model, the group is the addressed model, so the group's own model attachment is the only model attachment that runs. Guardrails attached to its member models do not run for that request. They run for requests that address the member directly. This narrows the model dimension only: environment, caller API key, and team attachments match a group request exactly as they match any other.

For example, attach a keyword guardrail to member model m and put m in group g. A request to m is screened by that guardrail. A request to g is not screened by it, even when g dispatches the request to m. To screen traffic that arrives through the group, attach the guardrail to g.

A wildcard alias is not a group and does not behave this way. The wildcard model is itself the entry the caller addressed, so a guardrail attached to openai/* covers every request that model serves, including a request for openai/gpt-4o. An exact alias wins over a wildcard, so adding an openai/gpt-4o model later takes that name out of the wildcard's coverage: the request then addresses the new alias, and only the guardrails attached to it run.

Enforcement Modes

A guardrail's enforcement_mode controls what AISIX does after a guardrail detects matching content.

Choose the mode according to whether the policy is ready to affect traffic:

Mode	Policy match	When to use it
block (default)	An input match stops the request before the provider call. An output match withholds the response from the caller. Masking rules rewrite supported content.	Enforce a policy that you have already validated.
monitor	AISIX allows the request or response unchanged and records what the guardrail would have blocked or masked.	Measure false positives and tune a new policy against live traffic before enforcing it.

Detection-based guardrails can produce false positives and false negatives. Monitor mode lets you compare a policy's matches with representative traffic before allowing it to affect requests.

AISIX records each observation on the request's usage record as a monitor hit. A hit carries the guardrail name, the hook that matched, and what the guardrail would have done: would_block with a code-owned guardrail kind and outcome summary, would_mask with safe match counts, or would_mask_unsupported with safe match counts where the surface could not have applied the mask. See Surfaces That Cannot Apply a Mask. If evaluation was unavailable, the would_block summary also includes the bounded, code-owned failure tag, such as custom_timeout; it does not include the script's reason or screened content. Custom scripts use the fixed count name custom and the number of text segments that would change; script-provided count names and values are ignored.

AISIX Cloud stores usage record detail verbatim. Records created by older gateways can therefore retain legacy script-provided reasons, count names, and values, including values derived from request content or secrets. Upgrading the gateway protects new records but does not rewrite historical records; keep access to legacy usage data restricted accordingly.

Use these records to observe the hit rate, tune the policy, and then switch it to block. For a would-block observation, AISIX also writes this gateway log line at info level:

guardrail in monitor mode observed a violation; not blocking (enforcement_mode=monitor)

Streaming Output

When an enforcing guardrail covers the output hook, AISIX holds streamed response content until the relevant check can run. Depending on the guardrail kind, AISIX may hold the full response or inspect buffered windows. This prevents blocked or unmasked content from reaching the caller before the guardrail decision, and it lets AISIX detect spans that cross stream chunks.

A guardrail in monitor mode never causes streamed output to be held back. When every output guardrail runs in monitor mode, AISIX forwards the stream as it arrives. After the stream ends, AISIX runs the checks over the whole generated output and records any matches. No size cut-off applies to that scan, so a match anywhere in a long response is recorded. This includes streamed tool-call arguments, /v1/responses streams, and the transcript of a streamed audio transcription.

The buffer cap below does not apply to a monitor-only output chain. AISIX delivers and observes an oversized response instead of rejecting it. If any enforcing guardrail also covers the output hook, AISIX holds the stream so the enforcing chain can inspect it before delivery.

Guardrail kinds that expose stream buffer settings use these defaults:

{
  "max_buffer_bytes": 262144,
  "on_buffer_exceeded": "fail_closed"
}

For AISIX Cloud, add these fields inside the guardrail's config object. For the open-source AISIX gateway, add them directly to the guardrail entry in resources.yaml. fail_closed blocks oversized output. fail_open releases output that AISIX could not fully inspect or rewrite, so use it only when availability is more important than strict output enforcement. Guardrail kinds that do not expose these fields use the built-in cap and fail closed on overflow.

max_buffer_bytes counts the model-generated content held back: assistant text, reasoning, and tool-call arguments. SSE and JSON framing is not counted, so the same response reaches the cap at the same point on chat completions, /v1/messages, /v1/responses, and passthrough routes. On a passthrough stream AISIX does not recognize as chat, completions, or Responses, each frame's whole data: payload counts.

Frames that carry no generated content, such as pings, keep-alives, and empty deltas, add nothing to that count but still occupy memory. Each held stream is therefore also capped at 128 times max_buffer_bytes of raw bytes, 32 MiB at the default. Crossing either cap is the same overflow and follows on_buffer_exceeded. An ordinary token stream carries less framing than that and reaches the content cap first; a stream of Chinese text from OpenAI, for example, frames at about 75 times its content.

A chain applies the smallest max_buffer_bytes among its output guardrails that hold the whole response, and it refuses on overflow when any of those guardrails fails closed. Which guardrails count depends on their streaming mode; see Window Mode on Messages and Responses. Under fail_open, the held content is released unscanned and unmasked, the rest of the response streams without an output scan, and the usage record carries the bypass reason output_buffer_exceeded. Under fail_closed, the refusal records an enforced hit with action blocked_buffer_exceeded that names the guardrail whose cap was reached. Raise that guardrail's max_buffer_bytes to let larger responses through.

Window Mode on Messages and Responses

Azure Text Moderation, Alibaba Cloud Content Moderation, Alibaba Cloud AI Guardrails, and custom guardrails default to window mode. On /v1/chat/completions and passthrough routes, window mode releases the stream window by window as each window scans clean. /v1/messages and /v1/responses cannot release a stream window by window, so whenever the output chain can block or mask, they hold the whole response and deliver it only after the scan completes. The caller therefore receives the first byte later than with a token-by-token relay. This applies to window mode as well as buffer_full mode, and it is what lets a custom output guardrail judge a streamed /v1/messages response. A monitor-only chain still relays the stream live and scans it once at the end.

When a stream is held whole and no output guardrail in the chain uses buffer_full, the window-mode guardrails' own max_buffer_bytes and on_buffer_exceeded govern the hold. Several such guardrails fold the way buffer_full guardrails do: the smallest cap applies, and overflow fails closed unless every one of them fails open. Set both fields per guardrail, inside config in AISIX Cloud or on the guardrail entry in resources.yaml.

A chain with any buffer_full output guardrail takes its cap and overflow policy from its buffer_full guardrails alone, on every route, so the settings of a window-mode guardrail in the same chain neither tighten nor loosen it. In both cases, blocked_buffer_exceeded names the guardrail whose cap set the chain's.

On /v1/chat/completions, a chain whose output guardrails all use window mode applies the same folded cap to the streamed tool-call names and arguments it collects for scanning. Collecting more than the cap is an overflow and follows on_buffer_exceeded.

Frames AISIX Cannot Scan

This section applies only when an enforcing output guardrail holds the stream back. Without one, /v1/responses and /v1/messages relay the upstream frames verbatim.

AISIX resolves the buffered stream before checking or masking it, so both operations receive the same set of readable frames. A data: payload is readable when it is one JSON document, empty, or the [DONE] sentinel.

Upstream frame	What AISIX does
Several data: lines in one frame	Joins the lines with a newline, following the server-sent events specification, then scans and masks the complete payload.
Final frame without a terminator	Completes, processes, and delivers the frame when its payload is readable or carries no content to scan.
Terminated frame with a payload that is not one JSON document	Scans the raw text for a blocking match. If the text passes, AISIX drops the frame because a mask cannot be written back safely.
Final unterminated frame with an unreadable payload	Drops the frame before the checks run. Its text is neither released nor scanned.
Comment, keepalive, empty payload, or [DONE]	Preserves the frame because it carries no content to scan.

When AISIX drops content, the caller receives a stream with missing frames. A warn log records the hook, reason, and number of frames and bytes removed.

AISIX refuses the response rather than returning an empty 200 when removal leaves nothing it can scan. The same refusal occurs on /v1/messages when a held-back stream accumulates more than 1 MiB without a frame terminator. The failure tag is unscannable_body; a hold-back buffer limit instead uses output_buffer_exceeded. On /v1/responses, the refusal is a 422:

{
  "error": {
    "message": "response rejected: a guardrail could not evaluate it (unscannable_body)",
    "type": "content_filter",
    "code": "guardrail_unavailable"
  }
}

On /v1/messages the response has already started as a stream, so the same refusal arrives as a terminal SSE error event carrying the same message. That event uses the Anthropic envelope, so it carries invalid_request_error as error.type, with the same guardrail_unavailable code:

{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "code": "guardrail_unavailable",
    "message": "response rejected: a guardrail could not evaluate it (unscannable_body)"
  }
}

In both cases the request is recorded as guardrail-blocked. These refusals ignore the guardrail's failure-policy setting; see Failures While Holding a Stream.

Buffering Does Not Mean Every Byte Was Scanned

AISIX can preserve a valid JSON event that it does not model. The frame parses, but no scan or masking pass extracts text from it, so the bytes reach the caller unchanged. Treat buffered output guardrails as enforcement over response text AISIX recognizes, not as proof that every relayed byte was inspected.

Neighboring endpoints handle unreadable streamed content differently:

Endpoint	Behavior
/v1/chat/completions	Rebuilds every outgoing chunk from decoded events, so content the decoder cannot read is dropped rather than relayed unscanned. If an upstream answers a streaming request with plain JSON, the decoder yields no events; a configured streaming read budget can instead cause failover and eventually an error.
/v1/responses and /v1/messages	Follow the upstream response type. A plain JSON answer to a streaming request goes through the buffered scan and masking path.
Passthrough routes	Buffered responses follow recognized protocol fields, or the opaque-content rules for unrecognized traffic. Recognized streams scan their protocol carriers. An opaque SSE stream scans only a bare JSON string; other content-bearing payload shapes cannot be evaluated. Passthrough guardrails return a verdict but never rewrite the response, so allowed content is relayed unchanged.

When a Guardrail Cannot Complete Its Check

Two conditions can prevent a guardrail check from completing:

Failure	Affected guardrails	Examples
The screening backend does not return a decision	Guardrails that call a remote service, semantic screening, and custom scripts	Connection and provider errors, throttling, timeouts, a script exception, or an unreadable script verdict.
AISIX cannot present the content to the guardrail as delivered	Every guardrail kind, including keyword and pii	An unreadable Anthropic request, an invalid MCP tool result, or a plain-text audio transcript containing invalid UTF-8.

Remote guardrails include AWS Bedrock Guardrails, Azure AI Content Safety and text moderation, Alibaba Cloud Content Moderation and AI Guardrails, Lakera Guard, OpenAI Moderation, and Presidio.

The applicable failure policy governs both causes. Built-in keyword and PII guardrails never call a backend, but their policy still determines what happens when AISIX cannot present content to them.

fail_open Now Reaches keyword and pii

Earlier gateways treated fail_open as a no-op for those two kinds. A keyword or pii guardrail carrying fail_open: true — set at a time when the field did nothing — stops refusing bodies AISIX cannot read as soon as the gateway is upgraded, with no operator action and no control-plane change. Review those rows before you upgrade rather than after.

Which Hook Each Setting Governs

Setting	Check Side	Applies To	Default
fail_open	Input	Every kind	false
output_fail_open	Output	Every kind except keyword and pii	false
fail_open	Output	keyword and pii only	false

A kind that calls out carries its own output_fail_open for the output hook, which leaves fail_open governing the input hook. keyword and pii have no output_fail_open — they never call out — so their single fail_open governs both of their hooks, and sending either kind an output_fail_open is rejected.

In block mode, true releases traffic the guardrail could not check and records a bypass; false refuses it. Both sides therefore default to fail closed. Monitor mode and held streams have narrower exceptions described below.

In AISIX Cloud, fail_open is a top-level field on the guardrail and appears on the dashboard's create and edit forms for every kind. output_fail_open sits inside the guardrail's config object, and both the dashboard's forms and the Admin API offer it on every kind that has one. For the open-source AISIX gateway, both sit directly on the guardrail entry in resources.yaml.

What Happens to Content AISIX Cannot Read

For an unreadable request, MCP tool result, or non-streaming audio transcript, AISIX evaluates each applicable guardrail independently. At least one guardrail must both read that side of the exchange and fail closed on that side before AISIX refuses the content.

Applicable guardrails	Outcome	Usage record
At least one guardrail reads that side and fails closed	AISIX refuses the content with the failure tag unscannable_body.	Guardrail blocked.
Every guardrail that reads that side fails open	AISIX releases the content.	Bypass reason unscannable_body.
No guardrail reads that side	AISIX handles the content as it would with no guardrail.	No guardrail bypass or refusal is recorded.

The direction and failure policy must belong to the same guardrail. For example, an output-only fail-closed guardrail does not make an input-only fail-open guardrail refuse an unreadable request. Because hook_point defaults to both and failure behavior defaults closed, this distinction matters when either setting is changed explicitly.

Inference and audio routes return this refusal with HTTP 422. MCP returns HTTP 200 with an error-marked tool result so the agent can handle it as tool output. See Guardrail Refusals for the endpoint-specific envelopes.

Transcripts AISIX Could Not Decode

A non-streaming /v1/audio/transcriptions or /v1/audio/translations response can contain invalid UTF-8. AISIX scans a best-effort decode with replacement characters, but would otherwise relay the original bytes. The output failure policy determines whether those unreadable bytes can reach the caller:

Output guardrail state	Outcome
At least one applicable output guardrail fails closed	AISIX returns 422 with unscannable_body and records the request as guardrail-blocked.
Every applicable output guardrail fails open	If the readable text passes its checks, AISIX relays the original bytes and records a bypass.
No guardrail reads the output	AISIX relays the original bytes without a guardrail record.

The fail-closed response uses this error:

{
  "error": {
    "message": "response rejected: a guardrail could not evaluate it (unscannable_body)",
    "type": "content_filter",
    "code": "guardrail_unavailable"
  }
}

The readable part is scanned whenever an output guardrail applies, even if the original bytes are eventually relayed. A policy match in that text can therefore block the response independently, and one request can carry both guardrail_blocked and a bypass reason of unscannable_body. An unreadable request body differs: AISIX cannot present any of it to the guardrail, so no best-effort scan runs.

This behavior applies only to bytes that do not decode. A valid UTF-8 text, srt, or vtt transcript is unaffected. A json or verbose_json transcript that parses as JSON is also unaffected because the JSON parser cannot return invalid text. AISIX checks the bytes the provider returned, not the requested response_format. A body that does not parse as JSON follows the plain-text path regardless of the requested format.

How Monitor Mode Handles Failures

Monitor mode changes a guardrail's own verdict, not failures AISIX raises while preparing or rewriting content:

Failure	Monitor-only chain
Screening backend fails	AISIX serves the request. A fail-closed evaluation is recorded as would_block; a fail-open evaluation records a bypass.
Unreadable request, MCP result, or non-streaming transcript	The applicable failure policy still decides whether AISIX refuses or releases the content.
Held stream exceeds its buffer, leaves nothing AISIX can scan, or cannot accept a mask	Cannot occur because monitor mode does not hold or rewrite output. If an enforcing guardrail shares the chain, that guardrail can trigger the refusal.

To make an unavailable screening backend refuse traffic, use block mode with the applicable failure policy set to fail closed. See Guardrail Refusals for the caller-visible failures.

Failures While Holding a Stream

Held streams use a stricter rule than the unreadable-content outcomes above.

Once an enforcing output guardrail holds a stream back, AISIX refuses these outcomes regardless of the applicable failure policy:

Failure tag	Why AISIX refuses
unscannable_body	Nothing scannable remains in the held stream.
output_buffer_exceeded	Releasing the overflow would expose buffered content that was never fully inspected or rewritten. on_buffer_exceeded: fail_open is the one setting that releases it instead.
mask_writeback_failed	AISIX cannot safely apply the requested mask to the response body.

keyword uses the default hold-back policy and pii defines its own, so an enforcing output guardrail of either kind can reach these outcomes. A monitor-only row cannot because it never holds output back.

MCP tool results and non-streaming audio transcripts remain governed by the normal output failure policy because they are not held-back streams. When that policy allows traffic without a complete check, AISIX records the bypass as described in Caller Response and Telemetry.

Caller Response and Telemetry

When a guardrail blocks proxy traffic, AISIX preserves the error convention of the requested endpoint:

• OpenAI-compatible routes return 422 with content_filter.
• Passthrough routes return 422 with the same OpenAI-style content_filter envelope.
• Anthropic-compatible non-streaming errors return 422 with invalid_request_error and an error.code of content_filter for a policy match or guardrail_unavailable for a fail-closed refusal.
• Anthropic-compatible streaming responses surface the block as an Anthropic SSE error frame with the same error.type and error.code.
• Realtime sessions surface the block as a WebSocket error frame followed by a close, since there is no status line left to carry it.
• A2A calls return 422 with a JSON-RPC error envelope.

For the exact envelope on each surface, and for the failure-tag vocabulary a fail-closed refusal names in its message, see Guardrail Refusals.

MCP guardrail blocks follow the MCP protocol shape instead. AISIX returns HTTP 200 with a JSON-RPC result flagged isError, not a protocol error, so the calling agent reads the refusal as tool output and can adapt. See Headers and Error Codes.

Guardrail Metrics

AISIX records each timed guardrail execution in the aisix_guardrail_latency_seconds Prometheus histogram. Labels identify the guardrail name, kind, phase, and result, including monitor-mode would_block/would_mask/would_mask_unsupported outcomes and fail-open bypassed results. On gateways with this behavior, the error_type label carries the bounded failure cause for a fail-closed blocked, a bypassed, and a failed monitor-mode would_block. Ordinary policy matches use none. The result label stays blocked for a fail-closed refusal rather than taking a value of its own, so filter on error_type to separate an outage from a policy decision. Streaming window mode can record several executions for one output response. Synchronous per-field PII and keyword masking operations are not timed by this histogram. Use it to track per-guardrail latency (P50/P95/P99), compare local and remote detection, and monitor block and bypass rates. See Metrics Reference for labels and PromQL examples.

To see guardrail cost in the time a client waits for its first streamed frame, compare the side="downstream" series of aisix_request_ttft_seconds with its upstream series. See Separate Gateway Wait from Upstream TTFT.

The Bypass Reason

A guardrail that fails open does not block the request or response. AISIX records the reason in guardrail_bypassed_reason, shown in AISIX Cloud as Bypass reason. Every proxied route's usage event carries the field; retroactive Batch billing rows do not because no gateway request or guardrail chain produced them.

The value is either the guardrail kind's failure tag, such as lakera_timeout, bedrock_5xx, or custom_script_error, or one of the gateway-raised unscannable_body and output_buffer_exceeded. The same tag identifies a fail-closed refusal and a fail-open bypass. AISIX limits tags to 64 lowercase letters, digits, and underscores. If a request bypasses more than once, the first reason wins.

Read the bypass reason together with the block state:

Fields	Meaning
Bypass reason only	At least one applicable check failed open and traffic proceeded.
Bypass reason and guardrail_blocked	One check failed open, but another check or hook later blocked the request. Some content may already have reached the provider unscreened.
Empty bypass reason	No recorded fail-open event. This does not prove every relayed byte was inspected.

Two paths scan a best-effort text decode without consulting the failure policy when a non-JSON body contains invalid UTF-8. Neither records unscannable_body, and a fail-closed guardrail does not change that behavior:

Path	Behavior
Batch or Fine-tuning response	An applicable output hook scans the best-effort decode, then AISIX can relay the original bytes.
Opaque non-JSON passthrough route request or buffered response	An applicable hook scans the best-effort decode, then AISIX can relay the original body.

A malformed recognized request or buffered response does not use this exception. AISIX applies the configured failure policy and records unscannable_body when that policy fails open.

One shape that looks like it belongs on that list does not. A multipart prompt part that is not valid UTF-8, on /v1/audio/transcriptions, /v1/audio/translations, or /v1/images/edits, is rejected with 400 before any guardrail runs. This applies regardless of guardrail attachment or fail_open. It is a structural check on the request, not a guardrail decision, so it neither refuses as a guardrail nor records a bypass.

A held-back stream that outgrows max_buffer_bytes under on_buffer_exceeded: fail_open also releases buffered content without inspection or masking and skips the end-of-stream scan. It is recorded as a bypass with the gateway-raised reason output_buffer_exceeded.

The two Prometheus signals cover different events. aisix_guardrail_bypasses_total counts both guardrail executions that ended in a bypass and proxy-raised unscannable_body and output_buffer_exceeded passes. The result="bypassed" slice of aisix_guardrail_latency_seconds counts only timed guardrail executions. A backend failure therefore appears in both metrics and the usage record, while a proxy-raised pass appears in the bypass counter and usage record but not the latency histogram.

Next Steps

Choose a provider or configure an in-gateway policy next:

• Choosing a Guardrail Provider: compare built-in and remote guardrail options.
• Built-in Keyword Guardrails: create an in-gateway blocklist.
• PII Detection and Redaction: detect, mask, or block sensitive data inside AISIX.