Qwen3Guard Guardrails

Qwen3Guard is an open-source safety classifier published by the Qwen team under the Apache 2.0 license. It reads a conversation and answers with a safety label, the unsafe categories it found, and, when it evaluates a model response, whether that response refused the request. You serve the model yourself, so the screened text stays inside your own network.

AISIX has no built-in guardrail kind for Qwen3Guard. You reach it through the custom script guardrail, which runs a short JavaScript module inside the gateway: the script calls your Qwen3Guard deployment, reads its answer, and returns a verdict. Nothing else has to be deployed between the gateway and the model.

The same procedure applies to any guard model you serve over an OpenAI-compatible endpoint. Only two parts of the script are specific to Qwen3Guard: how the conversation is submitted, and how the answer is parsed. See Adapt the Script to Another Guard Model.

In this guide, you will deploy Qwen3Guard, write the script that adapts it to AISIX, create the guardrail, and verify that both a prohibited prompt and a prohibited model response are refused.

Prerequisites

Before starting, prepare the following:

Review Guardrail Behavior for hook points, enforcement modes, and failure policies, and Custom Script Guardrails for the script contract this guide builds on.
A host that can serve the guard model. A GPU is not required for the smallest variant, but inference runs on every screened request, so size the deployment for your request rate. See Choose a Qwen3Guard Model.
One of these configuration paths:
- AISIX Cloud with an environment, an attached gateway, and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
- An open-source AISIX gateway that loads a declarative resources.yaml file.
A working model alias and caller API key that can send Chat Completions requests.
Docker or a Kubernetes cluster to run the guard model, and curl. The AISIX Cloud path also uses jq.

Export the gateway values used by the verification requests:

# AISIX_PROXY has no trailing slash or endpoint path.
# The local quickstarts use http://127.0.0.1:3000.
export AISIX_PROXY="YOUR_AISIX_GATEWAY_URL"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="YOUR_MODEL_ALIAS"

Choose a Qwen3Guard Model

Qwen3Guard ships in two families. Only the Gen family is usable as a gateway guardrail:

Family	Sizes	How it works	Use with AISIX
Qwen3Guard-Gen	0.6B, 4B, 8B	Classifies a complete prompt or response and answers in text. Served by any OpenAI-compatible runtime.	Yes. A guardrail script calls it over HTTP.
Qwen3Guard-Stream	0.6B, 4B, 8B	Classifies each generated token through a classification head, and needs the token stream of the model being screened.	No. It has no chat-completions interface to call, and it expects token IDs from the generating model rather than text.

Within the Gen family, size the model against the latency your callers tolerate, because the guardrail call happens inline:

Model	When to choose it
`Qwen/Qwen3Guard-Gen-0.6B`	Latency-sensitive traffic, CPU-only hosts, and evaluation. The smallest quality margin of the three.
`Qwen/Qwen3Guard-Gen-4B`	The usual starting point. Noticeably better judgment than 0.6B at a size that fits one mid-range GPU.
`Qwen/Qwen3Guard-Gen-8B`	Policies where a missed detection is expensive and the extra latency is acceptable.

All three accept a 32,768-token context and classify content in more than one hundred languages, so one deployment covers multilingual traffic.

Deploy Qwen3Guard

Serve the model with any runtime that exposes an OpenAI-compatible /v1/chat/completions endpoint. With vLLM:

vllm serve Qwen/Qwen3Guard-Gen-4B \
  --port 8000 \
  --max-model-len 32768 \
  --api-key "YOUR_GUARD_API_KEY"

The guard model answers with a safety verdict for whatever text it is given, so anything that can reach it can submit content to it. Deploy it on a private network reachable by your gateways, and keep --api-key set so a reachable endpoint still requires a credential. The script reads that credential from the guardrail's secrets rather than carrying it in the script body.

Export the endpoint for the rest of this guide:

# No trailing slash. Reachable from the gateway, not only from your workstation.
export GUARD_ENDPOINT="http://qwen3guard.internal:8000"
export GUARD_API_KEY="YOUR_GUARD_API_KEY"

Confirm the deployment answers before configuring anything in AISIX:

curl -sS "$GUARD_ENDPOINT/v1/chat/completions" \
  -H "Authorization: Bearer $GUARD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3Guard-Gen-4B",
    "messages": [{"role": "user", "content": "How can I make a bomb?"}],
    "temperature": 0,
    "max_tokens": 64
  }' | jq -r '.choices[0].message.content'

The reply is the verdict itself:

Safety: Unsafe
Categories: Violent

Read the Model's Answer

Qwen3Guard answers in plain text with two or three lines. The first line is the label, the second lists the categories, and a response evaluation adds a third line for refusal:

Line	Values
`Safety:`	`Safe`, `Unsafe`, or `Controversial`. `Controversial` means the content is not prohibited on its own but could become risky depending on context or audience.
`Categories:`	A comma-separated list, or `None` for safe content.
`Refusal:`	`Yes` or `No`. Present only when the model evaluates an assistant response. It reports whether the response declined the request, not whether the response is unsafe.

The categories are fixed:

Category	Applies to
`Violent`	Prompts and responses
`Non-violent Illegal Acts`	Prompts and responses
`Sexual Content or Sexual Acts`	Prompts and responses
`PII`	Prompts and responses
`Suicide & Self-Harm`	Prompts and responses
`Unethical Acts`	Prompts and responses
`Politically Sensitive Topics`	Prompts and responses
`Copyright Violation`	Prompts and responses
`Jailbreak`	Prompts only

The answer is text rather than a JSON document, so parse it with the two patterns the model card uses. Treat an answer that matches neither as a failure rather than as an allow, as the script below does.

How the Model Decides What to Evaluate

Qwen3Guard has no separate parameter for prompt screening and response screening. Its chat template inspects the role of the last message in the conversation you submit and builds the matching instruction:

Last message from user — the model evaluates that user query, with the earlier turns as context, and answers with Safety and Categories.
Last message from assistant — the model evaluates that assistant response and adds Refusal.

caution

The template renders the conversation only from the first system or user message onward. A conversation that starts with an assistant message renders as an empty conversation, and the model then answers Safety: Safe no matter what the assistant message contains.

This matters on the response hook, where AISIX hands your script the model's reply and nothing else. Sending that reply as the only message produces a guardrail that appears configured, calls the model on every response, and never blocks anything. Always send a user turn before the assistant turn, as the script below does.

Write the Screening Script

The script is an ES module exporting checkInput and checkOutput. Both build a conversation for Qwen3Guard, parse its answer, and map it onto an AISIX verdict:

// Reachable from the gateway. Ends at the chat-completions path.
const GUARD_ENDPOINT = "http://qwen3guard.internal:8000/v1/chat/completions";
const GUARD_MODEL = "Qwen/Qwen3Guard-Gen-4B";

// Labels that block. Add "Controversial" for a stricter policy.
const BLOCKING_LABELS = ["Unsafe"];
// Categories that block, when the label blocks. An empty list blocks every category.
const BLOCKING_CATEGORIES = [];

async function classify(ctx, messages) {
  const headers = { "content-type": "application/json" };
  if (ctx.secrets.GUARD_API_KEY) {
    headers.authorization = "Bearer " + ctx.secrets.GUARD_API_KEY;
  }
  const resp = await fetch(GUARD_ENDPOINT, {
    method: "POST",
    headers: headers,
    body: JSON.stringify({
      model: GUARD_MODEL,
      messages: messages,
      temperature: 0,
      max_tokens: 64,
    }),
  });
  if (!resp.ok) {
    throw new Error("Qwen3Guard returned " + resp.status);
  }
  const content = resp.json().choices[0].message.content;
  const label = (content.match(/Safety:\s*(Safe|Unsafe|Controversial)/) || [])[1];
  if (!label) {
    throw new Error("Qwen3Guard returned an unreadable verdict");
  }
  const categories = (content.match(/Categories:\s*(.*)/) || ["", ""])[1]
    .split(",")
    .map(function (c) { return c.trim(); })
    .filter(function (c) { return c !== "" && c !== "None"; });
  console.log("qwen3guard " + ctx.hook + " -> " + label + " [" + categories.join("|") + "]");
  return { label: label, categories: categories };
}

function decide(verdict) {
  if (BLOCKING_LABELS.indexOf(verdict.label) === -1) {
    return { action: "none" };
  }
  if (BLOCKING_CATEGORIES.length > 0 &&
      !verdict.categories.some(function (c) { return BLOCKING_CATEGORIES.indexOf(c) !== -1; })) {
    return { action: "none" };
  }
  return {
    action: "block",
    reason_code: "qwen3guard_" + verdict.label.toLowerCase(),
    reason: verdict.categories.join(", ") || "no category reported",
  };
}

export async function checkInput(ctx) {
  return decide(await classify(ctx, [{ role: "user", content: ctx.text }]));
}

export async function checkOutput(ctx) {
  // The user turn is required: a conversation that starts with an assistant
  // message renders as empty and always comes back Safe.
  return decide(await classify(ctx, [
    { role: "user", content: "N/A" },
    { role: "assistant", content: ctx.text },
  ]));
}

Four decisions in that script are worth understanding before you adapt it.

The request hook submits ctx.text, not the individual messages. ctx.text is every text slot of the request joined together, so one call screens the whole request. Screening only the newest turn is cheaper but weaker. A caller controls the entire messages array it sends, including any conversation history it claims took place, so a policy that inspects only the last turn can be fed prohibited content in a fabricated earlier turn.

Neither hook reads the reported message roles. ctx.messages carries the slots AISIX is screening, but a Chat Completions request reports every slot with the user role, and the response hook reports the reply the same way. Build the Qwen3Guard conversation from ctx.text and ctx.hook, as above, rather than from the roles.

A failure throws instead of returning a verdict. A non-2xx answer, an unreachable endpoint, and a reply the script cannot read all raise. That hands the decision to the guardrail's failure policy. Returning {action: "none"} on failure would turn every guard-model outage into an open gateway.

The block reason carries the label and categories, not the content. reason_code and reason reach your gateway logs and the request's usage event; neither reaches the caller, and neither should carry the text that was screened.

Create the Guardrail

Start with the guardrail attached to a single model rather than the whole environment. The script is code you just wrote, and a scoped attachment keeps a mistake on traffic you control.

AISIX Cloud

Export the control-plane connection details:

# AISIX_CP includes /api and has no trailing slash.
# The local On-Premises quickstart uses http://localhost:8080/api.
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_BASE_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"
export MODEL_ID="YOUR_MODEL_ID"

Write the script to a file, then create the guardrail with jq so the script does not have to be escaped by hand:

export GUARDRAIL_ID=$(jq -n \
  --rawfile script ./qwen3guard.js \
  --arg key "$GUARD_API_KEY" \
  '{
    name: "qwen3guard",
    enabled: false,
    kind: "custom",
    hook_point: "both",
    fail_open: false,
    config: {
      script: $script,
      secrets: { GUARD_API_KEY: $key },
      timeout_ms: 5000,
      output_fail_open: false
    }
  }' | curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/guardrails" \
    -H "Authorization: Bearer $AISIX_TOKEN" \
    -H "Content-Type: application/json" \
    -d @- | jq -r '.guardrail.id')

The script is parsed when the guardrail is saved, so a syntax error is refused here, with its line and column, rather than on the first request that reaches it. Secrets are stored encrypted and never returned; a later read shows their names only, so you can see what the script may read without exposing the values.

Attach the guardrail to one model:

curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/guardrails/$GUARDRAIL_ID/attachments" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"scope_type": "model", "scope_id": "'"$MODEL_ID"'"}' | jq

Enable it once the attachment exists:

curl -sS -X PATCH "$AISIX_CP/environments/$ENV_ID/guardrails/$GUARDRAIL_ID" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"enabled": true}' | jq

The dashboard offers the same fields, with a code editor for the script and a name-and-value editor for its secrets, under Guardrails in the environment.

Open-Source AISIX Gateway

Add the guardrail to the resources file that defines your model and caller API key. A block scalar keeps the script readable, and $${...} is not needed here because the script contains no interpolation the loader would try to resolve:

resources.yaml
guardrails:
  - name: qwen3guard
    enabled: true
    kind: custom
    hook_point: both
    fail_open: false
    output_fail_open: false
    timeout_ms: 5000
    script: |
      const GUARD_ENDPOINT = "http://qwen3guard.internal:8000/v1/chat/completions";
      const GUARD_MODEL = "Qwen/Qwen3Guard-Gen-4B";
      const BLOCKING_LABELS = ["Unsafe"];
      const BLOCKING_CATEGORIES = [];

      async function classify(ctx, messages) {
        const headers = { "content-type": "application/json" };
        if (ctx.secrets.GUARD_API_KEY) {
          headers.authorization = "Bearer " + ctx.secrets.GUARD_API_KEY;
        }
        const resp = await fetch(GUARD_ENDPOINT, {
          method: "POST",
          headers: headers,
          body: JSON.stringify({
            model: GUARD_MODEL,
            messages: messages,
            temperature: 0,
            max_tokens: 64,
          }),
        });
        if (!resp.ok) {
          throw new Error("Qwen3Guard returned " + resp.status);
        }
        const content = resp.json().choices[0].message.content;
        const label = (content.match(/Safety:\s*(Safe|Unsafe|Controversial)/) || [])[1];
        if (!label) {
          throw new Error("Qwen3Guard returned an unreadable verdict");
        }
        const categories = (content.match(/Categories:\s*(.*)/) || ["", ""])[1]
          .split(",")
          .map(function (c) { return c.trim(); })
          .filter(function (c) { return c !== "" && c !== "None"; });
        console.log("qwen3guard " + ctx.hook + " -> " + label + " [" + categories.join("|") + "]");
        return { label: label, categories: categories };
      }

      function decide(verdict) {
        if (BLOCKING_LABELS.indexOf(verdict.label) === -1) {
          return { action: "none" };
        }
        if (BLOCKING_CATEGORIES.length > 0 &&
            !verdict.categories.some(function (c) { return BLOCKING_CATEGORIES.indexOf(c) !== -1; })) {
          return { action: "none" };
        }
        return {
          action: "block",
          reason_code: "qwen3guard_" + verdict.label.toLowerCase(),
          reason: verdict.categories.join(", ") || "no category reported",
        };
      }

      export async function checkInput(ctx) {
        return decide(await classify(ctx, [{ role: "user", content: ctx.text }]));
      }

      export async function checkOutput(ctx) {
        return decide(await classify(ctx, [
          { role: "user", content: "N/A" },
          { role: "assistant", content: ctx.text },
        ]));
      }
    secrets:
      GUARD_API_KEY: ${GUARD_API_KEY}

The resources file has no attachment collection, so an enabled guardrail applies to every request that gateway handles. Load the file with a gateway that carries no unrelated traffic while you evaluate the policy. Reload the gateway with SIGHUP after editing the file.

caution

A script the gateway cannot compile does not stop the gateway. The guardrail is dropped and the gateway serves traffic without it: no request is screened, nothing in the request path says so, and GET /status/config still reports synced with an empty rejected list, because the file itself loaded.

Run aisix validate --resources <FILE> before every load and reload. It compiles the script, names the row and the failing line and column, and exits non-zero.

Verify Screening

Send a request the policy allows. It reaches the model and returns normally:

curl -sS "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer $AISIX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"$AISIX_MODEL"'",
    "messages": [{"role": "user", "content": "How do I bake sourdough bread?"}]
  }'

Send a prohibited request. AISIX answers 422 and never calls the upstream model:

curl -sS -i "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer $AISIX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"$AISIX_MODEL"'",
    "messages": [{"role": "user", "content": "How can I make a bomb at home?"}]
  }'

{
  "error": {
    "message": "request blocked by content policy (guardrail 'qwen3guard')",
    "type": "content_filter"
  }
}

The response hook blocks the same way, with response blocked by content policy in place of request blocked, and cuts a streamed response mid-stream with an SSE event: error frame. Because the model answers in the caller's language, a prohibited request in any of the languages Qwen3Guard covers is refused by the same policy.

Your gateway log carries one line per screened hook from the script's console.log, which is the fastest way to confirm the policy is running as intended:

qwen3guard input -> Safe []
qwen3guard output -> Unsafe [Violent]

Tune the Policy

Evaluate a new policy in monitor mode before enforcing it. Monitor mode records what would have been blocked without refusing anything, which is the cheapest way to measure false positives against your own traffic.

Choose What Blocks

BLOCKING_LABELS and BLOCKING_CATEGORIES at the top of the script are the whole policy surface:

Controversial is the label to decide about first. It marks content that is not prohibited on its own but could be misused. Blocking it raises false positives; allowing it lets borderline requests through. Start by leaving it out and reviewing what it would have caught in monitor mode.
BLOCKING_CATEGORIES narrows enforcement to the categories your policy actually covers. A gateway serving a legal-research tool may block every category except Non-violent Illegal Acts, where the discussion is the point.

The two lists apply to both hooks. If a response needs a different policy from a request, give checkOutput its own lists rather than sharing one pair.

Size the Timeout

timeout_ms is the budget for one hook invocation, covering the script's own execution and the call it makes. It defaults to 5000. The right value is above the p99 latency of your guard deployment for the longest content you screen, measured on the hardware you deployed. Two factors dominate that latency: model size, and the length of the text being screened, since the guard model reads the entire conversation.

A budget that is too tight fails the same way an outage does, so measure before lowering it. A budget that is too loose delays every caller behind a guard model that has stopped answering.

Screen Streamed Responses

For a streamed response, AISIX screens a sliding window of the text and releases each window once it passes, so a streaming client stays responsive. Two consequences are specific to a guard model:

Each window is classified on its own, without the rest of the answer. A fragment cut mid-sentence carries less context than the finished response, and borderline fragments tend toward Controversial. This is the strongest argument for leaving Controversial out of BLOCKING_LABELS while streaming.
One guard call happens per window, so a long streamed answer costs several classifications.

Set stream_processing_mode to buffer_full to hold the whole response and classify it once instead. That gives the guard model the complete answer and costs one call, at the price of the caller waiting for the full response.

Decide What Happens When the Model Is Unreachable

fail_open governs requests and output_fail_open governs responses. Both default to false, which blocks traffic the script could not screen, and both should stay there for a screening control: a guard model that stops answering is exactly when unscreened content would flow.

When the guard model is down, the gateway logs the cause and the policy that was applied:

custom guardrail script threw row=qwen3guard error=Error: error sending request for url (...)
custom guardrail script failed row=qwen3guard failure=Threw fail_open=false

If you do set a hook to fail open, the bypass is recorded on the request's usage event under custom_script_error, custom_timeout, custom_bad_verdict, or custom_engine_error, so bypassed traffic remains countable.

Adapt the Script to Another Guard Model

Any guard model served over an OpenAI-compatible endpoint fits this pattern. Three parts of the script change, and nothing else does:

Part of the script	What to change
`GUARD_ENDPOINT`, `GUARD_MODEL`	The deployment and model name.
The `messages` each hook builds	How the model expects the conversation. Qwen3Guard infers prompt or response screening from the last role; other models take a fixed instruction template, or a system prompt that carries the policy.
The parsing in `classify`	How the answer is read. A model that answers `safe` or `unsafe` on its own line, or that returns JSON, needs its own parse; keep the rule that an unrecognized answer throws.

For example, Llama Guard answers with the word safe, or unsafe followed by a category code on the next line, so classify reads the first line and treats anything else as a failure. ShieldGemma answers with a Yes or No policy violation judgment for the policy stated in the prompt, so its messages carry that policy text and the parse reads the judgment.

Whatever the model, keep the four properties the Qwen3Guard script has. Screen the whole request rather than the newest turn. Do not depend on the roles AISIX reports. Throw on any failure instead of returning an allow. Keep screened content out of the block reason.

Troubleshooting

Every response passes, including ones that should be blocked. Check that checkOutput sends a user turn before the assistant turn. Without it Qwen3Guard scores an empty conversation and answers Safety: Safe for any content. Confirm with a direct call to the guard endpoint using the two-message conversation from the script.

Every request is refused with content_filter while the guard model is healthy from your workstation. The script runs inside the gateway, so the endpoint must resolve and be reachable from the gateway's network, not yours. The gateway log distinguishes a policy block from an outage: an outage logs custom guardrail script threw with the underlying error.

The guardrail does not run at all. A script that fails to compile surfaces differently on each path. AISIX Cloud refuses the save and names the line and column. An open-source gateway loads the file, skips that guardrail, and keeps serving without it; aisix validate is what catches that before the gateway does. On both paths, the module must use export async function. A hook you do not export is skipped rather than treated as an error, so a typo in a hook name silently disables that direction.

Verdicts vary between identical requests. Set temperature: 0 in the guard call, as the script does. A guard model sampled at a non-zero temperature will disagree with itself.

Next Steps

Custom Script Guardrails — the full script contract, including content rewriting and the sandbox limits.
Guardrail Behavior — hook points, enforcement modes, scoping, and failure policies.
Choosing a Guardrail Provider — the built-in providers, for policies that do not need a model of your own.

Prerequisites​

Choose a Qwen3Guard Model​

Deploy Qwen3Guard​

Read the Model's Answer​

How the Model Decides What to Evaluate​

Write the Screening Script​

Create the Guardrail​

AISIX Cloud​

Open-Source AISIX Gateway​

Verify Screening​

Tune the Policy​

Choose What Blocks​

Size the Timeout​

Screen Streamed Responses​

Decide What Happens When the Model Is Unreachable​

Adapt the Script to Another Guard Model​

Troubleshooting​

Next Steps​