Calibrate Semantic Screening Guardrails
Semantic screening thresholds determine how closely traffic must resemble your examples before AISIX allows or blocks it. The right cutoff depends on the embedding model, example list, languages, and traffic you use, so a value that works for one policy does not transfer safely to another.
Calibrate in two stages. First, score representative text to find a candidate threshold. Then observe the scores from real traffic before switching the guardrail to block. This process also reveals policies that are not running where you expect or whose embedding calls are failing.
Before starting, create a semantic screening guardrail using either configuration path. In AISIX Cloud, keep it disabled while scoring samples in the dashboard, then use monitor mode to observe real traffic. An open-source gateway should begin in monitor mode. Both deny_threshold and allow_threshold are required whenever their respective example lists are non-empty, and AISIX provides no portable default for newly configured policies.
Score Text in AISIX Cloud
Open the guardrail in Guardrails, expand Test this guardrail, paste the text you want judged, and select Run test. AISIX Cloud embeds the text and the guardrail's saved examples, then reports:
- the verdict the gateway would reach for that text, allow or block, and which gate refused it;
- one row per example list, including the measured similarity, configured threshold, and highest-scoring example, or a note that the list is empty and that direction cannot block;
- the embedding model that produced the scores.
Running a test requires write permission on guardrails. A read-only role receives a 403 and should calibrate from per-request scores instead. This permission protects the example text included in the test result. Because the route uses POST, authorization maps it to the write action.
Score texts you expect the guardrail to block and texts you expect it to serve, then choose a threshold that reflects the trade-off between those groups. The panel is on the guardrail's edit form, so save the guardrail first. It scores the stored configuration, not unsaved edits.
For an input guardrail, test one message at a time. The gateway screens messages separately, newest first, while a whole conversation pasted into the panel is embedded as one text and may score differently. For an output guardrail, paste the complete model response because the output hook screens the whole response as one text. Test request and response samples separately for a guardrail whose hook_point is both.
The panel tests the configuration, not its deployment. The control plane performs the embedding and comparison without involving a gateway. It warns that a guardrail has no attachments only when the saved policy is enabled and the tested text would be blocked. The absence of that warning therefore does not prove the policy is attached.
The panel cannot confirm that an attachment covers a specific model or caller API key, or that the gateway can resolve the embedding model. The embedding endpoint must also be reachable from the control plane. Use real-traffic scores for a model that is reachable only inside your private network.
The panel reports what the guardrail would decide. A block verdict does not mean a disabled or monitor policy will block a request. The panel can also score text without knowing whether it came from the request or response side.
The test panel is available in AISIX Cloud. An open-source gateway has no equivalent panel, so begin with a placeholder threshold and enforcement_mode: monitor, then calibrate entirely from the real-traffic scores below.
Observe Scores on Real Traffic
A semantic guardrail records what it measured on every request it screens, including allowed requests and policies in either monitor or block mode. Monitor hits alone are not enough: they exist only when the guardrail would have blocked. A threshold set just beyond the scores it receives produces no monitor hits and can look like a guardrail that never ran.
Several endpoints support only the input hook. /a2a records scores, enforced hits, monitor hits, and bypass reasons, but resolves no model or MCP server. Only environment-, API-key-, and team-scoped guardrails can therefore reach it.
rerank, /v1/embeddings, /v1/images/*, /v1/videos, /v1/audio/speech, and /v1/messages/count_tokens also support only the input hook. A semantic score is sufficient attribution for a screened rerank request to emit a usage event even when the upstream provides no usage data. An unscreened request to such an upstream can leave no event. The other listed endpoints emit a usage event for every dispatched request.
Diagnose Missing Scores
A request can have no semantic scores for several reasons:
- No semantic guardrail screened it.
- The policy does not cover that endpoint. An output guardrail does not run on an input-only endpoint, and a model-scoped guardrail does not run on
/a2a, an MCP tool call, or a passthrough route because those requests resolve no model. - The request was recorded by a gateway version from before semantic scores were added.
- An earlier guardrail blocked the request. The chain stops at the first block in attachment-priority order.
- A streamed output exceeded
max_buffer_bytesand was refused before the semantic guardrail ran. - The row belongs to a superseded retry, failover attempt, or ensemble member. Only the terminal row carries the scores, and it is not necessarily the last row displayed.
- The request had no text to screen. Empty text is skipped, so a request containing only an image makes no embedding call with the default
text_source: user_messages. - The embedding call failed. Screening stops before it records a score when the embedding model times out or returns an error.
An embedding failure is recorded outside the score array. Check the field that matches the policy mode and failure setting:
| Policy | Result | Recorded evidence |
|---|---|---|
block, failing closed | Request or response is refused | guardrail_enforced_hits with action blocked_unavailable and the failure tag in error_type; the dashboard shows check unavailable under Enforced hits |
monitor, failing closed | Traffic is served | guardrail_monitor_hits with action would_block and reason semantic guardrail evaluation unavailable (…); the dashboard shows would block under Monitor hits |
| Either mode, failing open | Traffic is served without this guardrail's evaluation; later guardrails can still block | guardrail_bypassed_reason; the dashboard shows Bypass reason |
The request hook uses fail_open, and the response hook uses output_fail_open. Both default to failing closed, so check the setting for the hook you are investigating.
Read the Score Fields
In Logs, expand a request to see Semantic guardrail scores. Each entry identifies the guardrail, hook, example-list direction, score, threshold, embedding model, and line number of the closest example.
The array contains at most one entry for each guardrail, hook, and example list. It summarizes the request rather than reporting every screened message. A deny-list entry keeps the highest similarity, the point closest to being refused. An allow-list entry keeps the lowest score among the messages that were evaluated. Evaluation stops at the first blocking result, so an older message might have scored lower, and a deny-list refusal can prevent the allow list from being evaluated at all.
The same values are available outside the dashboard:
| Where | Field |
|---|---|
| Request-log export, CSV | guardrail_scores column |
| Request-log export, JSON, and the AISIX Cloud Admin API usage event | guardrail_scores array |
| Gateway log exporters | guardrail_scores on the usage record; aisix.guardrail_scores in Datadog because fields without OpenTelemetry semantic-convention names receive the aisix. prefix. OTLP traces do not include this field. |
Each entry includes guardrail_name, hook, direction, score, threshold, matched, top_example_index, and embedding_model. In both directions, matched means score >= threshold. It describes similarity, not the final verdict: a matched deny example is refused, while an allow list refuses when none of its examples match.
Neither the screened text nor the example text is recorded. top_example_index is a zero-based index into that direction's list. The dashboard displays it as a line number counting from one, so you can find the example in your configuration without exposing policy text in the logs.
See Logging and Auditing for the rest of the request log and its export.
Interpret the Score Distribution
Compare the scores from text you intend to block with the scores from text you intend to serve. The guide's examples show how deny and allow thresholds differ; API7's broader measurement shows why the resulting values do not transfer to another model or traffic set.
Understand the Guide's Example Thresholds
The main guide uses a deny threshold of 0.47 with text-embedding-3-small. API7 measured that operating point for the guide's example list and probe set. It puts the guide's allowed and blocked verification requests on opposite sides of the threshold.
The same value is below the unrelated-traffic noise floor measured for bge-m3 (0.480) and gemini-embedding-001 (0.542). Copying it to either model can therefore block ordinary traffic and make the allowed verification request fail.
The topic allow-list example uses 0.3 with text-embedding-3-small. Against that list, the measured support requests ranged from 0.309 for an order-status question to 0.574 for a delivery question; return and cancellation requests scored 0.427 and 0.452. Off-topic requests stayed at or below 0.180. The example threshold admits those support probes, but its margin for the lowest-scoring request is small.
Deny and allow thresholds face opposite pressures. Benign traffic pushes a deny threshold higher so fewer ordinary requests are refused. In-scope traffic pushes an allow threshold lower so more intended requests are admitted. The classes can still overlap, so these are not hard bounds, and the two thresholds do not transfer even when they use the same embedding model.
About the API7 Measurement
The guidance below comes from a sweep API7 ran on 1 September 2026 against five embedding models from four vendors: text-embedding-3-small and text-embedding-3-large from OpenAI, gemini-embedding-001 from Google, qwen3-embedding-8b from Alibaba, and bge-m3 from BAAI. API7 evaluated the same 250-probe set with each model. The set was split evenly between English and Chinese across weapons, illicit drugs, jailbreak, PII disclosure, and competitor-mention policies. Each category used its own four-example deny list in both languages.
Each probe belongs to one class. Five classes are attempts the policy should catch: a near paraphrase, a same-intent rewrite, role-play framing, an indirect approach, and a divergent attempt. The divergent attempt reaches the same goal in unrelated words. Two classes represent ordinary traffic: a benign request on a related topic and a benign unrelated request.
Each model and language condition contains 75 attack probes, 15 per attack class, and 40 benign probes, 25 related and 15 unrelated. The remaining ten probes are controls excluded from the rates below. Two examples from each deny list were scored against themselves to confirm that an exact copy reaches approximately 1.0.
This is a dated measurement of specific models on a specific probe set, not a product guarantee. Models change, and your traffic will have a different score distribution. The patterns are useful; the exact values are not portable thresholds.
Thresholds Do Not Transfer Between Embedding Models
The first comparison is the noise floor, the highest score an ordinary unrelated request reached against a deny list. These results use an English deny list and English probes.
| Embedding model | Highest unrelated score | Highest related-but-ordinary score | Median attack score |
|---|---|---|---|
text-embedding-3-large | 0.198 | 0.462 | 0.487 |
text-embedding-3-small | 0.228 | 0.498 | 0.448 |
qwen3-embedding-8b | 0.459 | 0.657 | 0.664 |
bge-m3 | 0.480 | 0.629 | 0.600 |
gemini-embedding-001 | 0.542 | 0.676 | 0.731 |
The floors span 0.344, from 0.198 to 0.542. A threshold of 0.55 sits above every ordinary probe for text-embedding-3-large. The same value refuses 55% of the benign probes for gemini-embedding-001, where related ordinary traffic scores as high as 0.676. AISIX therefore records the embedding model beside every score.
Changing the threshold also cannot fully separate an attempt from ordinary traffic on the same topic. On four models, 44% to 57% of the attack probes scored no higher than the single highest benign related request:
| Embedding model | Attack probes scoring at or below the highest benign related probe |
|---|---|
bge-m3 | 57% |
text-embedding-3-small | 56% |
qwen3-embedding-8b | 49% |
text-embedding-3-large | 44% |
gemini-embedding-001 | 12% |
On those four models, a threshold low enough to catch most attempts also refuses some ordinary related traffic. Choose that trade-off deliberately instead of looking for a cutoff that removes the overlap.
Write Examples in the Languages Your Traffic Uses
The same deny list can score lower against traffic in another language, and the difference depends on the model. These values are median attack scores for three conditions. They compare an English deny list against English probes, the same list against Chinese probes, and a Chinese deny list against those Chinese probes.
| Embedding model | English list, English traffic | English list, Chinese traffic | Chinese list, Chinese traffic |
|---|---|---|---|
text-embedding-3-small | 0.448 | 0.362 | 0.464 |
text-embedding-3-large | 0.487 | 0.378 | 0.463 |
qwen3-embedding-8b | 0.664 | 0.557 | 0.660 |
bge-m3 | 0.600 | 0.609 | 0.648 |
gemini-embedding-001 | 0.731 | 0.705 | 0.746 |
The drop changes verdicts for some models. The text-embedding-3-large median falls from 0.487 to 0.378 against its 0.43 operating point. The qwen3-embedding-8b median falls from 0.664 to 0.557 against its 0.63 operating point. The text-embedding-3-small median starts at 0.448 against a 0.47 threshold and drops further. bge-m3 and gemini-embedding-001 are close to language-agnostic on this probe set; the bge-m3 median rises slightly.
Writing examples in the traffic's own language recovers the loss: the third column exceeds the first for three models and is within 0.024 for the other two. Include deny examples in every language your callers use. AISIX does not detect the request language or warn when a deny list no longer matches it.
Measure Coverage by Attempt Type
The table reports each model's recall by attack class at its own operating point. This is the lowest threshold, on a 0.01 grid, that refuses no more than 5% of the model's 40 benign probes. Each cell represents 15 attack probes, using English lists and English traffic.
| Embedding model | Threshold | Near paraphrase | Same intent | Role-play | Indirect | Divergent |
|---|---|---|---|---|---|---|
text-embedding-3-small | 0.47 | 100% | 53% | 27% | 33% | 20% |
text-embedding-3-large | 0.43 | 100% | 67% | 40% | 67% | 40% |
gemini-embedding-001 | 0.67 | 100% | 93% | 87% | 93% | 73% |
qwen3-embedding-8b | 0.63 | 100% | 73% | 40% | 60% | 20% |
bge-m3 | 0.61 | 100% | 67% | 40% | 27% | 13% |
Every model caught every near paraphrase in the probe set. Four of the five caught only 13% to 40% of attempts that pursued the same goal using unrelated wording. Embedding similarity measures how close the texts are, so a semantic guardrail should not be the only control for an important policy. Pair it with a keyword guardrail for known literals and a dedicated moderation or injection-detection provider for classes that similarity examples cannot represent.
Review Existing Guardrails When Upgrading to 1.0.0
The AISIX Cloud 1.0.0 migration preserves an explicit threshold. A policy that omitted a threshold receives 0.75 for each non-empty example list, preserving the value it was already enforcing.
A policy with a JSON null in a threshold or another concrete field such as timeout_ms behaved differently. The gateway could not load it, so it screened no traffic even though the dashboard displayed it as active. The migration removes repairable null values only when the resulting configuration will load on the gateway. A successfully repaired policy returns to service and can begin refusing traffic that it did not previously screen.
The migration does not guess missing policy content. It leaves a row unchanged when embedding_model, deny_examples, or allow_examples is null, when an example list is malformed, or when the rewritten configuration still would not load. The startup log identifies rows that need an operator-supplied value.
Before upgrading an On-Premises control plane from a pre-1.0 release to 1.0.0, identify enabled semantic guardrails whose stored configuration contains null-valued or malformed fields. Disable each affected policy before correcting it or starting the upgrade. Saving a corrected policy while it remains enabled can put it back into service immediately; reactivation does not wait for the migration. Do not use 0.75 as the marker because unaffected legacy policies can already carry it. A repaired threshold whose example list is empty is removed instead of set to a number.
The control plane attempts this run-once migration at startup. If the migration fails as a whole, the failure is logged and retried at the next startup while the control plane continues to serve. A row left unchanged during a successful run is not retried automatically. During a rolling upgrade, an older control-plane replica can accept a request that explicitly writes a new null after the migration has finished. That row remains outside the completed sweep.
After upgrading to 1.0.0, review the startup log for policies the migration reactivated or left unchanged. If an enabled policy was reactivated, disable it before testing. Score representative text in the test panel while the policy is disabled. Then set it to monitor, enable it, and observe real traffic before restoring block mode.
For a policy left unchanged, the test panel returns 409 and names invalid configuration that it can load far enough to diagnose. An embedding-model reference that cannot be read or resolved instead returns 404 and reports that the semantic guardrail or its embedding model was not found. Keep the policy disabled while supplying the missing value and saving the corrected configuration. Then follow the same test and monitor sequence before restoring enforcement.
An open-source resources file is not rewritten during an upgrade. In 1.0.0 and later, a semantic guardrail with examples but no corresponding threshold fails validation. The whole file is rejected until every affected entry has the required threshold. The gateway does not start with that file, and a running gateway rejects an invalid reload.
Next Steps
After choosing candidate thresholds, verify the semantic screening guardrail with caller-visible requests and keep monitoring the score distribution as traffic changes. See Guardrail Behavior for attachment priority, enforcement modes, and failure policies.