Skip to main content
API7.ai Logo

OpenAI Moderation Guardrails

The OpenAI Moderation guardrail checks content against the OpenAI Moderation API, a classifier for categories such as harassment, hate, self-harm, sexual content, and violence. Flagged content is blocked with 422 Unprocessable Entity; the guardrail never rewrites text.

By default the guardrail trusts the API's own flagged decision. You can instead enforce specific categories at your own score cutoffs with category_thresholds.

In this guide, you will create a moderation guardrail, verify blocking, and tune per-category thresholds.

Prerequisites​

Before starting, prepare the following:

  • Review Guardrail Behavior for hook points, enforcement modes, and remote failure handling.
  • One of these configuration paths:
    • AISIX Cloud with an environment, an attached gateway, and a write-scoped admin token. For On-Premises, follow the AISIX Cloud Quickstart. To request Hybrid Cloud access, contact API7.
    • An open-source AISIX gateway that loads a declarative resources.yaml file.
  • A working model alias and caller API key that can send Chat Completions requests.
  • An OpenAI API key that can call the Moderation API.
  • curl. The AISIX Cloud path also uses jq.

Create a Moderation Guardrail​

The example below blocks content according to the moderation model's flagged decision. Choose one configuration path, then use the shared verification procedure.

Export the gateway and provider values used by both paths:

# AISIX_PROXY has no trailing slash or endpoint path.
# The local quickstarts use http://127.0.0.1:3000.
export AISIX_PROXY="YOUR_AISIX_GATEWAY_URL"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="gpt-4o-mini"
export OPENAI_API_KEY="YOUR_OPENAI_API_KEY"

AISIX Cloud​

Export the control-plane connection details:

# AISIX_CP includes /api and has no trailing slash.
# The local On-Premises quickstart uses http://localhost:8080/api.
export AISIX_CP="YOUR_AISIX_CLOUD_ADMIN_API_BASE_URL"
export AISIX_TOKEN="YOUR_ADMIN_TOKEN"
export ENV_ID="YOUR_ENVIRONMENT_ID"

Create an input guardrail in the environment and capture its ID:

export GUARDRAIL_ID=$(curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/guardrails" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "openai-moderation-input",
    "enabled": false,
    "hook_point": "input",
    "fail_open": false,
    "kind": "openai_moderation",
    "config": {
      "api_key": "'"${OPENAI_API_KEY}"'",
      "model": "omni-moderation-latest"
    }
  }' | jq -r '.guardrail.id')

❶ fail_open: false blocks the request if OpenAI Moderation fails or times out. The default is true.

❷ omni-moderation-latest is the default moderation model. Use another OpenAI moderation model only when an existing policy depends on that model's scores.

The guardrail calls https://api.openai.com/v1/moderations by default. Set config.endpoint to point at a compatible endpoint when needed.

A guardrail runs only where it is attached. Attach it to the whole environment so it applies to all traffic:

curl -sS -X POST "$AISIX_CP/environments/$ENV_ID/guardrails/$GUARDRAIL_ID/attachments" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "scope_type": "env"
  }'

To narrow the scope instead, set scope_type to model, api_key, or team and pass the target's ID as scope_id.

Enable the guardrail after its attachment exists:

curl -sS -X PATCH "$AISIX_CP/environments/$ENV_ID/guardrails/$GUARDRAIL_ID" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"enabled": true}'

The enabled configuration projects to attached gateways automatically.

If you set hook_point to output or both, review Streaming Output before enforcing the guardrail on streamed responses.

Open-Source AISIX Gateway​

Make OPENAI_API_KEY available to the gateway process, then add the guardrail to the resources file that already defines the example model and caller API key:

resources.yaml
guardrails:
  - name: openai-moderation-input
    enabled: true
    hook_point: input
    fail_open: false
    kind: openai_moderation
    api_key: ${OPENAI_API_KEY}
    model: omni-moderation-latest

The provider fields sit directly on the guardrail entry rather than under config. The default endpoint is https://api.openai.com/v1; set endpoint when the gateway should call another compatible service.

Every enabled guardrail in the resources file applies to every request handled by that gateway. Validate the complete file, then reload the gateway. See Reload a Resources File for the runnable Docker workflow.

Verify Blocking​

AISIX Cloud projection is asynchronous. If the first request does not reflect the guardrail, wait for the gateway to apply the latest revision and retry. See Resource Projection for convergence checks.

Send a prompt the moderation model flags:

curl -sSi -X POST "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  --data-binary @- <<EOF
  {
    "model": "${AISIX_MODEL}",
    "messages": [
      {
        "role": "user",
        "content": "describe in detail how to hurt someone"
      }
    ]
  }
EOF

A flagged request is rejected before the upstream model is called, with HTTP/1.1 422 Unprocessable Entity:

{
  "error": {
    "message": "request blocked by content policy (guardrail 'openai-moderation-input')",
    "type": "content_filter"
  }
}

The violated category names go to the gateway log and usage record; the flagged content itself is never echoed back or logged.

Per-Category Thresholds​

Set category_thresholds to take control of the decision. When it is non-empty:

  • Only the listed categories are enforced. Other categories are ignored even if the API flags them.
  • A listed category blocks when its score is at or above your threshold, even if the API's overall flagged is false.

AISIX Cloud​

Replace the existing guardrail's provider configuration with explicit category thresholds. Its attachment remains in place:

curl -sS -X PATCH "$AISIX_CP/environments/$ENV_ID/guardrails/$GUARDRAIL_ID" \
  -H "Authorization: Bearer $AISIX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "config": {
      "api_key": "'"${OPENAI_API_KEY}"'",
      "category_thresholds": {
        "violence": 0.3,
        "harassment/threatening": 0.5
      }
    }
  }'

Open-Source AISIX Gateway​

Add category_thresholds to the existing guardrail entry to replace its decision rule:

resources.yaml
guardrails:
  - name: openai-moderation-input
    enabled: true
    hook_point: input
    fail_open: false
    kind: openai_moderation
    api_key: ${OPENAI_API_KEY}
    category_thresholds:
      violence: 0.3
      harassment/threatening: 0.5

Validate the complete file and reload the gateway before testing the new thresholds.

Scores are between 0 and 1. Lower thresholds block more aggressively; start in enforcement_mode: monitor and tune against real traffic before enforcing.

Next Steps​

You have now configured OpenAI Moderation and verified blocking. Use these guides to tune behavior or compare related guardrails: