Skip to main content
API7.ai Logo

OpenAI Moderation Guardrails

The OpenAI Moderation guardrail checks content against the OpenAI Moderation API, a classifier for categories such as harassment, hate, self-harm, sexual content, and violence. Flagged content is blocked with 422 Unprocessable Entity; the guardrail never rewrites text.

By default the guardrail trusts the API's own flagged decision. You can instead enforce specific categories at your own score cutoffs with category_thresholds.

In this guide, you will create a moderation guardrail, verify blocking, and tune per-category thresholds.

Prerequisites​

Before starting, prepare the following:

  • Review Guardrail Behavior for hook points, enforcement modes, and remote failure handling.
  • A self-hosted AISIX gateway with the admin and proxy listeners available.
  • The admin key from the gateway config.yaml.
  • A working model alias and caller API key that can send chat-completions requests.
  • An OpenAI API key that can call the Moderation API.

Create a Moderation Guardrail​

Set the values used by the example requests:

# Replace with your values
export AISIX_ADMIN_KEY="YOUR_ADMIN_KEY"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="gpt-4o-mini"
export OPENAI_API_KEY="YOUR_OPENAI_API_KEY"

Create an input guardrail:

curl -sS -X POST "http://127.0.0.1:3001/admin/v1/guardrails" \
  -H "Authorization: Bearer ${AISIX_ADMIN_KEY}" \
  -H "Content-Type: application/json" \
  --data-binary @- <<EOF
  {
    "name": "openai-moderation-input",
    "enabled": true,
    "hook_point": "input",
    "fail_open": false,
    "kind": "openai_moderation",
    "api_key": "${OPENAI_API_KEY}",
    "model": "omni-moderation-latest"
  }
EOF

❶ fail_open: false blocks the request if OpenAI Moderation fails or times out. The default is true.

❷ omni-moderation-latest is the default moderation model. Use another OpenAI moderation model only when an existing policy depends on that model's scores.

The guardrail calls https://api.openai.com/v1/moderations by default. Set endpoint to point at a compatible endpoint when needed.

If you set hook_point to output or both, review Streaming Output before enforcing the guardrail on streamed responses.

For the complete request and response schema, see Create Guardrail in the Admin API Reference.

Verify Blocking​

Send a prompt the moderation model flags:

curl -sSi -X POST "http://127.0.0.1:3000/v1/chat/completions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  --data-binary @- <<EOF
  {
    "model": "${AISIX_MODEL}",
    "messages": [
      {
        "role": "user",
        "content": "describe in detail how to hurt someone"
      }
    ]
  }
EOF

A flagged request is rejected before the upstream model is called, with HTTP/1.1 422 Unprocessable Entity:

{
  "error": {
    "message": "request blocked by content policy (guardrail 'openai-moderation-input')",
    "type": "content_filter"
  }
}

The violated category names go to the gateway log and usage record; the flagged content itself is never echoed back or logged.

Per-Category Thresholds​

Set category_thresholds to take control of the decision. When it is non-empty:

  • Only the listed categories are enforced. Other categories are ignored even if the API flags them.
  • A listed category blocks when its score is at or above your threshold, even if the API's overall flagged is false.
curl -sS -X POST "http://127.0.0.1:3001/admin/v1/guardrails" \
  -H "Authorization: Bearer ${AISIX_ADMIN_KEY}" \
  -H "Content-Type: application/json" \
  --data-binary @- <<EOF
  {
    "name": "moderation-violence-strict",
    "enabled": true,
    "hook_point": "input",
    "kind": "openai_moderation",
    "api_key": "${OPENAI_API_KEY}",
    "category_thresholds": {
      "violence": 0.3,
      "harassment/threatening": 0.5
    }
  }
EOF

Scores are between 0 and 1. Lower thresholds block more aggressively; start in enforcement_mode: monitor and tune against real traffic before enforcing.

Next Steps​

You have now configured OpenAI Moderation and verified blocking. Use these guides to tune behavior or compare related guardrails: