Skip to main content
API7.ai Logo

ai-aliyun-content-moderation

The ai-aliyun-content-moderation plugin uses Aliyun Machine-Assisted Moderation Plus to evaluate selected request roles and LLM responses for risks such as profanity, hate speech, harassment, and violence. It rejects content whose evaluated risk reaches the configured threshold and can annotate or interrupt streaming responses according to stream_check_mode.

Please ensure that the access_key_secret is correctly configured in the plugin. If it is incorrectly configured, all requests will bypass the plugin to be directly forwarded to the LLM upstream, and you will see a Specified signature is not matched with our calculation in the gateway's error log from the plugin.

The ai-aliyun-content-moderation plugin should be used with either ai-proxy or ai-proxy-multi plugin for proxying LLM requests.

Demo​

The following demo demonstrates the moderate request content toxicity example in API7 Enterprise using the Dashboard, where you can moderate request content for toxicity and customize the rejection code and message.

Behavior by Request Format​

The plugin moderates Chat Completions, Responses API, Embeddings, Anthropic Messages, and Bedrock Converse requests using each protocol's native content structure.

The gateway identifies each request by checking URI-specific rules before body-only rules:

  • Bedrock Converse requires a URI ending in /converse and a messages array.
  • Anthropic Messages requires a URI ending in /v1/messages.
  • Responses API requires a URI ending in /v1/responses and an input field.
  • Chat Completions uses a messages array.
  • Embeddings uses input after the earlier rules do not match.
  • Other non-empty JSON objects use passthrough after none of the earlier rules match.
Request formatContent available for moderation
Bedrock ConverseText from system and messages.
Anthropic MessagesText from messages.
Responses APIText from instructions and input.
Chat CompletionsText from messages.
EmbeddingsA string or an array of strings in input.
Other JSON (passthrough)No request-format-specific text is extracted.

APISIX moderates all extracted content shown in the table. It moderates the latest user turn by default. Use request_check_roles to select user, tool, or system content and request_check_mode to select the latest or all matching turns. Introduced in API7 Enterprise 3.9.16 and 3.10.3, and APISIX 3.18.0.

With role-aware selection, the Anthropic top-level system prompt is available when the system role is selected. Tool-result moderation applies when the request format represents tool output as a distinct tool role or item. Anthropic Messages and Bedrock Converse nest tool results in user messages, so they are not extracted when only the tool role is selected.

To moderate the broadest supported request content, configure all available roles and all turns in the plugin configuration:

{
  "request_check_roles": ["user", "tool", "system"],
  "request_check_mode": "all"
}

This configuration scans all text that the detected protocol exposes for those roles. It does not reproduce raw-body moderation: nested Anthropic and Bedrock tool results remain part of user content rather than distinct tool messages, and unsupported non-AI structures follow fail_mode.

If Responses content is rejected, the plugin returns the configured message in Responses API format. Streaming requests receive typed server-sent events ending with response.completed.

Embeddings has no conversation roles or turns. With role-aware selection, its input is selected by the user role. Rejected requests receive an OpenAI-style error response.

If an Aliyun moderation request fails, the plugin logs the error and allows the content without a moderation verdict. The fail_mode setting governs unsupported or non-AI request formats; it does not make Aliyun service failures fail closed. Monitor moderation errors and Aliyun availability when this plugin is an enforcement control.

Examples​

The following examples will be using OpenAI as the upstream service provider.

Before proceeding, create an OpenAI account and obtain an API key. If you are working with other LLM providers, please refer to the provider's documentation to obtain an API key.

Additionally, create an Aliyun account, enable Machine-Assisted Moderation Plus, and obtain the endpoint, region ID, access key ID, and access key secret.

You can optionally save these information to environment variables:

# replace with your data
export OPENAI_API_KEY=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
export ALIYUN_ENDPOINT=https://green-cip.cn-shanghai.aliyuncs.com
export ALIYUN_REGION_ID=cn-shanghai
export ALIYUN_ACCESS_KEY_ID=LTAI5yXKZP77gR3BQQM9WJnA
export ALIYUN_ACCESS_KEY_SECRET=hT2YpkqLs9FIjh3dyznBw7RMux5OKv

Moderate Request Content Toxicity​

The following example demonstrates how you can use the plugin to moderate content toxicity in requests and customize rejection code and message.

Create a route to the LLM chat completion endpoint using the ai-proxy plugin and configure the integration details as well as the deny code and message in the ai-aliyun-content-moderation plugin:

curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
  -H "X-API-KEY: ${ADMIN_API_KEY}" \
  --data-binary @- <<EOF
  {
    "id": "ai-aliyun-content-moderation-route",
    "uri": "/anything",
    "plugins": {
      "ai-aliyun-content-moderation": {
        "endpoint": "$ALIYUN_ENDPOINT",
        "region_id": "$ALIYUN_REGION_ID",
        "access_key_id": "$ALIYUN_ACCESS_KEY_ID",
        "access_key_secret": "$ALIYUN_ACCESS_KEY_SECRET",
        "deny_code": 400,
        "deny_message": "Request contains forbidden content, such as hate speech or violence."
      },
      "ai-proxy": {
        "provider": "openai",
        "auth": {
          "header": {
            "Authorization": "Bearer $OPENAI_API_KEY"
          }
        }
      }
    }
  }
EOF

❶ Configure the rejection HTTP status code.

❷ Configure the rejection message.

Send a POST request to the route with a system prompt and a user question with a profane word in the request body:

curl -i "http://127.0.0.1:9080/anything" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4",
    "messages": [
      { "role": "system", "content": "You are a mathematician" },
      { "role": "user", "content": "Stupid, what is 1+1?" }
    ]
  }'

You should receive an HTTP/1.1 400 Bad Request response and see the following message:

{
  "object": "chat.completion",
  "usage": {
    "completion_tokens": 124,
    "prompt_tokens": 31,
    "total_tokens": 155
  },
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Request contains forbidden content, such as hate speech or violence."
      },
      "finish_reason": "stop",
      "index": 0
    }
  ],
  "model": "gpt-4",
  "id": "c9466bbf-e010-469d-949a-a10f25525964"
}

Send another request to the route with a typical question in the request body:

curl -i "http://127.0.0.1:9080/anything" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      { "role": "system", "content": "You are a mathematician" },
      { "role": "user", "content": "What is 1+1?" }
    ]
  }'

You should receive an HTTP/1.1 200 OK response with the model output:

{
  ...,
  "model": "gpt-4-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "1+1 equals 2.",
        "refusal": null
      },
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  ...
}

Adjust Risk Level Threshold​

The following example demonstrates how you can adjust the threshold of risk level, which regulates whether a request/response should be allowed through.

Create a route to the LLM chat completion endpoint using the ai-proxy plugin and configure the risk_level_bar in ai-aliyun-content-moderation to be high:

curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
  -H "X-API-KEY: ${ADMIN_API_KEY}" \
  --data-binary @- <<EOF
  {
    "id": "ai-aliyun-content-moderation-route",
    "uri": "/anything",
    "plugins": {
      "ai-aliyun-content-moderation": {
        "endpoint": "$ALIYUN_ENDPOINT",
        "region_id": "$ALIYUN_REGION_ID",
        "access_key_id": "$ALIYUN_ACCESS_KEY_ID",
        "access_key_secret": "$ALIYUN_ACCESS_KEY_SECRET",
        "deny_code": 400,
        "deny_message": "Request contains forbidden content, such as hate speech or violence.",
        "risk_level_bar": "high"
      },
      "ai-proxy": {
        "provider": "openai",
        "auth": {
          "header": {
            "Authorization": "Bearer $OPENAI_API_KEY"
          }
        },
        "model": "gpt-4"
      }
    }
  }
EOF

Send a POST request to the route with a system prompt and a user question with a profane word in the request body:

curl -i "http://127.0.0.1:9080/anything" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4",
    "messages": [
      { "role": "system", "content": "You are a mathematician" },
      { "role": "user", "content": "Stupid, what is 1+1?" }
    ]
  }'

You should receive an HTTP/1.1 400 Bad Request response and see the following message:

{
  "object": "chat.completion",
  "usage": {
    "completion_tokens": 124,
    "prompt_tokens": 31,
    "total_tokens": 155
  },
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Request contains forbidden content, such as hate speech or violence."
      },
      "finish_reason": "stop",
      "index": 0
    }
  ],
  "model": "gpt-4",
  "id": "c9466bbf-e010-469d-949a-a10f25525964"
}

Update the risk_level_bar in the plugin to max:

curl "http://127.0.0.1:9180/apisix/admin/routes/ai-aliyun-content-moderation-route" -X PATCH \
  -H "X-API-KEY: ${ADMIN_API_KEY}" \
  -d '{
    "plugins": {
      "ai-aliyun-content-moderation": {
        "risk_level_bar": "max"
      }
    }
  }'

Send the same request to the route:

curl -i "http://127.0.0.1:9080/anything" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      { "role": "system", "content": "You are a mathematician" },
      { "role": "user", "content": "Stupid, what is 1+1?" }
    ]
  }'

You should receive an HTTP/1.1 200 OK response with the model output:

{
  ...,
  "model": "gpt-4-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "1+1 equals 2.",
        "refusal": null
      },
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  ...
}

The request is forwarded because the word "stupid" has a risk level of high, which is below the configured threshold of max.

The plugin no longer emits the raw moderation request or successful moderation response as debug logs. Use the client-visible rejection status and your Aliyun monitoring data to investigate moderation outcomes without exposing prompt content in gateway logs.