Skip to main content
API7.ai Logo
Version: 3.9.x

Enforce AI Guardrails and Protect PII

API7 AI Gateway can layer ai-prompt-guard, ai-aws-content-moderation, and ai-request-rewrite on one route to centralize prompt filtering, content moderation, and request-side PII masking before traffic reaches the primary model.

When ai-request-rewrite is enabled, the gateway sends the original request body to the configured rewrite model before ai-prompt-guard evaluates the rewritten body. Request-side masking can therefore reduce PII exposure to the primary LLM, but the rewrite provider still receives the original content. Use a trusted or self-hosted rewrite provider when that content cannot leave your environment.

Overview​

Guardrails are most effective when enforced at the gateway layer, where policies are centralized and applied consistently across applications. A practical defense-in-depth model uses three layers:

  1. Prompt filtering to block prompt injection and disallowed instructions before the primary model invocation.
  2. Content moderation to detect harmful content categories and reject high-risk requests.
  3. PII masking to transform sensitive data before requests reach the primary LLM provider, while accounting for the separate rewrite-provider call.

Prerequisites​

  • Install Docker.

  • Install cURL to send requests to the services for validation.

  • Have a running API7 Gateway instance.

  • Create a token from the Dashboard and save it to an environment variable:

    export API_KEY=your-dashboard-token   # replace with your Dashboard token

  • Replace {gateway_group_id} with your gateway group ID. Use default if you are following the quickstart.

  • If you are following the Admin API examples, create or reuse a service in API7 Gateway. If you do not have one yet, follow Create or Reuse a Service, then save its ID to an environment variable:

    export SERVICE_ID=your-service-id         # replace with your service ID

Prompt Protection​

Use ai-prompt-guard to apply PCRE-based allow and deny patterns. In this example, the plugin checks user messages only (match_all_roles: false) and scans only the latest message (match_all_conversation_history: false). When a deny pattern matches, the request is rejected with HTTP 400.

allow_pattern='(?i)^(what|how|why|explain|summarize|translate)\\b'
deny_pattern='(?i)(ignore\\s+all\\s+previous\\s+instructions|reveal\\s+system\\s+prompt|bypass\\s+guardrails)'

curl -k "https://localhost:7443/apisix/admin/routes/ai-guardrails-prompt-protection?gateway_group_id={gateway_group_id}" -X PUT \
  -H "X-API-KEY: ${API_KEY}" \
  -H "Content-Type: application/json" \
  --data-binary @- <<EOF
  {
    "name": "ai-guardrails-prompt-protection",
    "service_id": "$SERVICE_ID",
    "paths": ["/ai/chat"],
    "plugins": {
      "ai-prompt-guard": {
        "allow_patterns": ["$allow_pattern"],
        "deny_patterns": ["$deny_pattern"],
        "match_all_roles": false,
        "match_all_conversation_history": false
      },
      "ai-proxy": {
        "provider": "openai",
        "auth": { "header": { "Authorization": "Bearer $OPENAI_API_KEY" } },
        "options": { "model": "gpt-4o" }
      }
    }
  }
EOF

❶ allow_patterns defines accepted prompt shape using PCRE syntax.

❷ deny_patterns blocks known injection and policy-bypass phrases.

❸ match_all_roles: false and match_all_conversation_history: false scope matching to the latest user message.

For the full configuration reference, see ai-prompt-guard.

Content Moderation​

Content moderation adds a second layer of filtering for harmful or abusive text.

AWS Comprehend Integration​

Use ai-aws-content-moderation to score six moderation categories with 0-1 thresholds. You can set per-category thresholds in moderation_categories together with an overall toxicity threshold in moderation_threshold. Requests that exceed a category threshold or the overall threshold can be blocked with a configurable rejection status code and message.

curl -k "https://localhost:7443/apisix/admin/routes/ai-guardrails-content-moderation?gateway_group_id={gateway_group_id}" -X PUT \
  -H "X-API-KEY: ${API_KEY}" \
  -H "Content-Type: application/json" \
  --data-binary @- <<EOF
  {
    "name": "ai-guardrails-content-moderation",
    "service_id": "$SERVICE_ID",
    "paths": ["/ai/chat"],
    "plugins": {
      "ai-aws-content-moderation": {
        "comprehend": {
          "access_key_id": "$AWS_ACCESS_KEY_ID",
          "secret_access_key": "$AWS_SECRET_ACCESS_KEY",
          "region": "us-east-1"
        },
        "moderation_categories": {
          "PROFANITY": 0.5,
          "HATE_SPEECH": 0.5,
          "INSULT": 0.5,
          "HARASSMENT_OR_ABUSE": 0.5,
          "SEXUAL": 0.5,
          "VIOLENCE_OR_THREAT": 0.5
        },
        "moderation_threshold": 0.5
      },
      "ai-proxy": {
        "provider": "openai",
        "auth": { "header": { "Authorization": "Bearer $OPENAI_API_KEY" } },
        "options": { "model": "gpt-4o" }
      }
    }
  }
EOF

❶ comprehend provides AWS credentials for the Comprehend API. access_key_id, secret_access_key, and region are required.

❷ Configure per-category thresholds for PROFANITY, HATE_SPEECH, INSULT, HARASSMENT_OR_ABUSE, SEXUAL, and VIOLENCE_OR_THREAT.

❸ moderation_threshold defines the overall toxicity threshold. Use moderation_categories to enforce specific category thresholds in addition to the global threshold.

For the full configuration reference, see ai-aws-content-moderation.

Custom Moderation Services​

You can call a dedicated model with ai-request-rewrite to classify or transform request content before forwarding it to the primary LLM route. The plugin replaces the request body with the model output, but it does not interpret a score or reject the request as a custom moderation decision. Other configured guardrails can still reject the rewritten request. Enforce a custom taxonomy with a component that explicitly returns a blocking decision.

PII Masking​

PII masking can reduce accidental exposure of names, phone numbers, account identifiers, and other sensitive fields to the primary LLM provider. It does not hide the original request from the rewrite provider and does not by itself establish compliance with a particular standard.

Request-Side PII Masking​

Use ai-request-rewrite to send incoming prompts to a separate model that detects and masks PII before the request reaches the primary LLM.

caution

The rewrite model receives the original request before masking. Use a provider that is approved to process the unmasked content, or host the rewrite model inside the required trust boundary.

curl -k "https://localhost:7443/apisix/admin/routes/ai-guardrails-pii-request?gateway_group_id={gateway_group_id}" -X PUT \
  -H "X-API-KEY: ${API_KEY}" \
  -H "Content-Type: application/json" \
  --data-binary @- <<EOF
  {
    "name": "ai-guardrails-pii-request",
    "service_id": "$SERVICE_ID",
    "paths": ["/ai/chat"],
    "plugins": {
      "ai-request-rewrite": {
        "provider": "openai",
        "auth": {
          "header": {
            "Authorization": "Bearer $OPENAI_API_KEY"
          }
        },
        "options": {
          "model": "gpt-4o"
        },
        "prompt": "Return the complete incoming OpenAI-compatible request as valid JSON. Preserve every field and structure. In textual message content only, replace emails with [REDACTED_EMAIL], phone numbers with [REDACTED_PHONE], payment card numbers with [REDACTED_CARD], and government identifiers with [REDACTED_ID]. Return JSON only."
      },
      "ai-proxy": {
        "provider": "openai",
        "auth": { "header": { "Authorization": "Bearer $OPENAI_API_KEY" } },
        "options": { "model": "gpt-4o" }
      }
    }
  }
EOF

❶ provider sets the model backend used for rewrite decisions.

❷ auth configures credentials for the rewrite model call.

❸ options.model selects the rewrite model. The rewrite output replaces the complete request body, so prompt requires a valid, complete AI request rather than only the sanitized text.

Response-Side PII Filtering​

ai-request-rewrite runs only on the client request and does not inspect or modify the model response. Apply response-side PII controls in the upstream application or another component that explicitly supports response processing.

For the full request-rewrite configuration reference, see ai-request-rewrite.

Combining Guardrails​

In most deployments, combine the controls on the same route. Their observable request flow is:

  1. ai-request-rewrite calls its configured rewrite model and replaces the complete request body.
  2. ai-prompt-guard evaluates the rewritten request and rejects matching prompt patterns.
  3. ai-proxy detects the request protocol and selects its provider configuration, then ai-aws-content-moderation evaluates the content later in the access phase.
  4. After the access phase completes, ai-proxy builds and sends the approved provider request.
note

Plugin execution order is determined by phase and priority, not by the order in the configuration. In particular, ai-request-rewrite runs before ai-prompt-guard, so prompt-guard rules do not protect the external rewrite-model call.

allow_pattern='(?i)^.{1,4000}$'
deny_pattern='(?i)(ignore\\s+all\\s+previous\\s+instructions|reveal\\s+system\\s+prompt|developer\\s+mode)'

curl -k "https://localhost:7443/apisix/admin/routes/ai-guardrails-combined?gateway_group_id={gateway_group_id}" -X PUT \
  -H "X-API-KEY: ${API_KEY}" \
  -H "Content-Type: application/json" \
  --data-binary @- <<EOF
  {
    "name": "ai-guardrails-combined",
    "service_id": "$SERVICE_ID",
    "paths": ["/ai/chat"],
    "plugins": {
      "ai-prompt-guard": {
        "allow_patterns": ["$allow_pattern"],
        "deny_patterns": ["$deny_pattern"],
        "match_all_roles": false,
        "match_all_conversation_history": false
      },
      "ai-request-rewrite": {
        "provider": "openai",
        "auth": {
          "header": {
            "Authorization": "Bearer $OPENAI_API_KEY"
          }
        },
        "options": {
          "model": "gpt-4o"
        },
        "prompt": "Return the complete incoming OpenAI-compatible request as valid JSON. Preserve every field and structure. Redact PII only in textual message content. Return JSON only."
      },
      "ai-aws-content-moderation": {
        "comprehend": {
          "access_key_id": "$AWS_ACCESS_KEY_ID",
          "secret_access_key": "$AWS_SECRET_ACCESS_KEY",
          "region": "us-east-1"
        },
        "moderation_categories": {
          "PROFANITY": 0.5,
          "HATE_SPEECH": 0.5,
          "INSULT": 0.5,
          "HARASSMENT_OR_ABUSE": 0.5,
          "SEXUAL": 0.5,
          "VIOLENCE_OR_THREAT": 0.5
        },
        "moderation_threshold": 0.5
      },
      "ai-proxy": {
        "provider": "openai",
        "auth": { "header": { "Authorization": "Bearer $OPENAI_API_KEY" } },
        "options": { "model": "gpt-4o" }
      }
    }
  }
EOF

❶ ai-prompt-guard evaluates the rewritten request and rejects deny-pattern matches with HTTP 400.

❷ ai-request-rewrite runs first, sends the original request to its configured model, and replaces the complete body with the returned JSON request.

❸ ai-aws-content-moderation provides AWS Comprehend credentials in the comprehend block and enforces toxicity and abuse thresholds before forwarding traffic.

Next Steps​