ai-request-rewrite

The ai-request-rewrite plugin processes client requests by forwarding them to LLM services for transformation before relaying them to upstream services. This enables LLM-powered modifications such as data redaction, content enrichment, or reformatting. The plugin supports the integration with OpenAI, DeepSeek, and other OpenAI-compatible APIs.

note

The plugin is currently only available in API7 Enterprise and will become available in APISIX 3.13.0 release.

Examples

The examples below demonstrate how you can configure ai-request-rewrite for different scenarios.

The examples will use OpenAI as the LLM service. To follow along, obtain the OpenAI API key and save it to an environement variable:

export OPENAI_API_KEY=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   # replace with your API key

Redact Sensitive Information

The following example demonstrates how you can use the ai-request-rewrite plugin to redact sensitive information before the request reaches the upstream service.

Create a route and configure the ai-request-rewrite plugin as such:

curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
  -H "X-API-KEY: ${ADMIN_API_KEY}" \
  -d '{
    "id": "ai-request-rewrite-route",
    "uri": "/anything",
    "methods": ["POST"],
    "plugins": {
      "ai-request-rewrite": {
        "provider": "openai",
        "auth": {
          "header": {
            "Authorization": "Bearer '"$OPENAI_API_KEY"'"
          }
        },
        "options":{
          "model": "gpt-4"
        },
        "prompt": "Given a JSON request body, identify and mask any sensitive information such as credit card numbers, social security numbers, and personal identification numbers (e.g., passport or driver'\''s license numbers). Replace detected sensitive values with a masked format (e.g., \"*** **** **** 1234\") for credit card numbers. Ensure the JSON structure remains unchanged."
      }
    },
    "upstream": {
      "type": "roundrobin",
      "nodes": {
        "httpbin.org:80": 1
      }
    }
  }'

❶ Specify the provider to be openai.

❷ Attach OpenAI API key in the Authorization header.

❸ Specify the name of the model.

❹ Specify what information to redact before the request reaches the upstream service.

Send a POST request to the route with some personally identifiable information:

curl "http://127.0.0.1:9080/anything" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "content": "John said his debit card number is 4111 1111 1111 1111 and SIN is 123-45-6789."
  }'

You should receive a response similar to the following:

{
  "args": {},
  "data": "{\n    \"content\": \"John said his debit card number is **** **** **** 1111 and SIN is ***-**-****.\"\n  }"
  ...,
  "json": {
    "messages": [
      {
        "content": "Client information from customer service calls",
        "role": "system"
      }, 
      {
        "content": "John said his debit card number is **** **** **** 1111 and SIN is ***-**-****."
        "role": "user"
      }
    ], 
    "model": "openai"
  }, 
  "method": "POST", 
  "origin": "192.168.97.1, 103.97.2.170", 
  "url": "http://127.0.0.1/anything"
}

Reformat Data

The following example demonstrates how you can use the ai-request-rewrite plugin to reformat data before the request reaches the upstream service.

Create a route and configure the ai-request-rewrite plugin as such:

curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
  -H "X-API-KEY: ${ADMIN_API_KEY}" \
  -d '{
    "id": "ai-request-rewrite-route",
    "uri": "/anything",
    "methods": ["POST"],
    "plugins": {
      "ai-request-rewrite": {
        "provider": "openai",
        "auth": {
          "header": {
            "Authorization": "Bearer '"$OPENAI_API_KEY"'"
          }
        },
        "options":{
          "model": "gpt-4"
        },
        "prompt": "Convert natural language queries into structured JSON format with intent and extracted parameters."
      }
    },
    "upstream": {
      "type": "roundrobin",
      "nodes": {
        "httpbin.org:80": 1
      }
    }
  }'

❶ Specify the provider to be openai.

❷ Attach OpenAI API key in the Authorization header.

❸ Specify the name of the model.

❹ Specify how to reformat before the request reaches the upstream service.

Send a POST request to the route with some personally identifiable information:

curl "http://127.0.0.1:9080/anything" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Book a flight from NYC to LA on April 10, 2022."
  }'

You should receive a response similar to the following:

{
  "args": {},
  "data": "{\n  \"intent\": \"BookFlight\",\n  \"parameters\": {\n    \"origin\": \"NYC\",\n    \"destination\": \"LA\",\n    \"date\": \"2022-04-10\"\n  }\n}", 
  ...,
  "json": {
    "intent": "BookFlight", 
    "parameters": {
      "date": "2022-04-10", 
      "destination": "LA", 
      "origin": "NYC"
    }
  },
  "method": "POST", 
  "origin": "192.168.97.1, 103.97.2.167", 
  "url": "http://127.0.0.1/anything"
}

Summarize Information

The following example demonstrates how you can use the ai-request-rewrite plugin to summarize information before the request reaches the upstream service.

Create a route and configure the ai-request-rewrite plugin as such:

curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
  -H "X-API-KEY: ${ADMIN_API_KEY}" \
  -d '{
    "id": "ai-request-rewrite-route",
    "uri": "/anything",
    "methods": ["POST"],
    "plugins": {
      "ai-request-rewrite": {
        "provider": "openai",
        "auth": {
          "header": {
            "Authorization": "Bearer '"$OPENAI_API_KEY"'"
          }
        },
        "options":{
          "model": "gpt-4"
        },
        "prompt": "Summarize lengthy input while preserving key details. Ensure the summary remains concise and informative."
      }
    },
    "upstream": {
      "type": "roundrobin",
      "nodes": {
        "httpbin.org:80": 1
      }
    }
  }'

❶ Specify the provider to be openai.

❷ Attach OpenAI API key in the Authorization header.

❸ Specify the name of the model.

❹ Specify the requirements to summarize information before the request reaches the upstream service.

Send a POST request to the route with some personally identifiable information:

curl "http://127.0.0.1:9080/anything" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Hey! So, I’m planning a trip to Japan next spring for about three weeks, and I want to visit Tokyo, Kyoto, and Osaka, but I’m not sure how to split my time between them. I really love history and cultural sites, so temples and shrines are a must. I’m also a big foodie, especially into ramen and sushi, so I’d love recommendations on the best spots. I prefer quieter areas for accommodation, but I don’t mind traveling into busy areas for sightseeing. Oh, and I’d also like to do a day trip somewhere outside these cities—maybe Hakone or Nara? I heard the cherry blossoms might still be in bloom in early April, so I’d love to catch them if possible. Also, what’s the best way to get around—should I get a JR Pass, or would individual tickets be better? Thanks!"
  }'

You should receive a response similar to the following:

{
  "args": {},
  "data": "The individual is planning a three-week trip to Japan in the spring, looking to visit Tokyo, Kyoto, and Osaka. They are interested in history, culture, temples, and shrines. They love ramen and sushi, so are seeking food recommendations. Accommodation should be in quieter areas, but they are open to busy sites for sightseeing. Along with these cities, they plan to make a day trip to either Hakone or Nara, hoping to see the cherry blossoms in early April. The best transport method between buying the JR Pass or individual tickets is also a query.", 
  ...,
  "method": "POST", 
  "origin": "192.168.97.1, 103.97.2.171", 
  "url": "http://127.0.0.1/anything"
}

ai-request-rewrite

Examples​

Redact Sensitive Information​

Reformat Data​

Summarize Information​

Examples

Redact Sensitive Information

Reformat Data

Summarize Information