Skip to main content
API7.ai Logo
Version: 3.9.x

Implement Prompt Templates and Decorators

This guide shows how to standardize prompt behavior at the gateway layer using ai-prompt-decorator and ai-prompt-template. You will configure reusable prompt templates, enforce system instructions, and combine both plugins with ai-proxy.

Overview​

Gateway-level prompt engineering keeps prompt policy out of application code. Platform teams can enforce consistent behavior across services, update instructions centrally, and reduce drift between teams.

API7 AI Gateway provides two complementary patterns:

  • Prompt decoration with ai-prompt-decorator: prepend or append messages to each request.
  • Prompt templating with ai-prompt-template: define named templates with {{variable}} substitution.

Prerequisites​

  • Install Docker.

  • Install cURL to send requests to the services for validation.

  • Have a running API7 Gateway instance.

  • Create a token from the Dashboard and save it to an environment variable:

    export API_KEY=your-dashboard-token   # replace with your Dashboard token

  • Replace {gateway_group_id} with your gateway group ID. Use default if you are following the quickstart.

  • If you are following the Admin API examples, create or reuse a service in API7 Gateway. If you do not have one yet, follow Create or Reuse a Service, then save its ID to an environment variable:

    export SERVICE_ID=your-service-id         # replace with your service ID

Prompt Decoration​

Use ai-prompt-decorator to inject global instructions before and after user messages.

curl -k "https://localhost:7443/apisix/admin/routes?gateway_group_id={gateway_group_id}" -X PUT \
  -H "X-API-KEY: ${API_KEY}" \
  --data-binary @- <<EOF
  {
    "id": "ai-prompt-decoration",
    "service_id": "$SERVICE_ID",
    "paths": ["/ai/decorated"],
    "plugins": {
      "ai-proxy": {
        "provider": "openai",
        "auth": { "header": { "Authorization": "Bearer $OPENAI_API_KEY" } },
        "options": { "model": "gpt-4o" }
      },
      "ai-prompt-decorator": {
        "prepend": [
          {
            "role": "system",
            "content": "You are a customer support assistant for Acme Corp. Reply with concise, policy-compliant answers."
          }
        ],
        "append": [
          {
            "role": "system",
            "content": "If the user asks for account actions, ask for ticket ID before proceeding."
          }
        ]
      }
    }
  }
EOF

❶ prepend inserts messages at the beginning of the conversation, typically for mandatory system instructions.

❷ append inserts messages at the end of the conversation, useful for additional policy reminders.

For the full configuration reference, see ai-prompt-decorator.

Prompt Templating​

Use ai-prompt-template to expose named prompt contracts. Callers provide variables while the gateway manages full prompt structure. This plugin uses body-transformer internally.

curl -k "https://localhost:7443/apisix/admin/routes?gateway_group_id={gateway_group_id}" -X PUT \
  -H "X-API-KEY: ${API_KEY}" \
  --data-binary @- <<EOF
  {
    "id": "ai-prompt-template",
    "service_id": "$SERVICE_ID",
    "paths": ["/ai/template"],
    "plugins": {
      "ai-proxy": {
        "provider": "openai",
        "auth": { "header": { "Authorization": "Bearer $OPENAI_API_KEY" } },
        "options": { "model": "gpt-4o" }
      },
      "ai-prompt-template": {
        "templates": [
          {
            "name": "support-reply",
            "template": {
              "model": "gpt-4o",
              "messages": [
                {
                  "role": "system",
                  "content": "You are a support assistant. Keep answers under 120 words."
                },
                {
                  "role": "user",
                  "content": "User issue: {{issue_summary}}"
                }
              ]
            }
          }
        ]
      }
    }
  }
EOF

❶ templates defines reusable named prompt templates. Clients select one using template_name in the request body.

❷ template contains the full request body shape, including model and messages with variable placeholders such as {{issue_summary}}.

For the full configuration reference, see ai-prompt-template.

Combining Decorators and Templates​

You can combine both plugins on the same route. A common pattern is:

  1. ai-prompt-template defines the task-specific prompt contract.
  2. ai-prompt-decorator injects organization-wide system policies.
curl -k "https://localhost:7443/apisix/admin/routes?gateway_group_id={gateway_group_id}" -X PUT \
  -H "X-API-KEY: ${API_KEY}" \
  --data-binary @- <<EOF
  {
    "id": "ai-template-and-decorator",
    "service_id": "$SERVICE_ID",
    "paths": ["/ai/templated-decorated"],
    "plugins": {
      "ai-proxy": {
        "provider": "openai",
        "auth": { "header": { "Authorization": "Bearer $OPENAI_API_KEY" } },
        "options": { "model": "gpt-4o" }
      },
      "ai-prompt-template": {
        "templates": [
          {
            "name": "incident-summary",
            "template": {
              "model": "gpt-4o",
              "messages": [
                {
                  "role": "user",
                  "content": "Summarize this incident for an operations report: {{incident_text}}"
                }
              ]
            }
          }
        ]
      },
      "ai-prompt-decorator": {
        "prepend": [
          {
            "role": "system",
            "content": "You are an SRE assistant. Use factual, neutral language and include concrete action items."
          }
        ],
        "append": [
          {
            "role": "system",
            "content": "End with a section named Next Steps."
          }
        ]
      }
    }
  }
EOF

❶ ai-prompt-template defines the reusable task template selected by template_name.

❷ ai-prompt-decorator enforces shared system behavior across all requests using that template.

Verify​

Send a request to the decorated route:

curl "http://127.0.0.1:9080/ai/decorated" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [
      { "role": "user", "content": "Can you reset my account password?" }
    ]
  }'

Expected result: the response follows the injected policy (for example, asking for ticket ID before account actions).

Send a request using a named template:

curl "http://127.0.0.1:9080/ai/template" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "template_name": "support-reply",
    "issue_summary": "Order #4721 has not arrived after 10 days"
  }'

Expected result: the gateway expands {{issue_summary}} into the template and returns a generated answer.

Send a request to the combined route:

curl "http://127.0.0.1:9080/ai/templated-decorated" -X POST \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "template_name": "incident-summary",
    "incident_text": "Database CPU reached 95% for 18 minutes, causing elevated p99 latency in checkout." 
  }'

Expected result: the output follows both the template intent and decorator-enforced style (SRE tone and a Next Steps section).

Next Steps​