Instructor

Instructor is a Python library for extracting structured data from LLM responses with Pydantic models. It adds response-model validation and retry behavior on top of provider clients.

Instructor can wrap an OpenAI-compatible client that already points to AISIX. Use this setup when an application should keep Instructor's response-model workflow while routing model traffic through the gateway.

This guide uses Instructor with the OpenAI Python client. You will configure that client with an AISIX proxy URL, caller API key, and model alias, then pass it to Instructor.

Prerequisites

Before starting, prepare the following:

• A Python environment supported by Instructor.
• A running AISIX gateway with the proxy listener available.
• An AISIX caller API key.
• A model alias the caller API key can access through the OpenAI-Compatible API.

Configure Instructor

Install the Python packages if the application does not already include them:

pip install instructor openai "pydantic>=2"

Set the values that the application will use:

# Replace with your values
export AISIX_BASE_URL="http://127.0.0.1:3000/v1"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="gpt-4o-prod"

Create an OpenAI client for AISIX, then pass it to Instructor:

import os

import instructor
from openai import OpenAI
from pydantic import BaseModel


class Ticket(BaseModel):
  category: str
  priority: str


openai_client = OpenAI(
  base_url=os.environ["AISIX_BASE_URL"],
  api_key=os.environ["AISIX_API_KEY"],
)

client = instructor.from_openai(openai_client)

ticket = client.chat.completions.create(
  model=os.environ["AISIX_MODEL"],
  response_model=Ticket,
  messages=[
    {
      "role": "user",
      "content": "Classify this ticket: production login failures for all users.",
    }
  ],
)

print(ticket.model_dump_json(indent=2))

The model value is the AISIX model alias, not the upstream provider model ID. AISIX governs the model request path: it authenticates the caller API key, resolves the alias, applies configured policies, records telemetry, and dispatches the request to the provider behind the alias. Instructor still owns schema validation and retry behavior.

Verify the Route

Run the script from the shell where the AISIX environment variables are set.

When the request succeeds, verify the following results:

• The script prints a validated Pydantic object.
• AISIX records one or more successful POST /v1/chat/completions requests for the selected model alias.

If you use AISIX Cloud or an on-premises managed control plane, verify the request in the managed gateway logs. For self-hosted gateways, use your configured logs, metrics, or upstream provider logs.

Handle Compatibility Issues

If a request fails, first confirm that the caller API key can access the selected model alias and that the OpenAI client uses the AISIX base_url.

Instructor may retry requests when validation fails. Account for those retries when configuring budgets, rate limits, and usage review. If validation keeps retrying, check whether the upstream model can satisfy the schema and whether the selected Instructor mode is supported by the AISIX route.

Next Steps

• OpenAI-Compatible API: review gateway-facing request behavior.
• Budgets: account for validation retries in spend controls.