Skip to main content
API7.ai Logo

Instructor

Instructor is a Python library for extracting structured data from LLM responses with Pydantic models. It adds response-model validation and retry behavior on top of provider clients.

Instructor can wrap an OpenAI-compatible client that already points to AISIX. Use this setup when an application should keep Instructor's response-model workflow while routing model traffic through the gateway.

This guide uses Instructor with the OpenAI Responses API. You will configure the OpenAI Python client with an AISIX proxy URL, caller API key, and model alias, then pass it to Instructor.

Prerequisites​

Before starting, prepare the following:

  • A Python environment supported by Instructor.
  • A running AISIX gateway with the proxy listener available.
  • An AISIX caller API key.
  • A model alias the caller API key can access through the Responses API.

If AISIX is already deployed for your organization, obtain the gateway URL, model alias, and caller API key from the team that manages it. Otherwise, follow the Open-Source AISIX Gateway Quickstart or AISIX Cloud Quickstart, or contact API7 for Hybrid Cloud access.

Configure Instructor​

Install the Python packages if the application does not already include them:

pip install instructor openai "pydantic>=2"

Set the values that the application will use:

# Replace with your values
export AISIX_BASE_URL="http://127.0.0.1:3000/v1"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="gpt-4o-prod"

Create an OpenAI client for AISIX, then pass it to Instructor in Responses mode:

import os

import instructor
from instructor import Mode
from openai import OpenAI
from pydantic import BaseModel


class Ticket(BaseModel):
  category: str
  priority: str


openai_client = OpenAI(
  base_url=os.environ["AISIX_BASE_URL"],
  api_key=os.environ["AISIX_API_KEY"],
)

client = instructor.from_openai(
  openai_client,
  mode=Mode.RESPONSES_TOOLS,
)

ticket = client.responses.create(
  model=os.environ["AISIX_MODEL"],
  response_model=Ticket,
  input="Classify this ticket: production login failures for all users.",
)

print(ticket.model_dump_json(indent=2))

The model value is the AISIX model alias, not the upstream provider model ID. AISIX governs the model request path: it authenticates the caller API key, resolves the alias, applies configured policies, records telemetry, and dispatches the request to the provider behind the alias. Instructor still owns schema validation and retry behavior.

Instructor also supports Chat Completions modes through client.chat.completions.create(...). Use those modes when a workflow needs broad OpenAI-compatible chat support.

Verify the Integration​

Run the script from the shell where the AISIX environment variables are set.

When the request succeeds, verify the following results:

  • The script prints a validated Pydantic object.
  • AISIX records one or more successful POST /v1/responses requests for the selected model alias.

If you use the AISIX Cloud control plane, verify the request in the AISIX gateway logs. For standalone gateways, use your configured logs, metrics, or upstream provider logs.

If the request fails, first confirm that the caller API key can access the selected model alias and that the OpenAI client uses the AISIX base_url.

Instructor may retry requests when validation fails. Account for those retries when configuring budgets, rate limits, and usage review. If validation keeps retrying, check whether the upstream model can satisfy the schema and whether the selected Instructor mode is supported by the AISIX route.

Next Steps​

  • Responses API: review gateway-facing Responses behavior.
  • OpenAI-Compatible API: use Chat Completions when a workflow needs broad OpenAI-compatible support.
  • Budgets: account for validation retries in spend controls.