Batch, Files, and Fine-Tuning

AISIX AI Gateway exposes the OpenAI-compatible Files, Batch, and Fine-tuning APIs as first-class proxy routes. Clients keep the standard OpenAI SDK calls while the gateway manages caller authentication, provider credentials, and usage attribution for completed batches.

In this guide, you will upload a batch input file, create and track a batch, and see how the gateway routes each call to the right provider.

Supported Routes

Family	Routes
Files	POST /v1/files, GET /v1/files, GET /v1/files/{id}, DELETE /v1/files/{id}, GET /v1/files/{id}/content
Batch	POST /v1/batches, GET /v1/batches, GET /v1/batches/{id}, POST /v1/batches/{id}/cancel
Fine-tuning	POST /v1/fine_tuning/jobs, GET /v1/fine_tuning/jobs, GET /v1/fine_tuning/jobs/{id}, POST /v1/fine_tuning/jobs/{id}/cancel

Provider support covers OpenAI-compatible providers (adapter openai, including custom api_base deployments) and Azure OpenAI (adapter azure-openai, resource-scoped routes with api-key authentication). Vertex AI, Bedrock, and Anthropic-native batch flows use different wire and storage models and are not served by these routes yet.

Prerequisites

Before starting, prepare the following:

• A running AISIX gateway that can serve proxy requests.
• A caller API key that can access the model alias.
• A model alias backed by an OpenAI-compatible or Azure OpenAI provider.

How Routing Works

A batch create request references a previously uploaded file id — there is no model field in the body for the gateway to route on. AISIX solves this with gateway-encoded resource ids:

1. When you upload a file, name the routing model once: a model multipart field, a ?model= query parameter, or an x-aisix-model header.
2. The file id returned by the gateway (aisix-…) embeds that model. Any later call that references the id — batch create, file retrieve or download, fine-tuning training_file — routes automatically.
3. Ids returned by create and retrieve responses (batch ids, output file ids, fine-tuning job ids) are encoded the same way, so follow-up calls need no extra hints.

Raw provider ids keep working: the gateway falls back to an explicit model query parameter or header, and then to the first OpenAI-compatible model the caller key can access. Pass an explicit model for deterministic routing.

Upload a File and Run a Batch

Upload the batch input file through the gateway, naming the routing model on the request:

curl -sS -X POST "http://127.0.0.1:3000/v1/files" \
  -H "Authorization: Bearer YOUR_CALLER_API_KEY" \
  -H "x-aisix-model: gpt-4o-prod" \
  -F purpose=batch \
  -F file=@batch-input.jsonl

The response id starts with aisix- and embeds the routing model. Create the batch with that id:

curl -sS -X POST "http://127.0.0.1:3000/v1/batches" \
  -H "Authorization: Bearer YOUR_CALLER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "aisix-…",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'

Track the batch and download results with the ids from the responses:

curl -sS "http://127.0.0.1:3000/v1/batches/aisix-…" \
  -H "Authorization: Bearer YOUR_CALLER_API_KEY"

curl -sS "http://127.0.0.1:3000/v1/files/aisix-…/content" \
  -H "Authorization: Bearer YOUR_CALLER_API_KEY"

The official OpenAI SDKs work unchanged — point base_url at the gateway and pass the routing hint as a header on files.create.

Fine-Tuning Jobs

Fine-tuning jobs route through the encoded training_file id. The model field in the job body is the provider's base model to fine-tune and is forwarded verbatim — it is not a gateway model alias:

curl -sS -X POST "http://127.0.0.1:3000/v1/fine_tuning/jobs" \
  -H "Authorization: Bearer YOUR_CALLER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini-2024-07-18",
    "training_file": "aisix-…"
  }'

Usage and Cost Attribution

Management calls (upload, create, list, cancel) record zero-token usage events for visibility in logs.

When a batch retrieve first observes status: "completed", the gateway downloads the batch output file, aggregates the per-line token usage grouped by the provider-billed model, and emits usage events with real token counts. These events carry a deterministic request id (batch-<id>), so repeated retrieves and gateway restarts do not double-count.

Governance Behavior

The full policy envelope applies to every route in this family: caller API key authentication and model access lists, per-model client IP restrictions, budgets and rate limits, and input/output guardrail scans over the transferred payloads. Upstream calls honor the model's request timeout, and transport failures count toward the model's cooldown.

Next Steps

You have now run file uploads, batches, and fine-tuning jobs through the gateway. Next, review Provider Passthrough for provider-native endpoints outside this surface.