Batch, Files, and Fine-Tuning

AISIX AI Gateway exposes the OpenAI-compatible Files, Batch, and Fine-tuning APIs as first-class proxy routes. Clients keep the standard OpenAI SDK calls while the gateway manages caller authentication, provider credentials, and usage attribution for completed batches.

In this guide, you will upload a batch input file, create and track a batch, and see how the gateway routes each call to the right provider.

Supported Proxy Routes

Family	Routes
Files	POST /v1/files, GET /v1/files, GET /v1/files/{id}, DELETE /v1/files/{id}, GET /v1/files/{id}/content
Batch	POST /v1/batches, GET /v1/batches, GET /v1/batches/{id}, POST /v1/batches/{id}/cancel
Fine-tuning	POST /v1/fine_tuning/jobs, GET /v1/fine_tuning/jobs, GET /v1/fine_tuning/jobs/{id}, POST /v1/fine_tuning/jobs/{id}/cancel

Provider support covers OpenAI-compatible providers (adapter openai, including custom api_base deployments) and Azure OpenAI (adapter azure-openai, resource-scoped routes with api-key authentication). Vertex AI, Bedrock, and Anthropic-native batch flows use different wire and storage models and are not served by these routes yet.

Prerequisites

Before starting, prepare the following:

• A running AISIX gateway that can serve proxy requests.
• A caller API key that can access the model alias.
• A model alias backed by an OpenAI-compatible or Azure OpenAI provider.

Export the gateway connection and request values:

# AISIX_PROXY has no trailing slash or endpoint path such as /v1.
# The local quickstarts use http://127.0.0.1:3000.
export AISIX_PROXY="YOUR_AISIX_GATEWAY_ORIGIN"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="gpt-4o-prod"

How AISIX Routes Files and Jobs

A batch create request references a previously uploaded file id. There is no model field in the body for the gateway to route on, so AISIX uses gateway-encoded resource ids:

1. When you upload a file, name the routing model once: a model multipart field, a ?model= query parameter, or an x-aisix-model header.
2. The file id returned by the gateway (aisix-…) embeds that model. Any later call that references the id routes automatically, including batch create, file retrieve or download, and fine-tuning training_file.
3. Ids returned by create and retrieve responses (batch ids, output file ids, fine-tuning job ids) are encoded the same way, so follow-up calls need no extra hints.

Raw provider ids keep working: the gateway falls back to an explicit model query parameter or header, and then to the first OpenAI-compatible model the caller key can access. Pass an explicit model for deterministic routing.

Upload a File and Run a Batch

Upload the batch input file through the gateway, naming the routing model on the request:

curl -sS -X POST "${AISIX_PROXY}/v1/files" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "x-aisix-model: ${AISIX_MODEL}" \
  -F purpose=batch \
  -F file=@batch-input.jsonl

The response id starts with aisix- and embeds the routing model. Create the batch with that id:

curl -sS -X POST "${AISIX_PROXY}/v1/batches" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "aisix-…",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'

Track the batch with the batch ID returned by the create request:

curl -sS "${AISIX_PROXY}/v1/batches/aisix-…" \
  -H "Authorization: Bearer ${AISIX_API_KEY}"

When the batch completes, download the result file with the output_file_id from the batch response:

curl -sS "${AISIX_PROXY}/v1/files/aisix-…/content" \
  -H "Authorization: Bearer ${AISIX_API_KEY}"

The official OpenAI SDKs work unchanged. Point base_url at the gateway and pass the routing hint as a header on files.create.

Fine-Tuning Jobs

Fine-tuning jobs route through the encoded training_file id. The model field in the job body is the provider's base model to fine-tune and is forwarded verbatim; it is not a gateway model alias. The returned job object also names a provider base model rather than a gateway alias: the gateway does not translate this field in either direction, so the job object reports whatever the provider records for the model you submitted:

curl -sS -X POST "${AISIX_PROXY}/v1/fine_tuning/jobs" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini-2024-07-18",
    "training_file": "aisix-…"
  }'

Usage and Cost Attribution

File and job management calls, such as upload, create, list, and cancel, record zero-token usage events for visibility in logs.

When a batch retrieve first observes status: "completed", the gateway downloads the batch output file and aggregates per-line token usage by provider-billed model. AISIX then emits usage events with real token counts. These events carry deterministic request IDs for each batch and model slice, so repeated attribution after a gateway restart retains the same identities.

Gateway Policy Behavior

Caller API key authentication and model access lists, per-model client IP restrictions, and rate limits apply to every route in this family. Matching AISIX Cloud budgets also apply.

Guardrail coverage differs. Input and output scans run on /v1/batches and /v1/fine_tuning/jobs, over the JSON body the caller submits and the JSON the provider returns, not over the file a job references. The five Files routes run no guardrail check in either direction. Uploaded and downloaded files are relayed unscreened, and this route family never screens the JSONL records that a batch job processes. See The Files Routes Are Not Screened.

Upstream calls honor the model's request timeout. When cooldown and its transport-error trigger are enabled for the model, connection, request-timeout, and response-read failures can take it out of rotation. Upstream HTTP status responses on these routes do not trigger cooldown.

Next Steps

You have now run file uploads, batches, and fine-tuning jobs through the gateway. Next, review Passthrough Routes for provider-native endpoints outside this surface.