Speech and Audio

Audio routes support speech-to-text, speech translation, and text-to-speech requests. They use the same gateway authentication, model aliases, and traffic controls as the other OpenAI-compatible proxy APIs.

AISIX resolves the caller-facing model alias, applies access checks and supported text guardrails, and forwards the request to an upstream that supports the same audio route. It preserves the endpoint-specific request and response shapes instead of converting them into a chat-style format.

In this guide, you will send speech-to-text and text-to-speech requests through AISIX, then review the behavior that differs across audio endpoints.

Prerequisites

Before starting, prepare the following:

• A running AISIX gateway that can serve proxy requests.
• A caller API key that can access the model alias.
• A model alias backed by a provider and model that support the audio route you want to call.

Export the gateway connection and caller API key used by the examples:

# AISIX_PROXY has no trailing slash or endpoint path such as /v1.
# The local quickstarts use http://127.0.0.1:3000.
export AISIX_PROXY="YOUR_AISIX_GATEWAY_ORIGIN"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"

Send a Transcription Request

Transcription is a multipart/form-data upload, not a JSON body. Send the audio file in the file field and the AISIX model alias in the model field:

curl -sS -X POST "${AISIX_PROXY}/v1/audio/transcriptions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -F "file=@meeting.wav" \
  -F "model=transcribe-prod"

For a successful request, the upstream returns the transcript:

{
  "text": "The quick brown fox jumps over the lazy dog."
}

AISIX rebuilds the multipart form with the upstream model ID and preserves the remaining fields. A configured input guardrail can block or mask a text-bearing prompt field before AISIX sends the form upstream.

To translate non-English speech into English text instead, send the same form to /v1/audio/translations.

Choose a Response Format

The optional response_format field selects the transcript representation. For a successful request, AISIX preserves the upstream response body and content type unless a configured output guardrail blocks or masks the transcript. The requested representation therefore remains intact when no guardrail changes it:

response_format	Response content type	Body
json (default)	application/json	{"text": "..."}
verbose_json	application/json	Transcript plus duration, language, and per-segment timings
text	text/plain	The transcript alone
srt	text/plain	SubRip subtitles with cue timings
vtt	text/plain	WebVTT subtitles with cue timings

Handle transcription responses by content type rather than assuming JSON. Only json and verbose_json produce a JSON body:

curl -sS -X POST "${AISIX_PROXY}/v1/audio/transcriptions" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -F "file=@meeting.wav" \
  -F "model=transcribe-prod" \
  -F "response_format=srt"

Format support is a property of the upstream model, not of AISIX. If a model rejects a format, inspect the error returned by AISIX and the provider's model documentation. Do not rely on the error body being a byte-for-byte copy of the upstream response.

Buffered Transcription Streaming

Some transcription models accept stream=true and return server-sent events. AISIX currently reads the upstream audio response in full before responding, so it delivers the events together after the upstream request finishes rather than incrementally during live transcription. Unless an output guardrail changes or blocks the transcript, the events are returned in the response and AISIX can extract usage from the terminal event.

Streaming support is model-specific. For example, OpenAI's file-transcription guide uses gpt-transcribe models for streaming, while the official SDK specification states that whisper-1 ignores stream. Check the upstream model documentation before relying on streamed transcription events.

Send a Speech Request

Send a speech-generation request through the gateway proxy with the AISIX model alias in the request body:

curl -sS -X POST "${AISIX_PROXY}/v1/audio/speech" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "tts-prod",
    "input": "Hello from AISIX.",
    "voice": "alloy"
  }' \
  --output aisix-speech.mp3

For a successful request, the output file should contain the audio bytes returned by the upstream provider. Handle the response as a binary file, not as a chat-style JSON response.

Check that the file was written as audio output:

file aisix-speech.mp3

You should see output that identifies the file as audio. The exact wording depends on the operating system and the upstream response format:

aisix-speech.mp3: MPEG ADTS, layer III, v2, 160 kbps, 24 kHz, Monaural

Audio Endpoint Behavior

Audio endpoints do not all use the same request or response shape:

Endpoint	Request body	Response body
Transcriptions	Multipart form with an audio file and model alias	Upstream transcription result, in the requested response_format
Translations	Multipart form with an audio file and model alias	Upstream translation result, in the requested response_format
Speech	JSON body with model alias, text input, and voice	Binary audio bytes

For transcription and translation requests, AISIX rebuilds the multipart form with the upstream model ID before forwarding it. It preserves the other form fields, including the uploaded file name and content type when they are present.

For speech requests, AISIX rewrites the model field in the JSON body and forwards the remaining request fields to the upstream provider.

For successful requests, the gateway preserves the upstream response body and content type unless a transcript guardrail changes or blocks the output. Clients should handle transcription and translation according to the requested response_format, and speech as binary audio output.

Provider Support

Audio support depends on the resolved provider and model. AISIX does not translate audio formats across provider families.

Use these routes with upstreams that expose matching OpenAI-style audio endpoints. If the upstream does not support the requested audio route, the failure is a provider capability or base-URL issue, not a caller-authentication issue.

Usage and Guardrail Behavior

Successful audio requests are attributed in gateway usage events. Token counts are populated only when the upstream response includes recognized token usage.

Transcription and translation requests can also report the audio length. AISIX reads the duration from a supported upstream response or, when needed, measures the uploaded file. This allows AISIX Cloud to price models billed by duration rather than by tokens, including requests that use a non-JSON response_format. Set the rate in Model Pricing. Speech requests report no audio duration, so AISIX duration pricing does not apply to them.

Input guardrails can inspect, block, or mask the input text on speech requests and the optional prompt field on transcription and translation requests before AISIX calls the provider. Output guardrails can inspect, block, or mask transcript text. If an output guardrail blocks a transcript, AISIX still records the billed usage because the upstream has already processed the audio.

Uploaded audio bytes and generated speech bytes are not scanned as text.

Troubleshoot Audio Requests

If a speech request succeeds but the client expects JSON, adjust the response handling. Speech returns audio bytes.

If a transcription or translation request returns 400 from AISIX or the upstream, check the multipart form construction. The request must include a model field and the expected audio file field.

If a speech guardrail does not block a request, check the request text. Speech guardrails inspect the input text, not the generated audio bytes.

If a requested transcription format fails, confirm that the resolved provider and model support it. If streamed transcription events arrive only after the request finishes, that is the current AISIX buffering behavior; also confirm that the upstream model supports stream=true.

Next Steps

You have now seen how AISIX forwards OpenAI-style audio requests and where audio response handling differs from JSON proxy routes. Next, continue with Provider Passthrough when you need a provider-native route that AISIX does not model directly.