Realtime API

An AISIX gateway relays the OpenAI Realtime API over WebSocket at GET /v1/realtime. Before accepting the connection, the gateway authenticates the client, resolves the model alias, and enforces access and rate-limit policy. It then relays events between the client and the provider in both directions.

In this guide, you will connect a Realtime client through the gateway and review the session behavior that matters for this endpoint.

Prerequisites

Before starting, prepare the following:

• A running AISIX gateway that can serve proxy requests.
• A caller API key that can access the model alias.
• A model alias backed by a provider that serves the OpenAI Realtime protocol.

Export the gateway connection and request values for the server-side example:

# AISIX_PROXY uses http or https and has no trailing slash or endpoint path.
# The local quickstarts use http://127.0.0.1:3000.
export AISIX_PROXY="YOUR_AISIX_GATEWAY_URL"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="realtime-prod"

Provider support covers OpenAI-compatible providers (adapter openai, including custom api_base deployments) and Azure OpenAI (adapter azure-openai, /openai/realtime with api-key authentication). Gemini Live and Bedrock use different realtime event protocols and are not served by this endpoint.

Connect a Client

Select the model with the model query parameter. Server-side clients authenticate with the standard headers:

import WebSocket from "ws";

const realtimeUrl = new URL("/v1/realtime", process.env.AISIX_PROXY);
realtimeUrl.protocol = realtimeUrl.protocol === "https:" ? "wss:" : "ws:";
realtimeUrl.searchParams.set("model", process.env.AISIX_MODEL);

const ws = new WebSocket(realtimeUrl, {
  headers: { Authorization: `Bearer ${process.env.AISIX_API_KEY}` },
});

Browser clients cannot set WebSocket headers. Pass the caller API key as a subprotocol item instead, matching the flow used by OpenAI browser examples. The gateway echoes the realtime subprotocol back:

// Use an absolute HTTP(S) gateway URL with no endpoint path, such as https://gateway.example.com
const aisixProxy = "YOUR_AISIX_GATEWAY_URL";
const aisixApiKey = "YOUR_CALLER_API_KEY";
const aisixModel = "realtime-prod";

const realtimeUrl = new URL("/v1/realtime", aisixProxy);
realtimeUrl.protocol = realtimeUrl.protocol === "https:" ? "wss:" : "ws:";
realtimeUrl.searchParams.set("model", aisixModel);

const ws = new WebSocket(realtimeUrl, [
  "realtime",
  `openai-insecure-api-key.${aisixApiKey}`,
]);

After the connection is established, send and receive Realtime events exactly as you would against the provider directly. AISIX relays events such as session.update, audio buffers, response.create, and server events without changing their shape.

The one value AISIX rewrites is the model name inside a session object. The session.created and session.updated events name the model alias the client connected with, not the provider's own model ID. A session.update that sends that alias back is translated to the provider's model ID on the way upstream, so a client can echo the session object it was given.

Authentication and Policy

Authentication, model access checks, client IP restrictions, and rate limits run before the WebSocket upgrade completes. Budget checks run at the same point. A request that fails any of these checks is rejected at the HTTP handshake (401, 403, or 429), which clients observe as a failed connection attempt.

During the session, configured guardrails scan text events in both directions. A blocked event produces an OpenAI-shaped error event followed by connection close.

Usage Tracking

The gateway harvests usage from the provider's response.done events (and transcription-completed events for transcription sessions) and records one aggregated usage event per session, including cached-token counts. Total session tokens count toward token-based rate limits.

Session Limits

AISIX applies an idle cap between events in either direction. It resolves the cap from the direct model's stream_timeout, then its timeout, and then the deployment-wide upstream.stream_timeout_ms or upstream.timeout_ms defaults. The default deployment cap is 6000 seconds. A silent session past the resolved deadline closes with code 1001 and reason idle timeout.

To opt a model out of the deployment backstop, set timeout: 0 and make sure the model does not set a nonzero stream_timeout. Deployment operators can instead set both upstream timeout defaults to 0. See How the Timeouts Relate for the complete precedence rules. Upstream connection failures close the session with code 1011, and count toward the model's cooldown when the model enables it. Opening that upstream connection has its own budget: upstream.connect_timeout_ms (5 seconds by default) bounds the whole dial, the WebSocket upgrade handshake included, so an upstream that never answers fails there instead of holding the session open.

Connection Close

AISIX completes the WebSocket close handshake in both directions:

• When the client closes the session, the client receives its own close code and reason back, and the upstream receives the same code and reason.
• When the upstream closes the session, the client receives the upstream's close code and reason. The reason is relayed as the upstream sent it, cut to 123 bytes.
• A close code that cannot be sent in a Close frame, such as a missing status code or 1006, reaches the other side as 1011. So does a Close frame of exactly 1002 with the reason Protocol violation, because AISIX cannot tell it apart from a frame its WebSocket library substituted for an invalid code. A 1002 with any other reason is relayed.
• Any end other than a normal close (1000) is preceded by a Realtime error event of type server_error that says what happened: the upstream's close code and reason, a close frame AISIX could not relay, a close without a status code, a failed upstream connection and its cause, or a connection that ended without a close frame.

Usage that arrives while the close completes is still counted. The access-log status of the session does not depend on the close code.

When the gateway shuts down, it waits for open Realtime sessions to close before it exits, as it does for streamed responses, so their usage is recorded. Size the gateway's termination grace period for the longest session you expect.

Reach the Upstream Through a Proxy

AISIX opens the upstream connection of a wss:// provider through HTTPS_PROXY, and of a ws:// provider through HTTP_PROXY, falling back to ALL_PROXY, unless NO_PROXY covers the host. The proxy must be an HTTP proxy that supports CONNECT. A SOCKS proxy is not supported: the session fails with an error event and close code 1011, and the access log records 502. A gateway at 1.6.x or earlier always connects to the upstream directly.

Next Steps

You have now connected a Realtime client through the gateway. Next, review Speech and Audio for the non-Realtime audio endpoints.