Skip to main content
API7.ai Logo

Realtime API

AISIX AI Gateway relays the OpenAI Realtime API over WebSocket at GET /v1/realtime. Before accepting the connection, the gateway authenticates the client, resolves the model alias, and enforces access and rate-limit policy. It then relays events between the client and the provider in both directions.

In this guide, you will connect a Realtime client through the gateway and review the session behavior that matters for this endpoint.

Prerequisites​

Before starting, prepare the following:

  • A running AISIX gateway that can serve proxy requests.
  • A caller API key that can access the model alias.
  • A model alias backed by a provider that serves the OpenAI Realtime protocol.

Export the gateway connection and request values for the server-side example:

# AISIX_PROXY uses http or https and has no trailing slash or endpoint path.
# The local quickstarts use http://127.0.0.1:3000.
export AISIX_PROXY="YOUR_AISIX_GATEWAY_ORIGIN"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="realtime-prod"

Provider support covers OpenAI-compatible providers (adapter openai, including custom api_base deployments) and Azure OpenAI (adapter azure-openai, /openai/realtime with api-key authentication). Gemini Live and Bedrock use different realtime event protocols and are not served by this endpoint.

Connect a Client​

Select the model with the model query parameter. Server-side clients authenticate with the standard headers:

import WebSocket from "ws";

const realtimeUrl = new URL("/v1/realtime", process.env.AISIX_PROXY);
realtimeUrl.protocol = realtimeUrl.protocol === "https:" ? "wss:" : "ws:";
realtimeUrl.searchParams.set("model", process.env.AISIX_MODEL);

const ws = new WebSocket(realtimeUrl, {
  headers: { Authorization: `Bearer ${process.env.AISIX_API_KEY}` },
});

Browser clients cannot set WebSocket headers. Pass the caller API key as a subprotocol item instead, matching the flow used by OpenAI browser examples. The gateway echoes the realtime subprotocol back:

const aisixProxy = "YOUR_AISIX_GATEWAY_ORIGIN";
const aisixApiKey = "YOUR_CALLER_API_KEY";
const aisixModel = "realtime-prod";

const realtimeUrl = new URL("/v1/realtime", aisixProxy);
realtimeUrl.protocol = realtimeUrl.protocol === "https:" ? "wss:" : "ws:";
realtimeUrl.searchParams.set("model", aisixModel);

const ws = new WebSocket(realtimeUrl, [
  "realtime",
  `openai-insecure-api-key.${aisixApiKey}`,
]);

After the connection is established, send and receive Realtime events exactly as you would against the provider directly. AISIX relays events such as session.update, audio buffers, response.create, and server events without changing their shape.

Authentication and Policy​

Authentication, model access checks, client IP restrictions, budgets, and rate limits run before the WebSocket upgrade completes. A request that fails any of them is rejected at the HTTP handshake (401, 403, or 429), which clients observe as a failed connection attempt.

During the session, configured guardrails scan text events in both directions. A blocked event produces an OpenAI-shaped error event followed by connection close.

Usage Tracking​

The gateway harvests usage from the provider's response.done events (and transcription-completed events for transcription sessions) and records one aggregated usage event per session, including cached-token counts. Total session tokens count toward token-based rate limits.

Session Limits​

AISIX applies an idle cap between events in either direction. It resolves the cap from the direct model's stream_timeout, then its timeout, and then the deployment-wide upstream.stream_timeout_ms or upstream.timeout_ms defaults. The default deployment cap is 6000 seconds. A silent session past the resolved deadline closes with code 1001 and reason idle timeout.

To opt a model out of the deployment backstop, set timeout: 0 and make sure the model does not set a nonzero stream_timeout. Deployment operators can instead set both upstream timeout defaults to 0. See How the Timeouts Relate for the complete precedence rules. Upstream connection failures close the session with code 1011 and count toward the model's cooldown.

Next Steps​

You have now connected a Realtime client through the gateway. Next, review Speech and Audio for the non-Realtime audio endpoints.