Realtime API

AISIX AI Gateway relays the OpenAI Realtime API over WebSocket at GET /v1/realtime. The gateway authenticates the client, resolves the model alias, enforces access and rate-limit policy before the connection is accepted, and then relays events between the client and the provider in both directions.

In this guide, you will connect a Realtime client through the gateway and review the session behavior that matters for this endpoint.

Prerequisites

Before starting, prepare the following:

• A running AISIX gateway that can serve proxy requests.
• A caller API key that can access the model alias.
• A model alias backed by a provider that serves the OpenAI Realtime protocol.

Provider support covers OpenAI-compatible providers (adapter openai, including custom api_base deployments) and Azure OpenAI (adapter azure-openai, /openai/realtime with api-key authentication). Gemini Live and Bedrock use different realtime event protocols and are not served by this endpoint.

Connect a Client

Select the model with the model query parameter. Server-side clients authenticate with the standard headers:

import WebSocket from "ws";

const ws = new WebSocket(
  "ws://127.0.0.1:3000/v1/realtime?model=realtime-prod",
  { headers: { Authorization: "Bearer YOUR_CALLER_API_KEY" } },
);

Browser clients cannot set WebSocket headers. Pass the caller API key as a subprotocol item instead — the same flow the OpenAI browser examples use — and the gateway echoes the realtime subprotocol back:

const ws = new WebSocket(
  "wss://gateway.example.com/v1/realtime?model=realtime-prod",
  [
    "realtime",
    "openai-insecure-api-key.YOUR_CALLER_API_KEY",
  ],
);

Once connected, send and receive Realtime events exactly as you would against the provider directly — session.update, audio buffers, response.create, and the server event stream relay unchanged.

Authentication and Policy

Authentication, model access checks, client IP restrictions, budgets, and rate limits run before the WebSocket upgrade completes. A request that fails any of them is rejected at the HTTP handshake (401, 403, or 429), which clients observe as a failed connection attempt.

During the session, configured guardrails scan text events in both directions. A blocked event produces an OpenAI-shaped error event followed by connection close.

Usage Tracking

The gateway harvests usage from the provider's response.done events (and transcription-completed events for transcription sessions) and records one aggregated usage event per session, including cached-token counts. Total session tokens count toward token-based rate limits.

Session Limits

If the model configures stream_timeout, it acts as the idle cap between events; a silent session past that deadline closes with a timeout. Without it, sessions have no gateway-imposed duration limit. Upstream connection failures close the session with code 1011 and count toward the model's cooldown.

Next Steps

You have now connected a Realtime client through the gateway. Next, review Speech and Audio for the non-realtime audio endpoints.