Skip to main content
API7.ai Logo

Realtime API

AISIX AI Gateway relays the OpenAI Realtime API over WebSocket at GET /v1/realtime. Before accepting the connection, the gateway authenticates the client, resolves the model alias, and enforces access and rate-limit policy. It then relays events between the client and the provider in both directions.

In this guide, you will connect a Realtime client through the gateway and review the session behavior that matters for this endpoint.

Prerequisites​

Before starting, prepare the following:

  • A running AISIX gateway that can serve proxy requests.
  • A caller API key that can access the model alias.
  • A model alias backed by a provider that serves the OpenAI Realtime protocol.

Provider support covers OpenAI-compatible providers (adapter openai, including custom api_base deployments) and Azure OpenAI (adapter azure-openai, /openai/realtime with api-key authentication). Gemini Live and Bedrock use different realtime event protocols and are not served by this endpoint.

Connect a Client​

Select the model with the model query parameter. Server-side clients authenticate with the standard headers:

import WebSocket from "ws";

const ws = new WebSocket(
  "ws://127.0.0.1:3000/v1/realtime?model=realtime-prod",
  { headers: { Authorization: "Bearer YOUR_CALLER_API_KEY" } },
);

Browser clients cannot set WebSocket headers. Pass the caller API key as a subprotocol item instead, matching the flow used by OpenAI browser examples. The gateway echoes the realtime subprotocol back:

const ws = new WebSocket(
  "wss://gateway.example.com/v1/realtime?model=realtime-prod",
  [
    "realtime",
    "openai-insecure-api-key.YOUR_CALLER_API_KEY",
  ],
);

After the connection is established, send and receive Realtime events exactly as you would against the provider directly. AISIX relays events such as session.update, audio buffers, response.create, and server events without changing their shape.

Authentication and Policy​

Authentication, model access checks, client IP restrictions, budgets, and rate limits run before the WebSocket upgrade completes. A request that fails any of them is rejected at the HTTP handshake (401, 403, or 429), which clients observe as a failed connection attempt.

During the session, configured guardrails scan text events in both directions. A blocked event produces an OpenAI-shaped error event followed by connection close.

Usage Tracking​

The gateway harvests usage from the provider's response.done events (and transcription-completed events for transcription sessions) and records one aggregated usage event per session, including cached-token counts. Total session tokens count toward token-based rate limits.

Session Limits​

If the model configures stream_timeout, it acts as the idle cap between events; a silent session past that deadline closes with a timeout. Without it, sessions have no gateway-imposed duration limit. Upstream connection failures close the session with code 1011 and count toward the model's cooldown.

Next Steps​

You have now connected a Realtime client through the gateway. Next, review Speech and Audio for the non-Realtime audio endpoints.