Skip to main content
API7.ai Logo

LiveKit Agents

LiveKit Agents is a framework for building realtime voice and media agents. It connects an agent's session logic with audio or video transport, speech recognition, language models, speech synthesis, turn detection, and tools. Together, these components allow the application to respond during a live interaction.

The framework keeps those components modular. A LiveKit application can continue using its existing room or transport and speech services while sending only the text language-model step through AISIX. This gives the application AISIX model aliases, access control, routing, and telemetry without moving the media session into the gateway.

The integration uses LiveKit's openai.LLM client with an AISIX proxy URL, caller API key, and model alias. It does not use the OpenAI Realtime model plugin; LiveKit continues to coordinate the audio pipeline around the text LLM.

Prerequisites​

Before starting, prepare the following:

  • A Python project using LiveKit Agents.
  • A running AISIX gateway the LiveKit worker can reach.
  • An AISIX caller API key.
  • A model alias the caller key can access through the OpenAI-Compatible API.

The worker also needs its existing LiveKit, speech-to-text, and text-to-speech configuration. Those credentials remain separate from the AISIX caller key.

Configure the LLM Plugin​

Install the LiveKit OpenAI plugin if the project does not already include it:

pip install "livekit-agents[openai]~=1.5"

Set the gateway values in the worker environment:

# AISIX_BASE_URL includes /v1 and has no trailing slash.
export AISIX_BASE_URL="https://gateway.example.com/v1"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="voice-agent-prod"

Create the LLM plugin with those values:

import os

from livekit.plugins import openai

llm = openai.LLM(
  api_key=os.environ["AISIX_API_KEY"],
  base_url=os.environ["AISIX_BASE_URL"],
  model=os.environ["AISIX_MODEL"],
)

Pass llm to the application's AgentSession. Keep the existing transport, STT, TTS, and turn-detection components unchanged.

Test the LLM Connection​

Before joining a LiveKit room, test the same plugin as a standalone streaming client:

import asyncio
import os

from livekit.agents import ChatContext
from livekit.plugins import openai


async def main():
  context = ChatContext()
  context.add_message(
    role="user",
    content="Reply with one short sentence about voice gateways.",
  )

  llm = openai.LLM(
    api_key=os.environ["AISIX_API_KEY"],
    base_url=os.environ["AISIX_BASE_URL"],
    model=os.environ["AISIX_MODEL"],
  )

  stream = llm.chat(chat_ctx=context)
  async for text in stream.to_str_iterable():
    print(text, end="", flush=True)

  await llm.aclose()


asyncio.run(main())

The script should print the streamed model response. AISIX should record POST /v1/chat/completions for the selected caller key and alias.

After this test passes, run the normal LiveKit worker and verify one short voice turn. If text generation succeeds but no audio is returned to the participant, troubleshoot the LiveKit TTS and room pipeline rather than changing the AISIX model endpoint.

Understand the API Choice​

LiveKit documents the Responses API plugin as its preferred path for direct OpenAI usage. This guide uses openai.LLM because LiveKit assigns that client to OpenAI-compatible Chat Completions endpoints, which is the broadly compatible AISIX integration path.

Do not point the LiveKit OpenAI Realtime plugin at AISIX unless the application specifically uses the gateway's Realtime API and has verified the complete audio event protocol.

Next Steps​