Pipecat

Pipecat is an open-source Python framework for realtime voice and multimodal agents. It represents a conversation as a pipeline of frames and services. Applications can compose transport, speech recognition, context management, language-model inference, speech synthesis, and other processors around their own interaction logic.

Because the services are modular, an existing Pipecat pipeline can route its text language-model requests through AISIX without changing the transport or speech components. AISIX then provides the model alias, access controls, routing, and telemetry for that stage, while Pipecat continues to move audio, transcripts, model text, and synthesized speech through the application pipeline.

Pipecat's OpenAILLMService accepts a custom OpenAI base URL and API key. Set them to the AISIX proxy API root and caller key, and use an AISIX model alias in the service settings.

Prerequisites

Before starting, prepare the following:

• A Python project using Pipecat.
• A running AISIX gateway the Pipecat process can reach.
• An AISIX caller API key.
• A model alias the caller key can access through the OpenAI-Compatible API.

The pipeline also needs its normal transport, speech-to-text, and text-to-speech services. Their credentials are not replaced by the AISIX caller key.

Configure OpenAILLMService

Install Pipecat's OpenAI integration if it is not already available:

pip install "pipecat-ai[openai]"

Set the gateway values:

# AISIX_BASE_URL includes /v1 and has no trailing slash.
export AISIX_BASE_URL="https://gateway.example.com/v1"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="voice-agent-prod"

Create the LLM service with the current settings-based configuration:

import os

from pipecat.services.openai.llm import OpenAILLMService

llm = OpenAILLMService(
  api_key=os.environ["AISIX_API_KEY"],
  base_url=os.environ["AISIX_BASE_URL"],
  settings=OpenAILLMService.Settings(
    model=os.environ["AISIX_MODEL"],
  ),
)

Place llm in the existing Pipecat pipeline between the user context aggregator and the TTS service. The model setting uses the AISIX alias, not the upstream provider's model ID.

Test the LLM Connection

Test streaming before starting the complete media pipeline:

import asyncio
import os

from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.services.openai.llm import OpenAILLMService


async def main():
  llm = OpenAILLMService(
    api_key=os.environ["AISIX_API_KEY"],
    base_url=os.environ["AISIX_BASE_URL"],
    settings=OpenAILLMService.Settings(
      model=os.environ["AISIX_MODEL"],
    ),
  )

  context = LLMContext(
    messages=[
      {
        "role": "user",
        "content": "Reply with one short sentence about voice gateways.",
      }
    ]
  )

  stream = await llm.get_chat_completions(context)
  async for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
      print(chunk.choices[0].delta.content, end="", flush=True)

  await llm.cleanup()


asyncio.run(main())

The script should print the streamed response. AISIX should record POST /v1/chat/completions for the configured alias.

The final cleanup line runs the service's public cleanup hook for this standalone smoke test. In a normal Pipecat application, the pipeline lifecycle manages service startup and cleanup.

Verify the Voice Pipeline

After the LLM test passes, start the normal Pipecat application and complete one voice turn. Use Pipecat metrics to separate speech recognition, LLM, and speech synthesis latency, and compare the LLM segment with AISIX request metrics.

If the standalone LLM test succeeds but the full pipeline does not speak, inspect the context aggregator, TTS service, and output transport. AISIX returns model text and tool calls; it does not create Pipecat audio frames.

Next Steps

• Streaming: review the gateway stream path.
• Tool Calling: validate Pipecat function handlers with AISIX model output.
• Metrics and Logs: compare gateway latency with Pipecat service metrics.