Skip to main content
API7.ai Logo

ElevenLabs Agents

ElevenLabs Agents is a hosted platform for building and deploying conversational voice agents. It coordinates speech recognition, voice synthesis, turn handling, tools, and conversation delivery. The agent can listen, decide what to do, and respond without the application assembling each media component itself.

Organizations may still want the agent's language-model traffic to use the same access controls, routing policy, and telemetry as their other AI applications. ElevenLabs supports a Custom LLM for this purpose. The agent continues to run its voice session in ElevenLabs while its OpenAI-compatible Chat Completions requests pass through AISIX to a configured model provider.

In this integration, ElevenLabs stores a dedicated AISIX caller API key as a secret, sends an AISIX model alias in model, and streams requests to the gateway's /v1/chat/completions endpoint. AISIX governs the model request and response; ElevenLabs continues to handle the audio and conversation experience.

Prerequisites​

Before starting, prepare the following:

  • An ElevenLabs account with access to ElevenLabs Agents.
  • An ElevenLabs agent you can edit.
  • A publicly reachable HTTPS AISIX proxy URL.
  • An AISIX caller API key dedicated to the agent.
  • A model alias the caller key can access through the OpenAI-Compatible API.

Do not store an upstream provider key in ElevenLabs. The Custom LLM secret should contain only the AISIX caller key.

Configure the Custom LLM​

Open the agent in the ElevenLabs dashboard, then configure the model:

  1. On the Agent page, open LLM.
  2. Open the primary model selector and choose Custom LLM.
  3. Keep Chat Completions as the API format.
  4. Set Server URL to the AISIX proxy API root, including /v1, for example https://gateway.example.com/v1. ElevenLabs appends /chat/completions.
  5. Set Model ID to the AISIX model alias, for example voice-agent-prod.
  6. Under API Key, create or select a secret containing the AISIX caller key.
  7. Select Test Connection.

The connection test should receive a successful OpenAI-compatible stream. If it fails, confirm the URL is reachable from the public internet and that the caller key can access the alias.

Close the LLM settings, then publish the agent when the connection test succeeds.

Verify a Conversation​

Open Preview and start one short conversation. Speak a simple request, such as “Reply with one short sentence.”

Confirm these results:

  • The agent returns a spoken response.
  • AISIX records a successful POST /v1/chat/completions request.
  • The recorded caller identity and model match the dedicated ElevenLabs key and AISIX alias.

ElevenLabs handles the speech-to-text result, generated voice, recording settings, and conversation transcript. AISIX sees the OpenAI-compatible model request and response text, not the original audio stream.

Use Tools Carefully​

ElevenLabs system tools and custom tools require the Custom LLM to produce compatible function calls. Before enabling a tool in production, verify that the selected model alias returns the tool name and arguments through AISIX and that ElevenLabs executes the tool as expected.

A successful plain-text connection test does not prove tool compatibility. Test every system or custom tool the agent uses, including any multi-tool turn, before production.

Troubleshoot the Connection​

SymptomCheck
Server URL is invalidEnter the HTTPS AISIX base through /v1, without adding /chat/completions; the dashboard adds that suffix.
Connection test returns 401Confirm the selected ElevenLabs secret contains the AISIX caller key.
Connection test returns 403Confirm the caller key allows the selected model alias.
Connection test returns 404Confirm the model ID is an AISIX model alias and the AISIX proxy route is reachable.
The agent waits without speakingCheck the AISIX stream and upstream latency, then verify the ElevenLabs voice and turn settings.

Next Steps​