# ElevenLabs Agents

[ElevenLabs Agents](https://elevenlabs.io/docs/eleven-agents/overview) is a hosted platform for building and deploying conversational voice agents. It coordinates speech recognition, voice synthesis, turn handling, tools, and conversation delivery. The agent can listen, decide what to do, and respond without the application assembling each media component itself.

Organizations may still want the agent's language-model traffic to use the same access controls, routing policy, and telemetry as their other AI applications. ElevenLabs supports a Custom LLM for this purpose. The agent continues to run its voice session in ElevenLabs while its OpenAI-compatible Chat Completions requests pass through AISIX to a configured model provider.

In this integration, ElevenLabs stores a dedicated AISIX caller API key as a secret, sends an AISIX model alias in `model`, and streams requests to the gateway's `/v1/chat/completions` endpoint. AISIX governs the model request and response; ElevenLabs continues to handle the audio and conversation experience.

## Prerequisites[​](#prerequisites "Direct link to Prerequisites")

Before starting, prepare the following:

* An ElevenLabs account with access to ElevenLabs Agents.
* An ElevenLabs agent you can edit.
* A publicly reachable HTTPS AISIX proxy URL.
* An AISIX caller API key dedicated to the agent.
* A model alias the caller key can access through the [OpenAI-Compatible API](https://docs.api7.ai/ai-gateway/endpoints/openai-compatible-chat.md).

Do not store an upstream provider key in ElevenLabs. The Custom LLM secret should contain only the AISIX caller key.

## Configure the Custom LLM[​](#configure-the-custom-llm "Direct link to Configure the Custom LLM")

Open the agent in the ElevenLabs dashboard, then configure the model:

1. On the **Agent** page, open **LLM**.
2. Open the primary model selector and choose **Custom LLM**.
3. Keep **Chat Completions** as the API format.
4. Set **Server URL** to the AISIX proxy API root, including `/v1`, for example `https://gateway.example.com/v1`. ElevenLabs appends `/chat/completions`.
5. Set **Model ID** to the AISIX model alias, for example `voice-agent-prod`.
6. Under **API Key**, create or select a secret containing the AISIX caller key.
7. Select **Test Connection**.

The connection test should receive a successful OpenAI-compatible stream. If it fails, confirm the URL is reachable from the public internet and that the caller key can access the alias.

Close the LLM settings, then publish the agent when the connection test succeeds.

## Verify a Conversation[​](#verify-a-conversation "Direct link to Verify a Conversation")

Open **Preview** and start one short conversation. Speak a simple request, such as “Reply with one short sentence.”

Confirm these results:

* The agent returns a spoken response.
* AISIX records a successful `POST /v1/chat/completions` request.
* The recorded caller identity and model match the dedicated ElevenLabs key and AISIX alias.

ElevenLabs handles the speech-to-text result, generated voice, recording settings, and conversation transcript. AISIX sees the OpenAI-compatible model request and response text, not the original audio stream.

## Use Tools Carefully[​](#use-tools-carefully "Direct link to Use Tools Carefully")

ElevenLabs system tools and custom tools require the Custom LLM to produce compatible function calls. Before enabling a tool in production, verify that the selected model alias returns the tool name and arguments through AISIX and that ElevenLabs executes the tool as expected.

A successful plain-text connection test does not prove tool compatibility. Test every system or custom tool the agent uses, including any multi-tool turn, before production.

## Troubleshoot the Connection[​](#troubleshoot-the-connection "Direct link to Troubleshoot the Connection")

| Symptom                          | Check                                                                                                         |
| -------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| Server URL is invalid            | Enter the HTTPS AISIX base through `/v1`, without adding `/chat/completions`; the dashboard adds that suffix. |
| Connection test returns `401`    | Confirm the selected ElevenLabs secret contains the AISIX caller key.                                         |
| Connection test returns `403`    | Confirm the caller key allows the selected model alias.                                                       |
| Connection test returns `404`    | Confirm the model ID is an AISIX model alias and the AISIX proxy route is reachable.                          |
| The agent waits without speaking | Check the AISIX stream and upstream latency, then verify the ElevenLabs voice and turn settings.              |

## Next Steps[​](#next-steps "Direct link to Next Steps")

* [Streaming](https://docs.api7.ai/ai-gateway/endpoints/streaming.md): review gateway stream behavior.
* [Tool Calling](https://docs.api7.ai/ai-gateway/endpoints/tool-calling.md): verify function calls through AISIX.
* [Metrics and Usage Events](https://docs.api7.ai/ai-gateway/observability/metrics-and-usage-events.md): monitor model latency and request outcomes.
