Embeddings

Embeddings convert text into vectors that applications can use for semantic search, retrieval pipelines, clustering, and similarity checks. An AISIX gateway lets embedding clients keep the OpenAI-compatible request and response format while the gateway manages caller authentication, model aliases, upstream credentials, and policy.

In this guide, you will send an embeddings request through AISIX and review the provider behavior that matters for this endpoint.

Prerequisites

Before starting, prepare the following:

• A running AISIX gateway that can serve proxy requests.
• A caller API key that can access the model alias.
• A model alias backed by a provider and model that support embeddings.

Export the gateway connection and request values:

# AISIX_PROXY has no trailing slash or endpoint path such as /v1.
# The local quickstarts use http://127.0.0.1:3000.
export AISIX_PROXY="YOUR_AISIX_GATEWAY_URL"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="text-embedding-prod"

Send an Embeddings Request

Send the embeddings request through the gateway proxy with the AISIX model alias in the request body:

curl -sS -X POST "${AISIX_PROXY}/v1/embeddings" \
  -H "Authorization: Bearer ${AISIX_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "'"${AISIX_MODEL}"'",
    "input": [
      "hello",
      "world"
    ]
  }'

AISIX resolves the model alias, checks the caller API key, rewrites the upstream model ID, and forwards the embeddings request to the provider.

The response keeps the OpenAI-compatible embeddings format:

{
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "index": 0,
      "embedding": [0.0123, -0.0456, 0.0789]
    },
    {
      "object": "embedding",
      "index": 1,
      "embedding": [0.0234, -0.0567, 0.0891]
    }
  ],
  "model": "text-embedding-prod",
  "usage": {
    "prompt_tokens": 2,
    "total_tokens": 2
  }
}

The embedding value may be a float array or a base64 string, depending on the requested encoding format and upstream response.

Provider and Request Behavior

The embeddings route accepts the OpenAI-compatible request shape and translates it to the provider's native embeddings API where needed:

Provider family	Upstream API	Notes
OpenAI-compatible (adapter openai)	POST {api_base}/embeddings	Forwarded in OpenAI shape, including custom api_base deployments, with user and every other field the caller sent.
Vertex AI / Gemini (adapter vertex)	google-publisher :predict	dimensions maps to outputDimensionality; per-input token counts sum into usage.
Bedrock (adapter bedrock)	InvokeModel	amazon.titan-embed-* (one call per input; dimensions on Text V2 models, and as embeddingConfig.outputEmbeddingLength on Titan Multimodal Embeddings) and cohere.embed-* (whole batch in one call). See Cohere on Bedrock.
Anthropic	No embeddings API	Requests return 501.

AISIX accepts embedding input as a single string, an array of strings, an array of token IDs such as [1, 2], or an array of token-ID arrays such as [[1, 2], [3]]. It preserves the caller's input shape when it forwards the request upstream. Callers do not need separate client-side logic to switch between a single input and a batch input.

Token-ID input is forwarded to OpenAI-compatible upstreams. The Bedrock and Vertex AI embedding models take text only, so AISIX refuses token-ID input for them with 400. When an upstream reports no usage for a token-ID request, AISIX counts the token IDs. A gateway at 1.7.0 or earlier refuses token-ID input on every upstream.

Input guardrails can inspect text inputs before AISIX calls the provider. Token IDs carry no text, so an input guardrail has nothing to scan in them and lets a token-ID request through. Do not rely on an input guardrail to screen embeddings callers that send token IDs.

The gateway records usage when the upstream returns token usage. Embeddings do not use completion tokens, response caching, streaming, or output guardrails on this proxy path.

Cohere on Bedrock

A cohere.embed-* model on Bedrock takes three Cohere request fields, which /v1/embeddings accepts in the request body:

Field	Sent to Cohere as
input_type	input_type, such as search_query or search_document. Defaults to search_document.
embedding_types	embedding_types. Without it, encoding_format names the type; a base64 encoding is served from float embeddings.
dimensions	output_dimension, on Embed v4 models only. Earlier models have a fixed vector length.

AISIX reads both Cohere response shapes, the float list and the Embed v4 by-type object, and returns the float vectors when several types come back. Usage comes from Cohere's meta.billed_units.input_tokens when the response carries it; otherwise the token count is unavailable.

Troubleshoot Embeddings

If AISIX returns 501, the resolved provider has no embeddings API (for example, Anthropic). Use a model backed by a provider family from the table above.

For batch requests, the response should return one embedding entry per input item. If fewer vectors come back, inspect the upstream response and gateway logs for provider-specific batch handling.

If an input guardrail does not block the request, check whether the input contains inspectable text. AISIX can scan a single string or an array of strings, but not token IDs.

Next Steps

You have now seen how AISIX proxies embeddings requests and where provider support can differ. Next, continue with Rerank when your application needs ranking behavior for retrieved documents.