Tool Calling

Tool calling allows a model to ask the application to run a named function, such as looking up weather, querying a database, calling an internal service, or running business logic. The model does not run the tool. It returns a structured tool call, and the application parses the arguments, runs the function, and sends the result back so the model can continue the conversation.

AISIX AI Gateway carries these tool-calling requests through the OpenAI-compatible chat-completions path and includes targeted translation between OpenAI-style and Anthropic-style tool formats. Applications can keep provider credentials and model routing behind AISIX while preserving the tool loop their SDK or agent framework expects.

In this guide, you will send a tool definition through AISIX, review the follow-up tool loop, and choose the request path that matches your client format.

Prerequisites

Before starting, prepare the following:

• A running AISIX gateway that can serve proxy requests.
• A caller API key that can access the model alias.
• A model alias backed by a provider and model that support tool calling.

Send a Tool-Calling Request

The example below uses the OpenAI-compatible chat-completions path. It sends a function definition and asks the model to call that function:

curl -sS -X POST "http://127.0.0.1:3000/v1/chat/completions" \
  -H "Authorization: Bearer YOUR_CALLER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-prod",
    "messages": [
      {"role": "user", "content": "What is the weather in Paris? Use the tool if needed."}
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get weather for a city.",
          "parameters": {
            "type": "object",
            "properties": {
              "city": {"type": "string"}
            },
            "required": ["city"]
          }
        }
      }
    ],
    "tool_choice": {
      "type": "function",
      "function": {"name": "get_weather"}
    }
  }'

The response remains OpenAI-compatible. You should see a tool call in the assistant message:

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "tool_calls": [
          {
            "id": "call_***",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"city\":\"Paris\"}"
            }
          }
        ]
      }
    }
  ]
}

If the model returns plain text instead, first confirm that the selected upstream model supports tool calling and that the request asks for a tool call.

Continue the Tool Loop

After the model returns a tool call, the application parses the tool arguments, runs the function, and sends the result back through the same chat-completions route. Use the tool_call_id returned by the assistant message so the model can connect the tool result to the original call.

Send the tool result as a follow-up message:

curl -sS -X POST "http://127.0.0.1:3000/v1/chat/completions" \
  -H "Authorization: Bearer YOUR_CALLER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-prod",
    "messages": [
      {"role": "user", "content": "What is the weather in Paris? Use the tool if needed."},
      {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_***",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"city\":\"Paris\"}"
            }
          }
        ]
      },
      {
        "role": "tool",
        "tool_call_id": "call_***",
        "content": "Sunny, 21 C."
      }
    ]
  }'

The gateway keeps the same caller authentication and model-alias behavior for the follow-up request. Provider credentials and upstream model IDs stay inside AISIX.

Choose a Tool-Calling Path

Use the path that matches the client format your application already uses:

Client format	Proxy path	Behavior
OpenAI-compatible tools	/v1/chat/completions	Use for OpenAI SDKs, OpenAI-style agent frameworks, and applications that expect OpenAI-style tool calls and follow-up tool messages.
Anthropic-style tools	/v1/messages	Fits clients already built around Anthropic Messages. Anthropic upstreams preserve more native behavior than translated upstreams.
Provider-native tools outside modeled routes	Provider passthrough	Use only when a first-class AISIX route does not model the provider endpoint you need.

For production tool loops, prefer provider-native tool-calling paths when possible and validate the exact client, provider, model, stream mode, and tool schema you plan to run.

Review Tool-Calling Behavior

Tool-calling behavior is most predictable when the client format and upstream provider format already match. Anthropic-native upstreams preserve richer Anthropic behavior than translated upstreams, while OpenAI-compatible upstreams preserve the OpenAI-style tool loop directly.

OpenAI-style requests to Anthropic-backed models can translate function tools, tool choice, assistant tool calls, and follow-up tool messages into Anthropic Messages API structures.

Anthropic-style requests to non-Anthropic upstreams can translate top-level tools and tool choice into OpenAI-style function tools. When a non-Anthropic upstream returns tool calls, AISIX can render them back to Anthropic-style tool-use blocks.

Cross-provider translation can work for common tool definitions, tool choice, assistant tool calls, and follow-up tool results, but it does not guarantee full provider parity. Streaming tool calls can also arrive as partial argument fragments, so validate the exact stream behavior before relying on translated tool calling in production.

Next Steps

You have now seen how AISIX handles tool-calling requests and translated tool definitions. For the default OpenAI-style chat path, see OpenAI-Compatible API. For Anthropic-style requests, see Anthropic-Style Messages API.