LangChain

LangChain is a framework for building LLM applications, including chains, agents, retrieval workflows, and tool-calling flows. Many LangChain applications use provider SDKs through integration packages, so the gateway integration point is usually the model client rather than the rest of the application.

The LangChain OpenAI package provides ChatOpenAI, a chat model client that can call a custom base URL. Use that setting when a LangChain application should send OpenAI-compatible chat requests through AISIX.

This guide uses LangChain Python with langchain-openai. You will configure ChatOpenAI with an AISIX proxy URL, caller API key, and model alias.

Prerequisites

Before starting, prepare the following:

• A Python environment supported by LangChain.
• A running AISIX gateway with the proxy listener available.
• An AISIX caller API key.
• A model alias the caller API key can access through the OpenAI-Compatible API.

Configure LangChain

Install the LangChain OpenAI integration if the application does not already include it:

pip install langchain-openai

Set the values that the LangChain application will use:

# Replace with your values
export AISIX_BASE_URL="http://127.0.0.1:3000/v1"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="gpt-4o-prod"

Create a ChatOpenAI client with the AISIX values:

import os

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
  model=os.environ["AISIX_MODEL"],
  api_key=os.environ["AISIX_API_KEY"],
  base_url=os.environ["AISIX_BASE_URL"],
)

response = llm.invoke("Write one sentence about why gateways help AI teams.")
print(response.content)

The model value is the AISIX model alias, not the upstream provider model ID. AISIX governs the model request path: it authenticates the caller API key, resolves the alias, applies configured policies, records telemetry, and dispatches the request to the provider behind the alias. The LangChain application still owns chains, prompts, tools, retrieval, and response handling.

Verify the Route

Run the script from the shell where the AISIX environment variables are set.

When the request succeeds, verify the following results:

• The script prints a chat response.
• AISIX records a successful POST /v1/chat/completions request for the selected model alias.

If you use AISIX Cloud or an on-premises managed control plane, verify the request in the managed gateway logs. For self-hosted gateways, use your configured logs, metrics, or upstream provider logs.

Handle Compatibility Issues

If a request fails, first confirm that the caller API key can access the selected model alias and that base_url points to the AISIX proxy API root with /v1.

LangChain may send optional OpenAI fields when you enable streaming, tool calling, or structured output. Validate the exact chain or agent workflow before relying on gateway policy or telemetry for that path.

Next Steps

• OpenAI-Compatible API: review gateway-facing request behavior.
• Tool Calling: confirm tool-call behavior before enabling LangChain tools or agents.
• Metrics and Logs: confirm gateway-side request metrics and logs.