Skip to main content
API7.ai Logo

LangChain and LangGraph

LangChain is a framework for building LLM applications, including chains, agents, retrieval workflows, and tool-calling flows. LangGraph builds on the LangChain ecosystem for stateful agents and graph-based workflows. In both cases, the gateway integration point is usually the model client rather than the rest of the application.

The LangChain OpenAI package provides ChatOpenAI, a chat model client that can call a custom base URL and opt in to the OpenAI Responses API. Use those settings when a LangChain application should send Responses API requests through AISIX.

This guide uses LangChain Python with langchain-openai. You will configure ChatOpenAI with an AISIX proxy URL, caller API key, model alias, and use_responses_api=True. LangGraph applications can reuse the same ChatOpenAI client inside graph nodes or agent workflows.

Prerequisites​

Before starting, prepare the following:

  • A Python environment supported by LangChain.
  • A running AISIX gateway with the proxy listener available.
  • An AISIX caller API key.
  • A model alias the caller API key can access through the Responses API.

If AISIX is already deployed for your organization, obtain the gateway URL, model alias, and caller API key from the team that manages it. Otherwise, follow the Open-Source AISIX Gateway Quickstart or AISIX Cloud Quickstart, or contact API7 for Hybrid Cloud access.

Configure LangChain​

Install the LangChain OpenAI integration if the application does not already include it:

pip install langchain-openai langgraph

Set the values that the LangChain application will use:

# Replace with your values
export AISIX_BASE_URL="http://127.0.0.1:3000/v1"
export AISIX_API_KEY="YOUR_CALLER_API_KEY"
export AISIX_MODEL="gpt-4o-prod"

Create a ChatOpenAI client with the AISIX values:

import os

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
  model=os.environ["AISIX_MODEL"],
  api_key=os.environ["AISIX_API_KEY"],
  base_url=os.environ["AISIX_BASE_URL"],
  use_responses_api=True,
)

response = llm.invoke("Write one sentence about why gateways help AI teams.")
print(response.content)

The model value is the AISIX model alias, not the upstream provider model ID. AISIX governs the model request path: it authenticates the caller API key, resolves the alias, applies configured policies, records telemetry, and dispatches the request to the provider behind the alias. The LangChain application still owns chains, prompts, tools, retrieval, and response handling.

Use the Client in LangGraph​

Pass the same ChatOpenAI client into LangGraph nodes or graph state transitions:

import os

from langchain_openai import ChatOpenAI
from typing_extensions import TypedDict

from langgraph.graph import END, START, StateGraph

llm = ChatOpenAI(
  model=os.environ["AISIX_MODEL"],
  api_key=os.environ["AISIX_API_KEY"],
  base_url=os.environ["AISIX_BASE_URL"],
  use_responses_api=True,
)


class State(TypedDict):
  prompt: str
  answer: str


def call_model(state: State):
  response = llm.invoke(state["prompt"])
  return {"answer": response.content}


graph = StateGraph(State)
graph.add_node("call_model", call_model)
graph.add_edge(START, "call_model")
graph.add_edge("call_model", END)

app = graph.compile()
result = app.invoke({
  "prompt": "Write one sentence about governed agents.",
})

print(result["answer"])

LangGraph owns state management, graph execution, retries, and tool orchestration. AISIX only sees the model requests emitted by the configured ChatOpenAI client.

Verify the Integration​

Run the script from the shell where the AISIX environment variables are set.

When the request succeeds, verify the following results:

  • The script prints a chat response.
  • AISIX records a successful POST /v1/responses request for the selected model alias.

If you use the AISIX Cloud control plane, verify the request in the AISIX gateway logs. For standalone gateways, use your configured logs, metrics, or upstream provider logs.

If the request fails, first confirm that the caller API key can access the selected model alias and that base_url points to the AISIX proxy API root with /v1.

LangChain can also use Chat Completions through the same ChatOpenAI class. Use that path only when an existing chain or dependency requires Chat Completions compatibility; for new OpenAI-family LangChain integrations, prefer the Responses API path shown here.

LangChain may send optional OpenAI fields when you enable streaming, tool calling, structured output, built-in tools, or reasoning options. Validate the exact chain or agent workflow before relying on gateway policy or telemetry for that path.

Next Steps​